Files
arena/environments/redaction_pressure

redaction-pressure v0.2

A one-turn Verifiers v1 environment. The model returns regex rules; the scanner runs them against later records from the same deterministic corpus stream.

Reward contract

  • Recall (0.35): a labelled secret counts only when every original character in its span is gone. Partial edits earn zero span recall.
  • Precision (0.35): source characters belonging to fully removed secret spans, divided by all removed source characters. Partial damage stays in the denominator.
  • Clean gate (0.30): every secret fully removed, every non-secret character and decoy preserved, and no validation or timeout errors.
  • Validity: malformed JSON, bad entries, more than 24 rules, unsafe replacements, invalid regexes, excessive matches, timeouts, or an exhausted episode budget multiply every reward by zero.

Replacement is intentionally limited to "[REDACTED]" or "". That prevents a rule from matching a complete secret and then leaking it through a capture expansion.

Splits

train and dev have separate, deterministic public seed ranges. private_eval derives seeds with HMAC from REDACTION_PRESSURE_PRIVATE_SEED, which must contain at least 32 characters. The secret is never task data; traces carry its SHA-256 commitment and reveal the derived task seed only in the completed trace so the episode can be reproduced by an authorized evaluator.

Never present public dev scores as private evaluation.

Trace identity

Each task records the environment, corpus, and reward-schema versions, split commitment, and a SHA-256 digest of the behavior-bearing installed Python sources. Scoring repeats the identity under trace.info.environment and records validation failures under trace.info.redaction_errors.

Checks

From the Arena repository root:

uv lock --project environments/redaction_pressure --check
uv sync --project environments/redaction_pressure --frozen
uv run --project environments/redaction_pressure \
  python -m unittest discover -s environments/redaction_pressure/tests -v

The committed tests demonstrate oracle 1.0 and inaction 0.0, reject the historical first-character exploit across all 64 public development tasks, exercise parser and rule validation, prove deterministic/private split behavior, and interrupt hostile regexes under the shared episode budget.