lumbridge-public/arena
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harden redaction-pressure reward and provenance

Browse Source
This commit is contained in:
karti
2026-08-19 00:52:56 -07:00
parent c28d864766
commit 9f8768acca
14 changed files with 825 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+15 -6
View File
@@ -24,18 +24,27 @@ Three rules follow, and every environment in this repository is held to them:
| | |
|---|---|
| [`redaction-pressure`](environments/redaction_pressure) | Write redaction rules for a ticket corpus. Graded on the next records the generator would have produced. Every secret class is shape-matched to a decoy, so a rule keyed on shape earns on one and pays on the other. Reward is a scanner — no judge model, no runtime, milliseconds per rollout. |
| [`redaction-pressure`](environments/redaction_pressure) | Write redaction rules for a ticket corpus. Source-character provenance makes partial edits residual secrets, while the clean gate requires character-for-character preservation of all non-secret source content. Hostile regexes share one episode budget. |
## Running one
```bash
uv run eval @ configs/redaction_pressure.toml --model <model-id>
uv run --project environments/redaction_pressure eval @ configs/redaction_pressure.toml --model <model-id>
```
Run its release gate from the repository root:
```bash
uv lock --project environments/redaction_pressure --check
uv sync --project environments/redaction_pressure --frozen
uv run --project environments/redaction_pressure \
python -m unittest discover -s environments/redaction_pressure/tests -v
```
## Why these, and not benchmark reimplementations
The Hub is filling with reimplementations, which are worth doing and are not what we have.
What Lumbridge has is a set of tools that already do verifiable work — Bench's
contamination gate, Forge's signed build chain — and those gates make good rewards
precisely because they were built to be un-arguable about a machine. `redaction-pressure`
is Forge's `redact` stage with the exit code turned into a gradient.
What Lumbridge has is a set of tools aimed at verifiable work. Those gates make good
rewards only after their ceiling, floor, adversarial cases, and replay identity are
committed beside the environment. `redaction-pressure` is the first small contract for
that path; Bench/Forge integration remains future work rather than an implied dependency.