RL environments from Lumbridge. Apache-2.0, built to the verifiers v1 spec so the same wheel can run locally or, after a separate owner-approved release, through the Prime Intellect Environments Hub unchanged.

An environment is an eval you take the gradient of. That is not a rename — it changes what the artifact has to carry. A benchmark's score is its output, so a benchmark can be a number in a table. An environment's score is the input to the next weight update, so a defect in one is inherited by the model rather than printed beside it.

Most of what is on the Hub today is a benchmark that was ported into the spec. A port cannot generate a held-out slice, because its dataset is fixed — so it is contaminated the moment anyone trains on it. And an exact-match reward is single-sided by construction, so it teaches whatever maxes the string comparison. Nothing here is a port.

The house rules

Every environment in this repository is held to three, and ships the probe that proves it. An interactive one is held to a fourth.

Graded on what the model did not see. If a held-out slice cannot be generated, the environment does not measure generalisation and does not ship.
No single-sided reward. Every reward has a counterweight in the same episode. A reward you can max by doing the crude thing is a reward that teaches the crude thing.
A zero floor and a reachable ceiling, demonstrated — per component, not per environment. Inaction scores 0.0 and an oracle scores 1.0, measured, in probe.py. An unreachable component is dead weight in the gradient; a floor above zero pays for doing nothing. The qualifier was added on 2026-08-21 and it was expensive: for a month this file printed oracle 1.000 and ok for four environments whose gate component scored exactly 0.000 mean and 0.000 max over 32 real rollouts each. The blend was true and the thing inside it was a constant. probe.py reports every weighted component's floor and ceiling now, and exits 1 on one that never moves.
A turn budget that binds. In an environment the model acts in over several turns, a policy that spends every turn must score measurably below one that spends the turns worth spending. Every reward here saturates, so without a cost for looking the budget is free and the multi-turn form measures nothing the one-shot form did not. In grand-exchange-live the gap is 0.369, stdev 0.054 over five blocks of twenty-four, worst block 0.316.

Rules 3 and 4 are not decoration. Every environment here was wrong the first time and the probe is what caught it — see the table below, and the git history.

The environments

	What it measures	The trap
`redaction-pressure`	Write provenance-safe redaction rules for a ticket corpus, run against the next records the generator would have produced.	Partial edits remain residual secrets, every non-secret source character is protected, and hostile regexes share one episode budget.
`canary-trap`	Write contamination probes that catch a model trained on your eval set.	Probes are run against a model that memorised the corpus and one that learned the same facts from a reworded copy. GUID-recall probes are sound and see only half of what is there.
`fault-localisation`	Read an incident log and name the root cause, its class, and the line proving it.	The service that broke emits one line and goes quiet; its dependents emit a dozen timeouts. Ranking by error volume answers the victim, every time.
`schema-migration`	Split a text column into a number and a unit, executed against held-out rows.	The visible rows are tidy. The graded ones carry thousands separators, negatives, a unit containing a slash, and a value with no unit at all.
`bot-detection`	Classify accounts as bot or human from activity logs, learning the rule from six labelled examples.	Click latency, session length and route count are drawn before the generator decides who is a bot, so every timing rule sits at chance. The bots that cloak leak on one behavioural channel each, and there are three.
`drop-table-inference`	Estimate a monster's drop rates from a kill log, graded on the next forty thousand kills.	Copying the observed frequencies is right about the common drops and asserts that the item which appeared zero times is impossible. Nothing in the counts distinguishes one silent item from another, so the only evidence about them is the rungs the observed items did not take and what the silent ones are worth.
`grand-exchange`	Read price and volume history for a basket of items and place limit orders, executed against the next thirty ticks.	One or two items per basket are a random walk, not a mean-reverting one — the widest-swinging lines on the board and the ones with no anchor to revert to — and you are not told how many there are, so counting is not a substitute for the shape statistic. And liquidity in gp per tick is uncorrelated with price, so the fattest visible margins sit where the purse cannot go.
`grand-exchange-live`	The same basket, traded while the window is running: eight looks over sixty ticks, with stock, offers standing between looks, and a purse that sale proceeds come back to.	Looking is not free. Re-quoting an item that already carries an offer freezes the whole book for one to four ticks, an amended offer restarts at the back of the fill queue, and the per-item buy limit is cumulative over the window rather than per offer — so the policy that re-quotes on every look scores 0.631 against a reference that looks four times and scores 1.000.

Tera world-simulation environments are maintained in the private Tera repository. Arena keeps the generic, Apache-2.0 environment work above; it does not ship current proprietary Tera source. See the source boundary.

What the probes measure

Reward for the degenerate strategies and for an oracle, from uv run python probe.py:

environment	crude maximiser	plausible attempt	oracle
`redaction-pressure`	0.387 (redact everything)	0.549	1.000
`canary-trap`	0.000 (public-knowledge probes)	0.525 (GUID recall)	1.000
`fault-localisation`	0.250 (blame the loudest)	0.500	1.000
`schema-migration`	0.000 (add columns, touch nothing)	0.641	1.000
`bot-detection`	0.000 (ban everyone)	0.400 (one tell of three)	1.000
`drop-table-inference`	0.179 (copy the frequencies, floor the rest at a memorised constant)	0.476 (per-item posterior over the grid)	1.000
`grand-exchange`	0.082 (buy and sell at market)	0.608 (anchor on the mean and size to the volume, trap included)	1.000
`grand-exchange-live`	0.126 (buy and sell at market, on one look)	0.406 (anchor on the mean, no trap filter, one look and no re-quote)	1.000

schema-migration's plausible rung moved from 0.630 to 0.641 on 2026-08-21 when its gate gained a margin — the naive split clears the loosened gate on one seed in twenty-four. Every other cell is unchanged.

And the same table one reward component at a time

The four columns above are blends, and a blend cannot show a constant inside it. Below each component's weighted floor, the best any rung below the oracle manages, and its ceiling. The interesting column is the middle one: where it equals the floor, nothing short of the oracle earns a fraction of that term.

environment	component	best below oracle	ceiling
`bot-detection`	caught	0.278	0.350
`bot-detection`	spared	0.317	0.400
`bot-detection`	gate	0.250	0.250
`canary-trap`	detection	0.350	0.350
`canary-trap`	specificity	0.350	0.350
`canary-trap`	gate	0.300	0.300
`drop-table-inference`	fit	0.242	0.450
`drop-table-inference`	rare	0.326	0.350
`drop-table-inference`	gate	0.025	0.200
`fault-localisation`	service	0.300	0.300
`fault-localisation`	fault	0.250	0.250
`fault-localisation`	evidence	0.250	0.250
`fault-localisation`	gate	0.000	0.200
`grand-exchange`	profit	0.450	0.450
`grand-exchange`	discipline	0.298	0.300
`grand-exchange`	gate	0.244	0.250
`grand-exchange-live`	profit	0.382	0.450
`grand-exchange-live`	discipline	0.201	0.300
`grand-exchange-live`	gate	0.048	0.250
`redaction-pressure`	recall	0.304	0.350
`redaction-pressure`	precision	0.350	0.350
`redaction-pressure`	gate	0.000	0.300
`schema-migration`	schema	0.300	0.300
`schema-migration`	integrity	0.422	0.450
`schema-migration`	gate	0.240	0.250

Two components are bold, and probe.py names both of them on the way past. They are not the same problem. fault-localisation's gate wants three fields correct at once and fires on 29 of 32 real rollouts — the healthiest gate here; the ladder simply has no rung standing at two-of-three. redaction-pressure's fires on 0 of 32, and unlike the two that were fixed below, no margin rescues it: see the sweep in its scanner.

The probe cannot tell those apart, which is why it warns rather than fails there. What tells them apart is tools/regate.py — it replays the real traces in outputs/ through a candidate gate and reports the fire rate, holding the model's behaviour fixed and varying only the reward.

The margins, and what they were measured against

Two gates demanded exact equality with the reference and paid nothing at all for anything short of it. Both were re-scored over the 32 real rollouts in outputs/run-20260821-1401 before being changed, and both floors were re-measured after:

environment	gate	before	after	plausible
`bot-detection`	`GATE_SLACK = 1` — the reference's bots bar one, still zero wrongful bans	0/32	5/32 = 0.156	0.000
`schema-migration`	`GATE_MARGIN = 0.15` — 34 of 40 rows recompose; schema and row count still exact	0/32	4/32 = 0.125	0.042
`redaction-pressure`	unchanged, and deliberately	0/32	0/32	0.000

The slack is on the catching clause only in both cases. A single wrongful ban still shuts bot-detection's gate outright, and a migration that leaves value_text standing or loses a row still fails schema-migration's — those are format preconditions, not matters of degree. schema-migration's 0.042 is the price of the margin, named rather than hidden: the naive parser's luckiest draw of forty awkward rows recomposes exactly 34 of them, and probe.py asserts that number stays under a tenth.

redaction-pressure was swept and left alone, which is the honest outcome rather than a gap. Every margin loose enough to fire on the measured population — half the secrets left standing — also pays a four-of-seven ruleset on five seeds in six; every margin tight enough to keep that at zero fires on none of the 32. The failure is joint rather than a threshold: recall maxed at 0.852 with never fewer than four secrets left, collateral_hits == 0 held on 6 of 32, and no rollout managed both at once. That gate is hard, not dead, and forcing a margin onto it would trade the meaning of "redacted" for nothing measurable.

Running one

uv run --project environments/redaction_pressure eval @ configs/redaction_pressure.toml --model <model-id>

Run its release gate from the repository root:

uv sync --project environments/redaction_pressure
uv run --project environments/redaction_pressure \
  python -m unittest discover -s environments/redaction_pressure/tests -v
uv run --with regex python probe.py

Arena contains independently published environment libraries, so per-environment uv.lock files are intentionally not committed. Redaction v0.2 pins its runtime contract in pyproject.toml; CI resolves it on Python 3.11 and 3.12, builds all eight public environments, and runs the auto-discovered probe for every one of them.

Publishing (deferred)

The layout mirrors verifiers' own, so an environment can eventually go to the Hub without a fork. No current environment is published there from this repository. Hub publication is an explicit owner-approved outward action; ordinary development and CI must never run it.

After that separate approval, the upstream command is:

prime env push redaction-pressure -v PUBLIC

What a model actually scores

First evaluation, 2026-08-19: Nemotron 3.5 Lightning 30B-A3B (NVFP4, thinking off) served on one DGX Spark. 16 tasks x 2 rollouts per environment, 32 rollouts each.

environment	reward	where it fails
`canary-trap`	0.570	specificity 1.00 — it never accuses a clean model. Detection 0.55: it writes GUID-recall probes and is blind to the paraphrase, which is the failure the environment was built to expose
`fault-localisation`	0.352	fault class 0.56, evidence 0.53, service 0.16. In 12 of the 17 cases where it cited the correct line it still named the wrong service — the service is printed on that line
`schema-migration`	0.350	schema 0.59, integrity 0.38. It reaches the target shape and loses the awkward rows
`redaction-pressure`	0.343	recall 0.45, precision 0.53
`grand-exchange`	0.219	profit 0.33, discipline 0.18
`bot-detection`	0.142	caught 0.19, spared 0.19 — it accuses humans, which is the expensive error
`drop-table-inference`	0.059	fit 0.01. It answers in fractions (`48/128`, `1/3072`), so it has the structural idea, and the rates are still wrong

Every gate is at or near zero — 0.00 on four of the seven. None of these is solved, and probe.py shows none is unsolvable. That gap is the whole point: an environment a model already passes has no gradient left in it, and one nothing can pass has none yet.

⚠️ Four of those zeros were 0.000 mean and 0.000 max on all 32 rollouts, which is a different claim: not a hard gate, a constant. docs/GATE_DIAGNOSIS.md is the post-mortem. Two have since been given measured margins, so re-scoring the same traces under the shipped scorers moves bot-detection from 0.3488 to 0.3879 and schema-migration from 0.4432 to 0.4745. Those two rows are stale above and will be replaced by a fresh run, not patched — and every number in that table was sampled with thinking OFF, which is its own retraction and is documented in docs/FIRST_EVAL.md.

The replies parse: 32/32 rollouts produced non-zero metrics in every environment, so these are model scores rather than a harness failing to read its own output.