Three OSRS environments, and the first real model scores

grand-exchange, bot-detection and drop-table-inference. All three produce
unbounded or irreducible-error scores, unlike the first four, so all three
normalise against a REFERENCE STRATEGY rather than an absolute -- which is what
makes 1.000 reachable rather than aspirational.

drop-table-inference is the clearest case. Scoring KL against the true drop
table would put the ceiling out of reach, because sampling error is irreducible:
ground truth scores 0.900 against the Bayesian posterior's 1.000, and that
ordering is correct. The best estimate available from 1,200 kills is not the
true table, and an environment that demands it is measuring luck.

grand-exchange needed the market to carry structure a model can actually infer,
or profit is noise and no oracle exists. Measured over 120 baskets: the
reference earns 38,821 gp and is profitable in 120/120, random orders lose 561,
and trading only the random-walk items -- which look like the widest-swinging
lines on the board -- loses 16,860.

bot-detection generates naive bots, cloaked bots that jitter on purpose, and
efficient humans who look bot-like on every naive statistic. Timing features are
drawn BEFORE the generator decides who is a bot, so every latency rule sits at
chance. Reference discriminator F1 1.000, random 0.430.

Each was built by one agent then attacked by two independent reviewers on
exploitability and soundness. They earned their keep: drop-table-inference's
first reward let a memorised constant score 0.901 without opening the kill log,
and its reference was 19% worse than a flat number. probe.py now carries three
named fences asserting those attacks stay dead.

First evaluation also lands. Nemotron 3.5 Lightning, thinking off, 32 rollouts
per environment: 0.570 down to 0.059, every gate at or near zero, nothing
solved and nothing unsolvable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

README.md

@@ -37,6 +37,9 @@ what caught it — see the table below, and the git history.
 | [`canary-trap`](environments/canary_trap) | Write contamination probes that catch a model trained on your eval set. | Probes are run against a model that memorised the corpus **and** one that learned the same facts from a reworded copy. GUID-recall probes are sound and see only half of what is there. |
 | [`fault-localisation`](environments/fault_localisation) | Read an incident log and name the root cause, its class, and the line proving it. | The service that broke emits one line and goes quiet; its dependents emit a dozen timeouts. Ranking by error volume answers the victim, every time. |
 | [`schema-migration`](environments/schema_migration) | Split a text column into a number and a unit, executed against held-out rows. | The visible rows are tidy. The graded ones carry thousands separators, negatives, a unit containing a slash, and a value with no unit at all. |
+| [`bot-detection`](environments/bot_detection) | Classify accounts as bot or human from activity logs, learning the rule from six labelled examples. | Click latency, session length and route count are drawn before the generator decides who is a bot, so every timing rule sits at chance. The bots that cloak leak on one behavioural channel each, and there are three. |
+| [`drop-table-inference`](environments/drop_table_inference) | Estimate a monster's drop rates from a kill log, graded on the next forty thousand kills. | Copying the observed frequencies is right about the common drops and asserts that the item which appeared zero times is impossible. Nothing in the counts distinguishes one silent item from another, so the only evidence about them is the rungs the observed items did not take and what the silent ones are worth. |
+| [`grand-exchange`](environments/grand_exchange) | Read price and volume history for a basket of items and place limit orders, executed against the next thirty ticks. | One or two items per basket are a random walk, not a mean-reverting one — the widest-swinging lines on the board and the ones with no anchor to revert to — and you are not told how many there are, so counting is not a substitute for the shape statistic. And liquidity in gp per tick is uncorrelated with price, so the fattest visible margins sit where the purse cannot go. |
 
 ## What the probes measure
 
@@ -48,6 +51,9 @@ Reward for the degenerate strategies and for an oracle, from `uv run python prob
 | `canary-trap` | 0.000 | 0.000 *(public-knowledge probes)* | 0.525 *(GUID recall)* | **1.000** |
 | `fault-localisation` | 0.000 | 0.250 *(blame the loudest)* | 0.500 | **1.000** |
 | `schema-migration` | 0.000 | 0.000 *(add columns, touch nothing)* | 0.630 | **1.000** |
+| `bot-detection` | 0.000 | 0.000 *(ban everyone)* | 0.400 *(one tell of three)* | **1.000** |
+| `drop-table-inference` | 0.000 | 0.179 *(copy the frequencies, floor the rest at a memorised constant)* | 0.476 *(per-item posterior over the grid)* | **1.000** |
+| `grand-exchange` | 0.000 | 0.082 *(buy and sell at market)* | 0.608 *(anchor on the mean and size to the volume, trap included)* | **1.000** |
 
 ## Running one
 
@@ -63,5 +69,24 @@ The layout mirrors `verifiers`' own, so an environment goes to the Hub without a
 prime env push redaction-pressure -v PUBLIC
 ```
 
-No model has been evaluated in any of these yet. That is a different sentence from having a
-leaderboard, and this file will say so until it is not true.
+## What a model actually scores
+
+First evaluation, 2026-08-19: **Nemotron 3.5 Lightning 30B-A3B** (NVFP4, thinking off) served
+on one DGX Spark. 16 tasks x 2 rollouts per environment, 32 rollouts each.
+
+| environment | reward | where it fails |
+|---|---|---|
+| `canary-trap` | 0.570 | specificity 1.00 — it never accuses a clean model. Detection 0.55: it writes GUID-recall probes and is blind to the paraphrase, which is the failure the environment was built to expose |
+| `fault-localisation` | 0.352 | fault class 0.56, evidence 0.53, **service 0.16**. In 12 of the 17 cases where it cited the correct line it still named the wrong service — the service is printed on that line |
+| `schema-migration` | 0.350 | schema 0.59, integrity 0.38. It reaches the target shape and loses the awkward rows |
+| `redaction-pressure` | 0.343 | recall 0.45, precision 0.53 |
+| `grand-exchange` | 0.219 | profit 0.33, discipline 0.18 |
+| `bot-detection` | 0.142 | caught 0.19, spared 0.19 — it accuses humans, which is the expensive error |
+| `drop-table-inference` | 0.059 | fit 0.01. It answers in fractions (`48/128`, `1/3072`), so it has the structural idea, and the rates are still wrong |
+
+**Every gate is at or near zero** — 0.00 on four of the seven. None of these is solved, and
+`probe.py` shows none is unsolvable. That gap is the whole point: an environment a model
+already passes has no gradient left in it, and one nothing can pass has none yet.
+
+The replies parse: 32/32 rollouts produced non-zero metrics in every environment, so these are
+model scores rather than a harness failing to read its own output.