Commit Graph
4 Commits
Author SHA1 Message Date
kartiandClaude Opus 5 94ab513eda probe: gate grand-exchange-live, the first interactive environment
Eight environments gated, one warning left (tera-crow-nav). The probe calls
measure_ladder.ladder() rather than re-deriving the policies, and seeds from the
skip-aware baskets() helper — a contiguous range() grades a different set of
baskets than the model is served.

Beyond the four-rung ladder it asserts what house rule 4 actually claims: the
turn budget binds in every block of twenty-four, sweeps all 64 cells of the
reference family, scans the rendered turns for held-out ticks, and checks that
hostile replies parse to a hold and score zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 17:44:20 -07:00
kartiandClaude Opus 5 8fe348ca82 probe: discover environments from manifests, not a hardcoded list
The environment list was written out four times — probe.py's shim tuple and its
results dict, tests/test_probe.py's ENVIRONMENTS, and ci.yml's wheel loop — so
adding an environment meant four edits and five workstreams collided on three
files. Discovery is now a tomllib scan of environments/*/pyproject.toml plus a
@probes(name) registry, with an optional [tool.arena] tasksets key for one
package that ships several (tera_spatial declares four).

An environment discovered without a registered probe WARNS rather than exits 1,
because interactive environments land their engine before their probe. That is
temporary; it becomes a hard failure with the last registration.

The warning is not a licence to stop being gated. test_probe.py pins the
ungated set to an explicit PENDING_PROBES allowlist that only ever shrinks —
without it, removing a @probes decorator moved the name from the table into a
warning line that already carries five, and every remaining assertion stayed
vacuously true while the environment silently stopped being gated.

Also untracks three environments/*/uv.lock files, which had been failing the
lock-policy step on main.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 15:54:56 -07:00
kartiandClaude Opus 5 16f4ee21ae Extend the merged CI and probe test to all seven environments
arena-environments / validate (3.11) (push) Failing after 5s
arena-environments / validate (3.12) (push) Failing after 5s
The redaction-pressure hardening landed a test asserting probe.py gates every
environment and a CI step building every wheel, and both carried a hardcoded
list of four. Merging three more environments in left the assertion counting
four "ok" lines against seven, and left the new wheels unbuilt by CI.

Both lists now name all seven. The count assertion is kept rather than relaxed:
a new environment missing from the list fails the build instead of quietly
narrowing what the test covers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 03:58:25 -07:00
karti d691acf131 Harden redaction-pressure reward and provenance
arena-environments / validate (3.11) (push) Successful in 1m8s
arena-environments / validate (3.12) (push) Successful in 31s
2026-08-19 01:26:21 -07:00