3 Commits

Include renames

Author	SHA1	Message	Date
karti	61939676a5	Pin Arena environments to stable Verifiers	2026-08-25 10:33:35 -07:00
kartiandClaude Opus 5	122a858b61	pin verifiers 0.3.1.dev59, and make every config say subprocess ... The 600-second per-model-call ceiling that capped the thinking-on run at n=6 is gone: DEFAULT_TIMEOUT is read=None in all nine venvs. There is NO verifiers 0.4.0. The `version = "0.4.0"` in the vendored tree belongs to a [[tool.uv.dependency-metadata]] block for nemo-gym, and reading it as verifiers' own sends you hunting a release that does not exist and concluding the bump is blocked. The vendored checkout is v0.3.0-59-g4bcb48e5, published to PyPI as 0.3.1.dev59, and the installed wheel diffs clean against the whole vendored v1 tree. Pinned as an ordinary index dependency: a path source is reproducible only on this box and a git URL pins a commit no wheel matches. ⚠️ dev59 changes the default agent runtime from subprocess to `prime`, a REMOTE sandbox. Without an explicit runtime every eval here exits "not authenticated with prime" before a single rollout — having already created its output directory and a zero-line traces.jsonl, so it reads as a failed eval rather than an unauthenticated one. All nine configs now say subprocess. Nothing here needs a sandbox: the reward is a pure function of the trace and the harness is null. Output layout is flat now — <env>--<model>--<harness>--<hex8>/ with configs/, traces.jsonl and a logs/eval.log that 0.3.0 never wrote. `-o` is the output_dir, not the run dir; `--run.dir` pins the leaf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 23:14:05 -07:00
kartiandClaude Opus 5	00d21025ff	grand-exchange-live: the stepped market engine, and the turn budget binds ... Arena's first interactive environment, as its own wheel beside grand-exchange so the one-shot scores stay comparable rather than conflated. One basket of orders becomes a loop: quote, see fills and the next tick, re-quote, with inventory, open orders and cash carrying between turns. The engine and its measurement only. No taskset, no config, no probe row — the spec's constants were derived from a throwaway engine and did not reconcile (impatient computed to 0.612 against a reported 0.623), so every constant here is chosen from a ladder re-measured in the code that ships. House rule 4 asks whether the turn budget binds. It does: exhaustive 0.631 vs oracle 1.000, gap 0.369, stdev 0.054 over five blocks, worst block 0.316, and 0.354-0.405 at four further seed bases. That is 18x the proposed margin and it clears the 0.25 quantum of the binary gate outright, so the comparison is not a cliff. The saturation worry does not fire: per-basket clipping is not clipping the mean, so exhaustive averages 0.849 on profit_ratio rather than 1.000. The first draft's gap was FAKE — a clean-looking 0.152 produced by an under-tuned denominator whose every-look sibling earned 22% more gp while scoring below it. It was caught only by sweeping the reference's own family. Nothing that scores below the oracle may earn more than it; that check belongs in every interactive environment that follows this one. freeze is derived from market.seed inside build_market so viability and scoring see one value, roc is on peak rather than average capital employed, and the reference is cached per seed so scoring never re-simulates the played episode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 15:55:12 -07:00