docs: sentence-case the redacted node name
arena-environments / validate (3.11) (push) Successful in 1m9s
arena-environments / validate (3.12) (push) Successful in 1m21s

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AaUFYkUTsJn1fnJ89qbvW
This commit is contained in:
2026-09-15 12:16:04 -07:00
co-authored by Claude Opus 5
parent ef01bde068
commit 7abbd24d34
3 changed files with 6 additions and 6 deletions
+4 -4
View File
@@ -49,10 +49,10 @@ see §5. The seven data environments have thinking-on copies in `outputs/thinkin
|---|---|
| `--model brain-qwen38-dspark` | **The served id, pinned.** `--model brain` does *not* 404. SGLang does not validate the model field on the chat route: it returns 200, Qwen answers, and the trace records `brain`. The old `outputs/canary-trap--brain--null/` dir is exactly that mistake. Pin the id `GET /v1/models` reports. |
| `--client.base-url` | The default is `https://api.pinference.ai/api/v1`, which needs a `PRIME_API_KEY` this box does not have. |
| `--client.api-key-var SPARK_API_KEY` | the reference node does not authenticate with a Prime key. The client still resolves *some* env var, so point it at a dummy one rather than leaving it on `PRIME_API_KEY` (empty → the failure mode of the one earlier attempt). |
| `--client.api-key-var SPARK_API_KEY` | The reference node does not authenticate with a Prime key. The client still resolves *some* env var, so point it at a dummy one rather than leaving it on `PRIME_API_KEY` (empty → the failure mode of the one earlier attempt). |
| `--no-push` | Hub push is parked (PLAN.md). Push is **on by default** and needs `$PRIME_API_KEY`. |
| `--no-rich` | The live dashboard is in-process only and unreadable in a log. Without it you get one `rollout done: … reward=…` line per rollout. |
| `-c 8` | the reference node is shared with Chatterbox, vox and ASR. The config's own `max_concurrent = 128` will hammer it. |
| `-c 8` | The reference node is shared with Chatterbox, vox and ASR. The config's own `max_concurrent = 128` will hammer it. |
| `--env.agent.runtime.type subprocess` | ⚠️ **New, and not optional.** 0.3.0 defaulted the agent runtime to `subprocess`; `0.3.1` defaults it to `prime` — a *remote* sandbox. A remote role makes `Env._runs_local()` false, which makes the interception pool mint a Prime tunnel, which calls `ensure_prime_auth()` and raises `SystemExit: not authenticated with prime`. The run dies before a single rollout, having already created its output directory and an empty `traces.jsonl`. The null harness needs no runtime at all; naming a local one is the whole fix. |
| `-o <dir>` + `--run.dir <name>` | `-o` is `output_dir` and the run writes to `output_dir / run.dir`. On 0.3.0 a bare `-o` gave you a flat directory; on 0.3.1 it gives you `outputs/<your-dir>/<env>--<model>--<harness>--<hex8>/`. Pass `--run.dir` to name the leaf yourself. |
@@ -329,7 +329,7 @@ Appended 2026-08-21, after re-measuring `grand-exchange`.
### The cause
the reference node's SGLang is started with a default that overrides the chat template:
The reference node's SGLang is started with a default that overrides the chat template:
```bash
curl -s "${SPARK_BASE_URL%/v1}/get_server_info" | jq '.default_chat_template_kwargs, .reasoning_parser'
@@ -466,7 +466,7 @@ false`. The only lever is `max_tokens`, and `max_tokens` cuts the answer, not th
Three consequences for anyone re-running this suite against a reasoning model:
1. **Keep `-c` low and `max_tokens` under the wall**, or budget the run knowing calls will be
cut at ten minutes. the reference node is shared — another lane's eval was running against it during
cut at ten minutes. The reference node is shared — another lane's eval was running against it during
this measurement, and contention alone moves a rollout across the line.
2. **A timed-out run leaves no evidence.** Zero traces, exit code 0. `count_rewards.py`
reports on a file that does not exist. This is the same trap as §3 with a bigger blast