docs: redact the reference node's address and hostname
arena-environments / validate (3.11) (push) Successful in 1m23s
arena-environments / validate (3.12) (push) Successful in 1m8s

The eval docs named spark-1 by fleet hostname and published its inference
endpoint (a tailnet address and port) plus the fact that it answers without
an API key. This repo is PUBLIC, so that was a standing disclosure of an
internal, unauthenticated service to anonymous readers.

Endpoints now come from $SPARK_BASE_URL, which the operator exports, and the
host is referred to as "the reference node" -- the convention the compute
repo's docs already use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AaUFYkUTsJn1fnJ89qbvW
This commit is contained in:
2026-09-15 12:15:37 -07:00
co-authored by Claude Opus 5
parent 3025c4cf85
commit ef01bde068
7 changed files with 33 additions and 30 deletions
+2 -2
View File
@@ -188,7 +188,7 @@ It scored zero because **31 of 32 rollouts placed no orders at all**: 18 complet
`{"expected_profit": 0, "orders": []}`, `finish_reason=stop`. `planned` had mean 65.6 and
median 0 — one rollout in 32 planned anything at all; `profit_ratio` maxed at 0.323, so even
that one reached 29% of the reference. Nothing in this environment was measured. See
`FIRST_EVAL.md` §5 — spark-1 serves with
`FIRST_EVAL.md` §5 — the reference node serves with
`default_chat_template_kwargs = {"enable_thinking": false}`, and this environment is the one
that cannot be answered without arithmetic.
@@ -258,7 +258,7 @@ uv run --with regex python probe.py # floors and ceilings, per c
`tools/regate.py` rebuilds each task from the seed in its trace, re-runs the shipped scorer
over the model's own reply, and asks a candidate predicate directly. That is a better
measurement than a fresh sample would have been: it holds the model's behaviour fixed and
varies only the reward, and it costs nothing on spark-1. Its `recorded_mean` reproduces
varies only the reward, and it costs nothing on the reference node. Its `recorded_mean` reproduces
0.3488, 0.4432 and 0.4040 exactly, which is the cross-check that it is replaying the same
population `eval.log` scored.