docs: redact the reference node's address and hostname
The eval docs named spark-1 by fleet hostname and published its inference endpoint (a tailnet address and port) plus the fact that it answers without an API key. This repo is PUBLIC, so that was a standing disclosure of an internal, unauthenticated service to anonymous readers. Endpoints now come from $SPARK_BASE_URL, which the operator exports, and the host is referred to as "the reference node" -- the convention the compute repo's docs already use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012AaUFYkUTsJn1fnJ89qbvW
This commit is contained in:
@@ -188,7 +188,7 @@ It scored zero because **31 of 32 rollouts placed no orders at all**: 18 complet
|
||||
`{"expected_profit": 0, "orders": []}`, `finish_reason=stop`. `planned` had mean 65.6 and
|
||||
median 0 — one rollout in 32 planned anything at all; `profit_ratio` maxed at 0.323, so even
|
||||
that one reached 29% of the reference. Nothing in this environment was measured. See
|
||||
`FIRST_EVAL.md` §5 — spark-1 serves with
|
||||
`FIRST_EVAL.md` §5 — the reference node serves with
|
||||
`default_chat_template_kwargs = {"enable_thinking": false}`, and this environment is the one
|
||||
that cannot be answered without arithmetic.
|
||||
|
||||
@@ -258,7 +258,7 @@ uv run --with regex python probe.py # floors and ceilings, per c
|
||||
`tools/regate.py` rebuilds each task from the seed in its trace, re-runs the shipped scorer
|
||||
over the model's own reply, and asks a candidate predicate directly. That is a better
|
||||
measurement than a fresh sample would have been: it holds the model's behaviour fixed and
|
||||
varies only the reward, and it costs nothing on spark-1. Its `recorded_mean` reproduces
|
||||
varies only the reward, and it costs nothing on the reference node. Its `recorded_mean` reproduces
|
||||
0.3488, 0.4432 and 0.4040 exactly, which is the cross-check that it is replaying the same
|
||||
population `eval.log` scored.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user