3 Commits

Include renames

Author	SHA1	Message	Date
kartiandClaude Opus 5	ef01bde068	docs: redact the reference node's address and hostname ... The eval docs named spark-1 by fleet hostname and published its inference endpoint (a tailnet address and port) plus the fact that it answers without an API key. This repo is PUBLIC, so that was a standing disclosure of an internal, unauthenticated service to anonymous readers. Endpoints now come from $SPARK_BASE_URL, which the operator exports, and the host is referred to as "the reference node" -- the convention the compute repo's docs already use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012AaUFYkUTsJn1fnJ89qbvW	2026-09-15 12:15:37 -07:00
karti	61939676a5	Pin Arena environments to stable Verifiers	2026-08-25 10:33:35 -07:00
kartiandClaude Opus 5	e4cc2bf1c3	docs: the scope, the plan, the post-review execution plan, and the eval runbook ... MULTI_TURN.md scopes the multi-turn gap and records the finding that made it cheap: multi-turn needs no container and no real harness. vf.Env.run programs the control flow host-side and the simulator runs in-process as the user, with the harness left null. PLAN.md carries the settled decisions. EXECUTION.md is the post-adversarial-review plan — every position in it survived a reviewer, and where a design spec was overturned the corrected position is what is written down. Two corrections are marked in place rather than silently rewritten, because both were believed and acted on: - The provenance hole is src/arena/index.ts, not offices/sites.ts. Mutating offices/sites.ts produces a bit-identical episode; index.ts re-exports the scripted baselines that ARE the reward denominator, and a halved office-nav baseline moves return 4.744 -> 4.098 while every tera hash gate prints ok. - office-jobs-v1 is not phantom. It was absent at 13:00 and present at 15:50, because another session fast-forward-merged it into the shared tera tree mid-session. The counts are twelve, five and twenty. docs/FIRST_EVAL.md is the runbook for the first evaluation Arena ever completed: seven environments, 224 episodes, 223 scored, against brain-qwen38-dspark. It warns that the mean is sum(score x weight) — an unweighted mean over components is a different number, and it is how six of seven means were first misreported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 15:55:43 -07:00