The first environment shipped through .claude/workflows/new-environment.js:
specification, three adversarial reviews (all 'fixable', none fatal), the
Python environment, the TypeScript port, captured rollouts, and the demo page.
Eleven agents, no errors.
The proof that the platform scales is one line long. Alert Triage has a
completely different shape from Word Five — JSON actions, priced lookups, an
analyst screen instead of a grid — and the only change under
src/components/demo/ is a comment edit, because the isolation lint refused the
word "wordle" there. Zero shell code changed. 415 contract checks now pass
against two demos, up from 206 against one.
The environment is honest by construction. Every alert is synthetic, generated
from the seed, and the banner saying so sits inside the board surface. Two of
the eleven scenario templates are hidden-suspicious: generated by the same code
as their benign twin with the signal overlaid only in lookup data, so the free
screen is identically distributed and a screen-only policy STRUCTURALLY cannot
tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden
seeds. That is the counterweight made real rather than asserted.
Twelve policies, thirteen ladder assertions, a genuine three-way trade:
fast 0.846 wins hours (0.85), misses every hidden case
targeted 0.894 wins the shipped total
thorough 0.820 wins evidence (1.00), spends 2.9 hours
None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at
delta 0, and conformance gated on world + scorer + protocol so the browser shows
the same alert for ?seed= that Python generated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The environment page was a linear scroll of eight narrative beats. That is an
essay, and it is the wrong shape for somebody who has just chosen an
environment and wants to use it. It is now four tabs — Play, Watch, Reward,
Evidence — opening on Play, with the board above the fold at 390x844 and the
anatomy strip directly beneath it. The landing page leads with the picker
instead of burying it under the thesis.
The contract changed rather than layering tabs over beats. `Narrative.beats` is
gone; `claims: Record<DemoTabId, string>` replaces it, one required sentence per
tab. Writing the claim is how an author discovers whether a tab has anything to
say — a tab whose claim is hard to write is usually a tab with nothing in it.
Doing this now costs one migration; doing it after eleven more environments
costs twelve.
Tabs are derived, never declared: Play iff the demo ships an `interactive` mode,
Watch iff it has recorded runs. A demo that could name its own tabs would mean
environment seven inventing a fifth one and the site ceasing to be one product.
One thing the browser caught that no gate would have. The header stat strip
describes the RECORDED RUN, and on Play it sat above the visitor's own empty
board reading "Outcome: failed" — which parses as your game having already
failed before you touch a key. It now renders only on the tabs whose subject is
that run, which also moved the board 54px up the page.
The picker is honest about the shape of the lineup by construction: one built
environment gets its own block and the demo's real board as its thumbnail,
twelve written specifications render dimmed with a Spec badge, and every count
on the page is derived from the data rather than typed.
206 contract checks pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
32 recorded rollouts, 8 seeds x 4 agents, every one replaying through the Python
engine to a delta of exactly 0.
arm solved economy consistency total
base-off 0/8 0.000 0.122 0.0244
base-on 4/8 0.469 0.479 0.4865
cautious 8/8 0.944 1.000 0.9831
solver 8/8 1.000 0.719 0.9437
Thinking-off solves none of eight. The same model on the same seeds, sampled
with thinking on, solves four. That is the headline, it is ours, and it needed
no training — which is also why the run is labelled an intervention and names
what was done to it, so a sampling change can never read as a training result.
`cautious` is new and it exists to make the reward editor honest. Until now the
page invited you to move a slider and watch the ranking change, and no slider
changed anything, because the solver dominated a model that solved nothing. A
candidate-only player — most informative guess among words that could still win
— takes consistency outright and pays for it in turns. Now:
only speed matters solver 0.9859 cautious 0.9606 -> solver
shipped weights solver 0.9437 cautious 0.9831 -> cautious
punish contradictions solver 0.8312 cautious 0.9972 -> cautious
The ranking really does flip, on recorded data, with one slider. The first
version of this arm picked the alphabetically-first candidate and opened on
'abaci', which made the policy the counterweight exists to reward look like a
straw man.
190 contract checks pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
pnpm build ends in prerender, which drives a real chromium. Without the install
step the build failed on its very last action, after typecheck, the contract
gates and the tests had all gone green — the most expensive place to discover a
missing dependency.
Two silent bugs in the prerender pass, and the second was caused by the fix for
the first.
`waitForSelector('#root > *')` defaults to waiting for VISIBILITY, and the app's
first child is the skip link, which is hidden until focused. So it burned the
full 30s timeout on every one of 17 routes — twelve minutes of a script that
printed nothing, because its output was buffered behind a pipe — while the page
had rendered the whole time. Switching to `state: 'attached'` then fired too
early instead: useSeo writes the head from an effect, so the title was still
index.html's for a tick, and every route would have baked the homepage's head.
That is the exact bug this script exists to prevent. It now waits for `main`,
then for readyState, then settles.
The board's yellow was --warning, 32 95% 31% — darkened until white text cleared
4.5:1, and at that lightness it renders BROWN. On a board where people arrive
knowing this square should be yellow, a brown square reads as a bug in the
scorer, which on a page arguing "the grader is correct" is the worst thing it
could look like. The fill is now a real yellow and the glyph went dark: more
expected AND higher contrast, 10.02:1 against 5.03:1.
Also measured something the Honesty page had honestly declined to claim. It said
our word list is easier than the original's because our dictionary rule keeps
plurals the original's editor removed by hand. Running the same greedy solver
over both pools, 250 sampled words each: original 2,315 needs 3.552 guesses
(opener RAISE, worst 5), ours 4,603 needs 3.700 (opener TARES, worst 6). The
doubled pool outweighs the plurals. Ours is harder, and the page now says so
with the table.
177 gate checks pass. Entry chunk 106.9 kB gzipped against 160 kB.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
Five parallel lanes plus an integration pass. The header, gallery, router and
sitemap are all generated from the demo registry, so adding src/demos/<slug>/
puts a demo everywhere with zero edits to shared files — which is the whole
reason demo nine cannot break demo one.
check-demos enforces the twelve contract rules: 142 checks over one live demo.
Two worth naming. The shell may not mention a specific slug, because an
'if (slug === wordle)' in src/components/demo/ is a contract bug wearing a
patch. And a spec-status demo must ship a real specification — task, actions,
grader, counterweight, eval command — since a coming-soon card reads worse than
an honest empty gallery.
Bundle budget holds: entry 108.79 kB gzipped against a 160 kB ceiling, the demo
chunk 21.15 kB against 90 kB. recharts is 108 kB gzipped and lives behind a lazy
import so it never touches the entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The browser engine is a port of the Python one and CI proves it: all 21.2M
(guess, answer) pairs hashed on both sides to the same SHA-256. Six TS tests,
including the duplicate-letter table and the twelve pinned seed vectors that
keep ?seed= permalinks pointing at the same word the recording used.
Word lists are split by how they are used. answers.json is inlined because the
board needs it before first paint to turn a seed into a word, and a fetch there
means a visibly empty board on a cold cache. guesses.json is fetched, because it
is three times larger and only needed the first time somebody presses Enter;
until it lands, validation falls back to the answer list, which accepts strictly
fewer words. The failure mode is 'your real word was briefly rejected', not 'a
non-word was accepted' — the right way round.
The solver runs in a worker constructed from a same-origin module URL, never
Vite's ?worker&inline: that yields a blob:, and production CSP has no
worker-src, so it falls back to default-src 'self' and the worker is blocked
with no console error. It would fail in production only.
deploy.sh smoke-tests the real public hostname from the deploying machine and
fails on a body under 1 kB, because the bind bug's signature is a valid
certificate over an empty 200 and a local --resolve check passes anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The site does no live inference. Rollouts are captured once against spark-1 and
replayed at their recorded wall-clock — a public demo with no auth cannot hold
an API key, and a recorded run can be scrubbed, permalinked, blind-compared and
verified in ways a live one cannot. What stops it being a video is that the
browser re-derives every number from the recorded moves.
verify_fixtures.py is the Python half of that: it replays every committed
fixture through the engine and reproduces its own rewards. All 16 land at
delta 0.0. A fixture that cannot be regenerated is a claim with no receipt.
First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats
guesses it has already played, invents words (trape, slith, postt, boomy),
and contradicts its own feedback — consistency 0.09 to 0.17. That is the
published failure taxonomy showing up in our own data on the first run, and it
is why `consistency` is a reward component rather than a footnote.
A capture failure is recorded as a turn with a null reply, never dropped. A
capture that silently discarded failed turns would be reporting a better model
than the one that ran.
CI gates both halves and four things that fail silently in production: the word
lists must rebuild byte-identically, the prerendered routes must carry their own
baked og tags (crawlers do not run JS, so without them every shared link
previews as the homepage), no blob: URL may reach the bundle (the site's CSP has
no worker-src, so it falls back to default-src 'self' and a blob worker is
blocked with no error), and the conformance digest must match across languages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The Python is the source of truth; src/demos/wordle/engine.ts will be a port of
it, and CI gates the two against a SHA-256 over all 21.2M (guess, answer)
pattern pairs rather than a hand-picked vector file — a vector file only ever
catches the cases somebody thought of.
The reward is three weighted components, and the third one is the reason this
demo is worth building. `solved` and `economy` pull toward winning. `consistency`
pulls against them, because a player maximising information deliberately guesses
words that cannot win — a word that splits the remaining candidates evenly
teaches more than a word that might happen to be right. That is good play, and
it costs consistency.
The probe ladder proves the tension is real rather than asserted:
inaction 0.0000 crude 0.0111 plausible 0.1224
candidate_only 0.8925 exhaustive 0.9031 oracle 0.9458
The two good policies are 0.05 apart and neither dominates — the entropy oracle
takes 1.00 economy and 0.73 consistency, the candidate-only player takes 0.75
and 1.00. Which one wins is a decision about what you want, which is the whole
argument the site exists to make. probe.py fails CI if either starts dominating.
Two traps found by building it. `consistency` is scored over turns SPENT, not
guesses accepted: counting only legal guesses hands a free 1.0 to a policy that
plays one word and then jams the parser five times — one guess, no
contradictions, perfect score. And `economy`'s denominator is the depth the
SHIPPED solver reaches, not a depth-optimal search: entropy-greedy is not
depth-optimal, so grading it against an exact optimum would make the oracle
rung fail its own assertion on some seeds.
The word lists are built from Wordnik (MIT) intersected with SCOWL, never from
the original game's 2,315 answers. 4,603 answers makes this materially harder
than the original, so the published SALET/3.4212 results are cited as belonging
to that list and our own reference player's TARES/3.72 is measured here.
verifiers is an optional extra. The engine, reward, solver and probe all run —
and gate — without an RL stack resolvable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
karti-ai