The first environment shipped through .claude/workflows/new-environment.js:
specification, three adversarial reviews (all 'fixable', none fatal), the
Python environment, the TypeScript port, captured rollouts, and the demo page.
Eleven agents, no errors.
The proof that the platform scales is one line long. Alert Triage has a
completely different shape from Word Five — JSON actions, priced lookups, an
analyst screen instead of a grid — and the only change under
src/components/demo/ is a comment edit, because the isolation lint refused the
word "wordle" there. Zero shell code changed. 415 contract checks now pass
against two demos, up from 206 against one.
The environment is honest by construction. Every alert is synthetic, generated
from the seed, and the banner saying so sits inside the board surface. Two of
the eleven scenario templates are hidden-suspicious: generated by the same code
as their benign twin with the signal overlaid only in lookup data, so the free
screen is identically distributed and a screen-only policy STRUCTURALLY cannot
tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden
seeds. That is the counterweight made real rather than asserted.
Twelve policies, thirteen ladder assertions, a genuine three-way trade:
fast 0.846 wins hours (0.85), misses every hidden case
targeted 0.894 wins the shipped total
thorough 0.820 wins evidence (1.00), spends 2.9 hours
None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at
delta 0, and conformance gated on world + scorer + protocol so the browser shows
the same alert for ?seed= that Python generated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
An environment ships on its scripted agents alone — they are what the reward
flip is built on and they need no model. A recorded model attempt is captured
once from spark-1 for the Watch tab when it is serving, and the environment does
not block on it when it is not.
The Reward tab's ranking was thirty-two rows carrying four distinct labels —
"Never wastes a guess 1.000" seven times in a row. The flip is the one thing
that tab exists to show and it was invisible under that scroll. Arms are now
one per agent, each component averaged over that agent's scored runs, with a
null staying null rather than becoming a zero. Four rows. "Found the word at
any cost" moves Best-known play up one place and the shipped weights move it
back, visibly, in the browser.
The default run was runs[0]: the weakest agent on seed 0, a failed game with
thinking off. So Watch opened on a loss with an empty reasoning panel and
Reward opened on a row of zeros — the model's least interesting attempt, chosen
by sort order. It now prefers a run with recorded reasoning, then a solved one,
then the lowest seed so the choice is stable across deploys.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The environment page was a linear scroll of eight narrative beats. That is an
essay, and it is the wrong shape for somebody who has just chosen an
environment and wants to use it. It is now four tabs — Play, Watch, Reward,
Evidence — opening on Play, with the board above the fold at 390x844 and the
anatomy strip directly beneath it. The landing page leads with the picker
instead of burying it under the thesis.
The contract changed rather than layering tabs over beats. `Narrative.beats` is
gone; `claims: Record<DemoTabId, string>` replaces it, one required sentence per
tab. Writing the claim is how an author discovers whether a tab has anything to
say — a tab whose claim is hard to write is usually a tab with nothing in it.
Doing this now costs one migration; doing it after eleven more environments
costs twelve.
Tabs are derived, never declared: Play iff the demo ships an `interactive` mode,
Watch iff it has recorded runs. A demo that could name its own tabs would mean
environment seven inventing a fifth one and the site ceasing to be one product.
One thing the browser caught that no gate would have. The header stat strip
describes the RECORDED RUN, and on Play it sat above the visitor's own empty
board reading "Outcome: failed" — which parses as your game having already
failed before you touch a key. It now renders only on the tabs whose subject is
that run, which also moved the board 54px up the page.
The picker is honest about the shape of the lineup by construction: one built
environment gets its own block and the demo's real board as its thumbnail,
twelve written specifications render dimmed with a Spec badge, and every count
on the page is derived from the data rather than typed.
206 contract checks pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The footer built `${REPO_URL}/blob/main/LICENSE` by hand. Gitea serves files
at /src/branch/<ref>/, not GitHub's /blob/<ref>/, and both hosts 200 on the repo
root — so the link read fine in review and 404s only when clicked. That is
exactly how it survived the move.
There is now one `repoFile()` helper, so a single place knows the shape.
Every repo link in the built output was resolved against the live Gitea
instance: both 200.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
git.karti.ai/PIG/PIG-Demo is now the source of truth, public and anonymously
cloneable. Every self-link in the site and the docs repoints there — Gitea
serves file paths at /src/branch/main/, not /blob/main/, so those needed
rewriting rather than a hostname swap.
CI moves with it. An archived GitHub repo is read-only and its Actions stop
firing, so leaving the workflow there would have meant a repo whose gates
silently never run. .github/ is deleted rather than kept for reference: a
workflow that can never execute is worse than no workflow, because it looks
like coverage.
The Gitea workflow is not a copy. That runner is aarch64 and installs pnpm
through corepack from `packageManager` rather than pnpm/action-setup, uses
checkout@v4 and setup-node@v4, and fetches uv from astral.sh directly. It is
also configured `container.network: host` — nothing here needs a service
container, but the comment says so, because that setting cost the sibling repo
three failed runs.
The header and footer icon changed from the GitHub mark to a neutral one. A
GitHub logo pointing at a Gitea instance is a small lie about where the code
lives, on a site whose argument is that you can go and check it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
One Caddy block, both names, same root. The prerendered HTML already carries
<link rel="canonical"> and og:url pointing at the primeintellectgrowth name on
every route, so the alias does not split search ranking or leave a shared link
ambiguous about which site it belongs to. Verified byte-identical: both
hostnames return the same sha256 for /demos/wordle.
Worth recording why the alias failed before this rather than 404ing.
lumbridgecorp.com resolves on a WILDCARD, so every subdomain of it points at
cloud-2 whether or not Caddy knows the name. DNS completes, TLS opens, Caddy
finds no certificate for that SNI and aborts the handshake — the browser reports
ERR_SSL_PROTOCOL_ERROR, which reads as "the site is down" rather than "wrong
hostname". primeintellectgrowth.com has no wildcard, which is why the canonical
name needed an explicit A record.
deploy.sh now smoke-tests the alias too: if the block is ever edited to drop the
second name, the failure mode is a TLS error, and nothing else would catch it.
The live block is mirrored into deploy/Caddyfile.demo so the config is
reviewable in the repo rather than only on the host.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
sitemap.mjs was a step I ran by hand, so the moment I rebuilt without it the
file vanished from dist and the deploy shipped a site with no sitemap — on a
site whose entire point is being findable. Nothing caught it because nothing
was looking.
It is now inside `pnpm build`, CI asserts the file exists, and deploy.sh
smoke-tests /sitemap.xml, /robots.txt and an OG card against the live hostname.
The 404 assertion was also a warning rather than a failure; it exits now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The snapshot path was single-quoted inside the ssh payload, so the remote shell
never expanded the command substitution: every deploy wrote into one directory
named after the un-expanded literal, and the second deploy died on 'File
exists'. It looked correct for exactly as long as there had only ever been one
deploy — which is the failure mode this repo keeps finding, and the reason
deploy.sh smoke-tests the public hostname instead of trusting an exit code.
Two real snapshots now exist on the host.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The interactive board collapsed to a stub: an empty board has no intrinsic
width, so `w-full` inside a shrink-to-fit card resolved to nothing — and empty
is exactly the state it starts in, so it was only ever seen broken. It now
carries an explicit width.
The keyboard stretched to the full page because its parent was unconstrained,
putting the keys a screen away from the board they belong to. The play column
is capped at 28rem, and the agent's replay moved below its own heading instead
of sitting beside a lone paragraph.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
32 recorded rollouts, 8 seeds x 4 agents, every one replaying through the Python
engine to a delta of exactly 0.
arm solved economy consistency total
base-off 0/8 0.000 0.122 0.0244
base-on 4/8 0.469 0.479 0.4865
cautious 8/8 0.944 1.000 0.9831
solver 8/8 1.000 0.719 0.9437
Thinking-off solves none of eight. The same model on the same seeds, sampled
with thinking on, solves four. That is the headline, it is ours, and it needed
no training — which is also why the run is labelled an intervention and names
what was done to it, so a sampling change can never read as a training result.
`cautious` is new and it exists to make the reward editor honest. Until now the
page invited you to move a slider and watch the ranking change, and no slider
changed anything, because the solver dominated a model that solved nothing. A
candidate-only player — most informative guess among words that could still win
— takes consistency outright and pays for it in turns. Now:
only speed matters solver 0.9859 cautious 0.9606 -> solver
shipped weights solver 0.9437 cautious 0.9831 -> cautious
punish contradictions solver 0.8312 cautious 0.9972 -> cautious
The ranking really does flip, on recorded data, with one slider. The first
version of this arm picked the alphabetically-first candidate and opened on
'abaci', which made the policy the counterweight exists to reward look like a
straw man.
190 contract checks pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
Three things the build was quietly missing.
The interactive board was never mounted. The contract has `interactive.init` and
`interactive.Controls`, the demo implemented both, and the shell's `split-play`
beat rendered only the replay — so the beat titled "you and the model get the
same word" showed one board. PlayYourself now renders the visitor's attempt from
the same seed as the run beside it, generically: it knows only the contract, so
any demo shipping an interactive mode gets it and one that does not renders
nothing rather than an empty pane.
solver.worker.ts was dead code — nothing constructed it, which is how CI caught
it: `new Worker(` appeared nowhere in the bundle. It is wired now behind "what
would the best player guess?", and it answers in 92ms from a real worker on
boards no recording covers. That is the difference between a demo and a video.
It also surfaces the moment the solver picks a word that CANNOT win, which is
the counterweight visible in one line instead of explained in a paragraph.
The CI check that found it was itself wrong: it grepped every bundled file for
`blob:`, which React's own code contains in a scheme check, so it failed on a
risk that was not present. It now greps for worker construction from a blob,
which is the thing production CSP actually blocks in silence.
And the run switcher was thirty buttons carrying four distinct labels. Split
into arm and seed, holding the seed across an arm change — comparing two agents
means comparing them on the same hidden word, and silently jumping seeds would
break that while looking fine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
pnpm build ends in prerender, which drives a real chromium. Without the install
step the build failed on its very last action, after typecheck, the contract
gates and the tests had all gone green — the most expensive place to discover a
missing dependency.
Two silent bugs in the prerender pass, and the second was caused by the fix for
the first.
`waitForSelector('#root > *')` defaults to waiting for VISIBILITY, and the app's
first child is the skip link, which is hidden until focused. So it burned the
full 30s timeout on every one of 17 routes — twelve minutes of a script that
printed nothing, because its output was buffered behind a pipe — while the page
had rendered the whole time. Switching to `state: 'attached'` then fired too
early instead: useSeo writes the head from an effect, so the title was still
index.html's for a tick, and every route would have baked the homepage's head.
That is the exact bug this script exists to prevent. It now waits for `main`,
then for readyState, then settles.
The board's yellow was --warning, 32 95% 31% — darkened until white text cleared
4.5:1, and at that lightness it renders BROWN. On a board where people arrive
knowing this square should be yellow, a brown square reads as a bug in the
scorer, which on a page arguing "the grader is correct" is the worst thing it
could look like. The fill is now a real yellow and the glyph went dark: more
expected AND higher contrast, 10.02:1 against 5.03:1.
Also measured something the Honesty page had honestly declined to claim. It said
our word list is easier than the original's because our dictionary rule keeps
plurals the original's editor removed by hand. Running the same greedy solver
over both pools, 250 sampled words each: original 2,315 needs 3.552 guesses
(opener RAISE, worst 5), ours 4,603 needs 3.700 (opener TARES, worst 6). The
doubled pool outweighs the plurals. Ours is harder, and the page now says so
with the table.
177 gate checks pass. Entry chunk 106.9 kB gzipped against 160 kB.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
Five parallel lanes plus an integration pass. The header, gallery, router and
sitemap are all generated from the demo registry, so adding src/demos/<slug>/
puts a demo everywhere with zero edits to shared files — which is the whole
reason demo nine cannot break demo one.
check-demos enforces the twelve contract rules: 142 checks over one live demo.
Two worth naming. The shell may not mention a specific slug, because an
'if (slug === wordle)' in src/components/demo/ is a contract bug wearing a
patch. And a spec-status demo must ship a real specification — task, actions,
grader, counterweight, eval command — since a coming-soon card reads worse than
an honest empty gallery.
Bundle budget holds: entry 108.79 kB gzipped against a 160 kB ceiling, the demo
chunk 21.15 kB against 90 kB. recharts is 108 kB gzipped and lives behind a lazy
import so it never touches the entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The browser engine is a port of the Python one and CI proves it: all 21.2M
(guess, answer) pairs hashed on both sides to the same SHA-256. Six TS tests,
including the duplicate-letter table and the twelve pinned seed vectors that
keep ?seed= permalinks pointing at the same word the recording used.
Word lists are split by how they are used. answers.json is inlined because the
board needs it before first paint to turn a seed into a word, and a fetch there
means a visibly empty board on a cold cache. guesses.json is fetched, because it
is three times larger and only needed the first time somebody presses Enter;
until it lands, validation falls back to the answer list, which accepts strictly
fewer words. The failure mode is 'your real word was briefly rejected', not 'a
non-word was accepted' — the right way round.
The solver runs in a worker constructed from a same-origin module URL, never
Vite's ?worker&inline: that yields a blob:, and production CSP has no
worker-src, so it falls back to default-src 'self' and the worker is blocked
with no console error. It would fail in production only.
deploy.sh smoke-tests the real public hostname from the deploying machine and
fails on a body under 1 kB, because the bind bug's signature is a valid
certificate over an empty 200 and a local --resolve check passes anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
CONTRACT.md states the twelve rules and, for each, the failure it prevents.
AGENTS.md lists the three directories a demo agent may touch and the seven traps
that have already cost time here — pnpm 11's settings move, the non-portable
RNG, the two-pass rule, the two reward-denominator subtleties, the silent CSP
worker block, and Caddy's silent bind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The site does no live inference. Rollouts are captured once against spark-1 and
replayed at their recorded wall-clock — a public demo with no auth cannot hold
an API key, and a recorded run can be scrubbed, permalinked, blind-compared and
verified in ways a live one cannot. What stops it being a video is that the
browser re-derives every number from the recorded moves.
verify_fixtures.py is the Python half of that: it replays every committed
fixture through the engine and reproduces its own rewards. All 16 land at
delta 0.0. A fixture that cannot be regenerated is a claim with no receipt.
First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats
guesses it has already played, invents words (trape, slith, postt, boomy),
and contradicts its own feedback — consistency 0.09 to 0.17. That is the
published failure taxonomy showing up in our own data on the first run, and it
is why `consistency` is a reward component rather than a footnote.
A capture failure is recorded as a turn with a null reply, never dropped. A
capture that silently discarded failed turns would be reporting a better model
than the one that ran.
CI gates both halves and four things that fail silently in production: the word
lists must rebuild byte-identically, the prerendered routes must carry their own
baked og tags (crawlers do not run JS, so without them every shared link
previews as the homepage), no blob: URL may reach the bundle (the site's CSP has
no worker-src, so it falls back to default-src 'self' and a blob worker is
blocked with no error), and the conformance digest must match across languages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
engine.ts used mulberry32 and engine.py used random.Random(seed). Same seed,
different word — so every ?seed= permalink on the site would have shown a
different puzzle than the recorded run it claimed to be replaying, and nobody
would have noticed until someone checked one by hand.
Both now derive the index from FNV-1a 32-bit over the decimal seed. A hash
rather than a PRNG because there is no honest one-line JavaScript equivalent of
Mersenne Twister, and this way there is nothing to keep in step: both sides
compute the same integer from the same string. Math.imul on the JS side is
load-bearing — a plain multiply overflows into a double and diverges after the
first few bytes.
Twelve seeds are pinned as a vector in both test suites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
The Python is the source of truth; src/demos/wordle/engine.ts will be a port of
it, and CI gates the two against a SHA-256 over all 21.2M (guess, answer)
pattern pairs rather than a hand-picked vector file — a vector file only ever
catches the cases somebody thought of.
The reward is three weighted components, and the third one is the reason this
demo is worth building. `solved` and `economy` pull toward winning. `consistency`
pulls against them, because a player maximising information deliberately guesses
words that cannot win — a word that splits the remaining candidates evenly
teaches more than a word that might happen to be right. That is good play, and
it costs consistency.
The probe ladder proves the tension is real rather than asserted:
inaction 0.0000 crude 0.0111 plausible 0.1224
candidate_only 0.8925 exhaustive 0.9031 oracle 0.9458
The two good policies are 0.05 apart and neither dominates — the entropy oracle
takes 1.00 economy and 0.73 consistency, the candidate-only player takes 0.75
and 1.00. Which one wins is a decision about what you want, which is the whole
argument the site exists to make. probe.py fails CI if either starts dominating.
Two traps found by building it. `consistency` is scored over turns SPENT, not
guesses accepted: counting only legal guesses hands a free 1.0 to a policy that
plays one word and then jams the parser five times — one guess, no
contradictions, perfect score. And `economy`'s denominator is the depth the
SHIPPED solver reaches, not a depth-optimal search: entropy-greedy is not
depth-optimal, so grading it against an exact optimum would make the oracle
rung fail its own assertion on some seeds.
The word lists are built from Wordnik (MIT) intersected with SCOWL, never from
the original game's 2,315 answers. 4,603 answers makes this materially harder
than the original, so the published SALET/3.4212 results are cited as belonging
to that list and our own reference player's TARES/3.72 is measured here.
verifiers is an optional extra. The engine, reward, solver and probe all run —
and gate — without an RL stack resolvable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
Foundation for a gallery of RL-environment demos. Three decisions worth
recording here rather than in a commit nobody reads:
The word lists are built, not copied. `envs/wordle_five/words/build_words.py`
intersects Wordnik (MIT, 11,846 five-letter words) with SCOWL's common-American
tier to produce 4,603 answers. The intersection is the point: the list is
derived from two permissive sources by a stated rule rather than copied from
anyone's editorial selection, and both inputs are committed so a rebuild is
byte-identical.
The design tokens are PIG's, inlined as literals. PIG writes its accent onto
the root at runtime because a user picks it; this site has no such choice, so
the runtime theme layer would be a moving part buying nothing. Board tiles get
their own named tokens with measured contrast ratios, because the board is the
one place where colour carries meaning.
pnpm 11 no longer reads the "pnpm" field in package.json. Settings live in
pnpm-workspace.yaml, and an unapproved build script makes `pnpm install` exit 1
rather than warn — so this would have failed CI on a clean checkout, not here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
karti-ai