Wordle module, cross-language tests, and the deploy path

The browser engine is a port of the Python one and CI proves it: all 21.2M
(guess, answer) pairs hashed on both sides to the same SHA-256. Six TS tests,
including the duplicate-letter table and the twelve pinned seed vectors that
keep ?seed= permalinks pointing at the same word the recording used.

Word lists are split by how they are used. answers.json is inlined because the
board needs it before first paint to turn a seed into a word, and a fetch there
means a visibly empty board on a cold cache. guesses.json is fetched, because it
is three times larger and only needed the first time somebody presses Enter;
until it lands, validation falls back to the answer list, which accepts strictly
fewer words. The failure mode is 'your real word was briefly rejected', not 'a
non-word was accepted' — the right way round.

The solver runs in a worker constructed from a same-origin module URL, never
Vite's ?worker&inline: that yields a blob:, and production CSP has no
worker-src, so it falls back to default-src 'self' and the worker is blocked
with no console error. It would fail in production only.

deploy.sh smoke-tests the real public hostname from the deploying machine and
fails on a body under 1 kB, because the bind bug's signature is a valid
certificate over an empty 200 and a local --resolve check passes anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
2026-08-28 15:53:25 -07:00
parent 2c2dcad9fd
commit b601511e7f
38 changed files with 3419 additions and 646 deletions
+48
View File
@@ -0,0 +1,48 @@
# Deploying
Static build, rsynced to cloud-2, served by Caddy.
```bash
pnpm build && bash deploy/deploy.sh
```
## The pieces
| | |
|---|---|
| Host | cloud-2, `ubuntu@100.92.185.76` (tailnet only) |
| Root | `/var/www/demo.primeintellectgrowth.com` |
| Snapshots | `…-rollbacks/`, last 10, hard-linked |
| DNS | OCI zone `primeintellectgrowth.com``170.9.14.61`, explicit A record, no wildcard |
| Caddy | a block appended to `/etc/caddy/Caddyfile` |
## The trap that costs an afternoon
**`bind 10.0.0.2` is mandatory in the Caddy block, and its absence is silent.**
Without it Caddy builds a second server on `*:443` that has never heard of this
hostname. Public traffic — which NATs to `10.0.0.2` — falls through to an empty
`200` behind a perfectly valid certificate. Worse, a
`curl --resolve demo.primeintellectgrowth.com:443:127.0.0.1` from cloud-2 itself
still passes.
`deploy.sh` therefore smoke-tests the real public hostname from the deploying
machine and fails on a response under 1 kB.
Do not use PIG's `deploy/Caddyfile.example` as a template — it omits the bind.
## Rolling back
```bash
ssh ubuntu@100.92.185.76
ls -1dt /var/www/demo.primeintellectgrowth.com-rollbacks/*/
sudo rsync -a --delete <that-dir>/ /var/www/demo.primeintellectgrowth.com/
```
No Caddy reload needed; the root path does not change.
## Editing the live Caddyfile
Scope the edit to this site's block. Several sites on cloud-2 carry
byte-identical header strings, so a naive global replace hits two of them. Slice
between `demo.primeintellectgrowth.com {` and the next hostname, and assert the
match is unique inside that slice.
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Ship dist/ to cloud-2.
#
# Run from a machine on the tailnet (cloud-2's :22 is tailnet-only, which is
# also why a GitHub-hosted runner cannot do this).
#
# bash deploy/deploy.sh
#
# Keeps the last 10 releases as hard-linked snapshots, so a rollback is a
# directory rename rather than a rebuild.
set -euo pipefail
HOST="${PIG_DEMO_HOST:-ubuntu@100.92.185.76}"
ROOT="/var/www/demo.primeintellectgrowth.com"
SNAPS="${ROOT}-rollbacks"
URL="https://demo.primeintellectgrowth.com"
cd "$(dirname "$0")/.."
[ -d dist ] || { echo "no dist/ — run 'pnpm build' first"; exit 1; }
[ -f dist/index.html ] || { echo "dist/index.html missing"; exit 1; }
# The prerender pass is what makes shared links preview correctly. A dist
# without it builds and serves fine, which is exactly why it needs asserting.
[ -f dist/404.html ] || { echo "dist/404.html missing — did prerender run?"; exit 1; }
echo "==> preflight on ${HOST}"
ssh "$HOST" "set -e
free=\$(df --output=avail -BG / | tail -1 | tr -dc 0-9)
[ \"\$free\" -ge 3 ] || { echo \"only \${free}G free on /\"; exit 1; }
mkdir -p '$SNAPS'
if [ -d '$ROOT' ] && [ -n \"\$(ls -A '$ROOT' 2>/dev/null)\" ]; then
cp -al '$ROOT' '$SNAPS/\$(date +%Y%m%d-%H%M%S)'
fi
ls -1dt '$SNAPS'/*/ 2>/dev/null | tail -n +11 | xargs -r rm -rf"
echo "==> rsync"
rsync -az --delete --checksum dist/ "$HOST:$ROOT/"
echo "==> smoke test against the public hostname"
# Against the real name from THIS machine, never --resolve from cloud-2: a
# missing `bind 10.0.0.2` in the Caddy block serves an empty 200 to the
# internet while a local --resolve check still passes.
code=$(curl -sS -o /tmp/pigdemo-smoke.html -w '%{http_code}' --max-time 30 "$URL/")
size=$(wc -c < /tmp/pigdemo-smoke.html)
echo " / -> $code, ${size}b"
[ "$code" = "200" ] || { echo "FAILED: / returned $code"; exit 1; }
[ "$size" -gt 1000 ] || { echo "FAILED: / is ${size}b — almost certainly the empty-200 bind bug"; exit 1; }
grep -q '<div id="root"' /tmp/pigdemo-smoke.html || { echo "FAILED: no app root in the HTML"; exit 1; }
demo=$(curl -sS -o /tmp/pigdemo-demo.html -w '%{http_code}' --max-time 30 "$URL/demos/wordle")
echo " /demos/wordle -> $demo"
[ "$demo" = "200" ] || { echo "FAILED: demo route returned $demo"; exit 1; }
grep -q 'og:title' /tmp/pigdemo-demo.html || { echo "FAILED: demo route has no baked og tags"; exit 1; }
missing=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 30 "$URL/nope-not-a-page")
echo " /nope-not-a-page -> $missing"
[ "$missing" = "404" ] || { echo "WARNING: unknown path returned $missing, expected 404"; }
echo "==> live: $URL"
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env python3
"""Build public/traces/manifest.json from the captured fixtures.
The manifest is what the browser reads to know which runs exist. It is
generated rather than hand-written so a fixture can never be referenced without
existing, or exist without being referenced.
"""
from __future__ import annotations
import json
from pathlib import Path
TRACES = Path(__file__).parent.parent / "public" / "traces"
ARMS = {
"base-off": ("Out of the box", "recorded"),
"base-on": ("Allowed to think", "intervened"),
"solver": ("Best-known play", "generated"),
}
# What was done to the run, for arms that had something done to them. Required
# by the contract on any `intervened` run so that a sampling change can never be
# presented as a training result by omitting to mention it.
INTERVENTIONS = {
"base-on": "Same model, same seeds, sampled with thinking enabled. No training, no fine-tuning.",
}
ORDER = ["base-off", "base-on", "solver"]
def main() -> int:
manifest: dict[str, list[dict]] = {}
for demo_dir in sorted(p for p in TRACES.iterdir() if p.is_dir()):
runs = []
for path in sorted(demo_dir.glob("*.json")):
data = json.loads(path.read_text())
arm = data["runId"].rsplit("-s", 1)[0]
label, kind = ARMS.get(arm, (arm, "recorded"))
run = {
"id": data["runId"],
"label": label,
"path": f"/traces/{demo_dir.name}/{path.name}",
"kind": kind,
"model": data["model"],
"capturedAt": data["capturedAt"],
"seed": data["seed"],
}
if arm in INTERVENTIONS:
run["intervention"] = INTERVENTIONS[arm]
runs.append(run)
runs.sort(key=lambda r: (ORDER.index(r["id"].rsplit("-s", 1)[0])
if r["id"].rsplit("-s", 1)[0] in ORDER else 99,
r["seed"]))
if runs:
manifest[demo_dir.name] = runs
out = TRACES / "manifest.json"
out.write_text(json.dumps(manifest, indent=2) + "\n")
for slug, runs in manifest.items():
by_arm: dict[str, list[dict]] = {}
for r in runs:
by_arm.setdefault(r["id"].rsplit("-s", 1)[0], []).append(r)
print(f"{slug}: {len(runs)} runs")
for arm, group in by_arm.items():
solved = 0
for r in group:
data = json.loads((TRACES.parent / r["path"].lstrip("/")).read_text())
solved += 1 if data["outcome"] == "solved" else 0
print(f" {arm:<10} {len(group)} runs, solved {solved}/{len(group)}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+1 -1
View File
@@ -13,7 +13,7 @@
"build": "tsc --noEmit && vite build && node scripts/prerender.mjs",
"preview": "vite preview --port 4173",
"typecheck": "tsc --noEmit",
"test": "node --test src/**/__tests__/*.test.ts",
"test": "tsx --test src/demos/*/__tests__/*.test.ts",
"check": "node scripts/check-demos.mjs && node scripts/check-receipts.mjs && node scripts/check-licenses.mjs",
"demo:new": "node scripts/new-demo.mjs",
"conformance": "node scripts/conformance.mjs"
+148
View File
@@ -0,0 +1,148 @@
{
"wordle": [
{
"id": "base-off-s0",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s0.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 0
},
{
"id": "base-off-s1",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s1.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 1
},
{
"id": "base-off-s2",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s2.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 2
},
{
"id": "base-off-s3",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s3.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 3
},
{
"id": "base-off-s4",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s4.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 4
},
{
"id": "base-off-s5",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s5.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 5
},
{
"id": "base-off-s6",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s6.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 6
},
{
"id": "base-off-s7",
"label": "Out of the box",
"path": "/traces/wordle/base-off-s7.json",
"kind": "recorded",
"model": "brain-qwen38-dspark",
"capturedAt": "2026-08-28",
"seed": 7
},
{
"id": "solver-s0",
"label": "Best-known play",
"path": "/traces/wordle/solver-s0.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 0
},
{
"id": "solver-s1",
"label": "Best-known play",
"path": "/traces/wordle/solver-s1.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 1
},
{
"id": "solver-s2",
"label": "Best-known play",
"path": "/traces/wordle/solver-s2.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 2
},
{
"id": "solver-s3",
"label": "Best-known play",
"path": "/traces/wordle/solver-s3.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 3
},
{
"id": "solver-s4",
"label": "Best-known play",
"path": "/traces/wordle/solver-s4.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 4
},
{
"id": "solver-s5",
"label": "Best-known play",
"path": "/traces/wordle/solver-s5.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 5
},
{
"id": "solver-s6",
"label": "Best-known play",
"path": "/traces/wordle/solver-s6.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 6
},
{
"id": "solver-s7",
"label": "Best-known play",
"path": "/traces/wordle/solver-s7.json",
"kind": "generated",
"model": "entropy-solver",
"capturedAt": "2026-08-28",
"seed": 7
}
]
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+5
View File
@@ -401,8 +401,13 @@ export function parseStringUnion(src, typeName) {
/* ------------------------------------------------------------ demo metadata */
const META_ANCHORS = [
// `export default defineMeta({...})` is the shape the template uses, and the
// one the registry's glob reads. It has to come first: a file can also carry
// a `meta:` key inside something else, and that would win on file order.
/\bdefineMeta\s*(?:<[^>]*>)?\s*\(\s*/,
/(?:export\s+)?const\s+meta\s*(?::\s*[^=]+)?=\s*/,
/(?:export\s+)?const\s+[A-Za-z_$][\w$]*Meta\s*(?::\s*[^=]+)?=\s*/,
/export\s+default\s*/,
/\bmeta\s*:\s*/,
];
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env node
/**
* The bundle budget.
*
* Two numbers, both gzipped, both enforced:
*
* entry chunk <= 160 kB what every visitor downloads before they see
* anything, on whatever connection the boardroom
* wifi is having that morning
* any demo <= 90 kB a demo is lazy, so this is what clicking one
* costs — and it is per demo, so the tenth demo
* cannot be paid for by the first nine
*
* The vendor chunks (`react`, `charts`, named in vite.config.ts) are printed
* but not capped. They are shared, cached across every route, and capping them
* here would only produce pressure to inline them into the entry, which is the
* opposite of what we want.
*
* Gzip, not brotli, and not raw: gzip is the floor every host actually serves,
* so it is the honest number. Brotli would flatter us by about 15%.
*
* Run after a build: `pnpm build && node scripts/bundle-budget.mjs`.
*/
import fs from 'node:fs';
import path from 'node:path';
import { abs, die, dim, exists, green, gzipSize, kb, red, rel, table, yellow } from './_lib.mjs';
const ENTRY_BUDGET = 160 * 1024;
const DEMO_BUDGET = 90 * 1024;
const DIST = abs('dist');
const ASSETS = path.join(DIST, 'assets');
const INDEX = path.join(DIST, 'index.html');
if (!exists(INDEX)) die(`${rel(INDEX)} does not exist. Run \`pnpm build\` first.`);
if (!exists(ASSETS)) die(`${rel(ASSETS)} does not exist. The build produced no assets, which is itself the bug.`);
/**
* The entry is whatever `index.html` loads directly.
*
* Derived rather than matched by filename: Vite's entry is `index-<hash>.js`
* today, but the moment someone renames the entry in rollupOptions, a
* name-matching version of this script starts measuring nothing and passing.
*/
const indexHtml = fs.readFileSync(INDEX, 'utf8');
const entryNames = new Set(
[...indexHtml.matchAll(/<script[^>]+type=["']module["'][^>]+src=["']([^"']+)["']/g)].map((m) => path.basename(m[1])),
);
if (entryNames.size === 0) {
die(`no module <script> found in ${rel(INDEX)}, so the entry chunk cannot be identified.`);
}
/** Named in vite.config.ts's manualChunks. Shared, cached, deliberately uncapped. */
const VENDOR = /^(react|charts)-/;
const files = fs
.readdirSync(ASSETS)
.filter((name) => name.endsWith('.js') || name.endsWith('.css'))
.sort();
const rows = [];
const failures = [];
let totalGzip = 0;
for (const name of files) {
const bytes = fs.readFileSync(path.join(ASSETS, name));
const gz = gzipSize(bytes);
totalGzip += gz;
const kind = entryNames.has(name)
? 'entry'
: name.endsWith('.css')
? 'css'
: VENDOR.test(name)
? 'vendor'
: 'demo/lazy';
const budget = kind === 'entry' ? ENTRY_BUDGET : kind === 'demo/lazy' ? DEMO_BUDGET : null;
const over = budget !== null && gz > budget;
if (over) {
failures.push(
`assets/${name} is ${kb(gz)} gzipped, over the ${kb(budget)} ${kind === 'entry' ? 'entry' : 'per-demo'} budget ` +
`by ${kb(gz - budget)}.`,
);
}
rows.push([
`assets/${name}`,
kind,
kb(bytes.length),
kb(gz),
budget === null ? '-' : `${((gz / budget) * 100).toFixed(0)}%`,
over ? red('OVER') : budget === null ? dim('n/a') : green('ok'),
]);
}
console.log(table(['file', 'kind', 'raw', 'gzip', 'of budget', ''], rows));
console.log('');
console.log(dim(`total gzipped: ${kb(totalGzip)} across ${rows.length} assets`));
if (failures.length) {
console.error('');
for (const failure of failures) console.error(`${red('FAIL')} ${failure}`);
console.error('');
console.error(
red('bundle-budget: over budget.') +
'\n A demo that costs 90 kB is a demo somebody closes before it renders. The usual causes, in ' +
'\n order: a chart library pulled into a demo chunk instead of its own tab, an icon set imported ' +
"\n as a namespace rather than by name, and a `?raw` receipt that grew a file it didn't need.",
);
process.exit(1);
}
// A lazy chunk nobody can reach is not a saving, it is dead weight that still
// costs disk and cache. Worth a look, never worth failing a build over.
const unreferenced = rows.filter((row) => row[1] === 'demo/lazy' && !indexHtml.includes(path.basename(row[0])));
if (unreferenced.length > 8) {
console.log(yellow(`warn ${unreferenced.length} lazy chunks. Worth checking none of them are orphaned.`));
}
console.log('');
console.log(green('bundle-budget: within budget.'));
+2 -1
View File
@@ -514,7 +514,8 @@ function probeAdapter(modulePath, traces) {
const probe = path.join(fs.mkdtempSync(path.join(os.tmpdir(), 'pig-adapt-')), 'probe.mts');
fs.writeFileSync(probe, adapterProbeSource(), 'utf8');
try {
const run = spawnSync(process.execPath, [tsx, probe, modulePath, ...traces], {
// The tsx bin is a POSIX shell shim, not a JS file: run it directly.
const run = spawnSync(tsx, [probe, modulePath, ...traces], {
cwd: abs('.'),
encoding: 'utf8',
timeout: 30_000,
+276
View File
@@ -0,0 +1,276 @@
#!/usr/bin/env node
/**
* The cross-language conformance gate.
*
* `envs/wordle_five/wordle_five/engine.py` is the reference implementation and
* `src/demos/wordle/engine.ts` is a port of it. The site's entire argument is
* that the board you play in the browser is the same environment the model was
* evaluated in, so the two implementations agreeing is not a nice-to-have — it
* is the claim.
*
* A hand-written vector file would only ever catch the cases somebody thought
* of, and the case nobody thinks of is always the same one: repeated letters,
* where a green must claim its letter before any yellow is assigned. So the
* gate is exhaustive instead. For every answer A in `words/answers.json`, in
* list order, concatenate `score(G, A)` for every guess G in that same list, in
* that same order, and stream the whole thing through SHA-256. That is the
* digest `engine.py`'s `conformance_digest()` computes, and it is the digest
* this script computes from the TypeScript.
*
* Both the answer list order and the two nested loop orders are part of the
* definition. Sorting the list, deduplicating it, or swapping the loops all
* produce a different, equally valid, completely useless number.
*
* Usage:
* node scripts/conformance.mjs full digest, compared to CONFORMANCE.txt
* node scripts/conformance.mjs --limit 50 first 50 answers only, no comparison
*
* Exits 0 with a note (not an error) when `engine.ts` does not exist yet — the
* TypeScript port and this script are allowed to arrive in either order.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { spawnSync } from 'node:child_process';
import { abs, cyan, die, dim, exists, green, read, red, rel } from './_lib.mjs';
const ANSWERS = abs('envs', 'wordle_five', 'words', 'answers.json');
const EXPECTED_FILE = abs('envs', 'wordle_five', 'CONFORMANCE.txt');
const limitFlag = process.argv.indexOf('--limit');
const limit = limitFlag === -1 ? 0 : Number(process.argv[limitFlag + 1] ?? 0);
if (limitFlag !== -1 && (!Number.isInteger(limit) || limit <= 0)) {
die('--limit takes a positive integer, e.g. `--limit 50`.');
}
/* ------------------------------------------------------ find the TS engine */
function findEngine() {
const preferred = abs('src', 'demos', 'wordle', 'engine.ts');
if (exists(preferred)) return preferred;
const demosDir = abs('src', 'demos');
if (!exists(demosDir)) return null;
const found = fs
.readdirSync(demosDir, { withFileTypes: true })
.filter((e) => e.isDirectory() && !e.name.startsWith('_'))
.map((e) => path.join(demosDir, e.name, 'engine.ts'))
.filter(exists);
// More than one engine.ts and no `wordle` directory means there is nothing to
// guess between; say so rather than silently conforming the wrong one.
return found.length === 1 ? found[0] : null;
}
const engine = findEngine();
if (!engine) {
console.log(dim('conformance: engine not present yet — no src/demos/wordle/engine.ts to check.'));
console.log(dim('The TypeScript port and this gate may land in either order; re-run once it exists.'));
process.exit(0);
}
if (!exists(ANSWERS)) die(`${rel(ANSWERS)} does not exist. It defines both the pairs and their order.`);
if (!exists(abs('node_modules', '.bin', 'tsx'))) die('tsx is not installed, and the engine is TypeScript. Run `pnpm install`.');
/* ------------------------------------------------------------------- run it */
const probeDir = fs.mkdtempSync(path.join(os.tmpdir(), 'pig-conformance-'));
const probe = path.join(probeDir, 'digest.mts');
fs.writeFileSync(probe, probeSource(), 'utf8');
const started = Date.now();
console.log(dim(`conformance: hashing every (guess, answer) pair through ${rel(engine)}...`));
// The tsx bin is a POSIX shell shim, not a JS file, so it is the executable
// here rather than an argument to node.
const run = spawnSync(abs('node_modules', '.bin', 'tsx'), [probe, engine, ANSWERS, String(limit)], {
cwd: abs('.'),
encoding: 'utf8',
maxBuffer: 16 * 1024 * 1024,
timeout: 15 * 60_000,
// The probe lives in a temp directory, where tsx would never find the
// repo's tsconfig — and without it the `@/` alias in engine.ts does not
// resolve and the import fails for a reason that looks nothing like the cause.
env: { ...process.env, TSX_TSCONFIG_PATH: abs('tsconfig.json') },
stdio: ['ignore', 'pipe', 'inherit'],
});
fs.rmSync(probeDir, { recursive: true, force: true });
if (run.error) die(`could not run the digest probe: ${run.error.message}`);
if (run.status !== 0) die(`the digest probe exited ${run.status}. See the error above.`);
let result;
try {
result = JSON.parse(String(run.stdout).trim().split('\n').pop());
} catch {
die(`the digest probe produced no parseable result. Output was:\n${run.stdout}`);
}
if (result.error) {
die(
`${rel(engine)}: ${result.error}\n` +
' The gate needs a scoring function it can recognise. Export one named `scoreGuess` (or `score`), ' +
'taking (guess, answer) and returning the pattern — either a "GYXXG" string or an array of ' +
"'exact' | 'present' | 'absent' tiles. Exporting `conformanceDigest()` directly also works.",
);
}
const seconds = ((Date.now() - started) / 1000).toFixed(1);
console.log(` scored ${result.pairs.toLocaleString('en-US')} pairs from ${result.answers.toLocaleString('en-US')} answers in ${seconds}s`);
console.log(` via ${cyan(result.via)}`);
console.log(` digest ${cyan(result.digest)}`);
if (limit) {
console.log('');
console.log(dim(`Partial run (--limit ${limit}): not compared to ${rel(EXPECTED_FILE)}. Run without --limit for the real gate.`));
process.exit(0);
}
/* ----------------------------------------------------------- compare it */
if (!exists(EXPECTED_FILE)) {
console.log('');
console.log(green('conformance: computed, but there is nothing to compare against yet.'));
console.log(`Write the digest to ${rel(EXPECTED_FILE)} once the Python agrees:`);
console.log(dim(` python -c "from wordle_five.engine import conformance_digest; print(conformance_digest())"`));
process.exit(0);
}
const expected = read(EXPECTED_FILE).match(/\b[0-9a-f]{64}\b/)?.[0];
if (!expected) {
die(`${rel(EXPECTED_FILE)} contains no 64-character hex digest.`);
}
if (expected !== result.digest) {
console.error('');
console.error(`${red('FAIL')} the TypeScript engine and ${rel(EXPECTED_FILE)} disagree.`);
console.error(` expected ${expected}`);
console.error(` got ${result.digest}`);
console.error('');
console.error(
' One of the two implementations is wrong, and the board on the site is therefore not the\n' +
' environment the model was evaluated in. The usual culprit is the two-pass scoring: every\n' +
' green must claim its letter out of the pool BEFORE any yellow is assigned, or a guess like\n' +
' SASSY against BASIS marks an S yellow that the greens have already spent.\n' +
' If the Python changed on purpose, re-run its own `conformance_digest()` and update the file.',
);
process.exit(1);
}
console.log('');
console.log(green(`conformance: the TypeScript engine matches ${rel(EXPECTED_FILE)} exactly.`));
/* ------------------------------------------------------------------ probe */
/**
* Runs under tsx so it can import the TypeScript engine directly.
*
* Kept here as a string rather than as a checked-in `.mts`: it is an
* implementation detail of this script, and a stray TypeScript file in
* `scripts/` would be swept into a typecheck it is deliberately outside of.
*
* A function declaration, not a `const`, so it is hoisted above the code that
* writes it to disk near the top of this file.
*/
function probeSource() {
return `
import { createHash } from 'node:crypto';
import { readFileSync } from 'node:fs';
const [enginePath, answersPath, limitRaw] = process.argv.slice(2);
const limit = Number(limitRaw ?? 0);
const say = (value: unknown) => console.log(JSON.stringify(value));
let mod: any;
try {
mod = await import(enginePath);
} catch (error: any) {
say({ error: 'could not import it: ' + String(error?.message ?? error).split('\\n')[0] });
process.exit(0);
}
const answers: string[] = JSON.parse(readFileSync(answersPath, 'utf8'));
const pool = limit > 0 ? answers.slice(0, limit) : answers;
// If the engine ships the digest itself, trust it over anything reconstructed
// here — it is the implementation's own statement of the definition.
if (typeof mod.conformanceDigest === 'function' && limit === 0) {
const digest = await mod.conformanceDigest();
say({ digest, via: 'engine.conformanceDigest()', pairs: pool.length * pool.length, answers: pool.length });
process.exit(0);
}
// Argument order is the Python's: score(guess, answer). Getting it backwards
// produces a valid-looking digest that matches nothing, so if this disagrees
// with CONFORMANCE.txt, check the signature before you check the algorithm.
const NAMES = ['scoreGuess', 'score', 'scorePattern', 'scoreWord', 'feedback', 'pattern'];
const name =
NAMES.find((n) => typeof mod[n] === 'function' && mod[n].length >= 2) ??
NAMES.find((n) => typeof mod[n] === 'function') ??
(typeof mod.default === 'function' ? 'default' : null);
if (!name) {
say({ error: 'no scoring function is exported. Found: ' + Object.keys(mod).join(', ') });
process.exit(0);
}
const score = mod[name];
/** Every spelling of green/yellow/grey this port might reasonably have chosen. */
const TILE: Record<string, string> = {
g: 'G', y: 'Y', x: 'X',
green: 'G', yellow: 'Y', grey: 'X', gray: 'X',
exact: 'G', present: 'Y', absent: 'X',
correct: 'G', misplaced: 'Y', miss: 'X', hit: 'G',
};
function normalise(value: any): string {
if (typeof value === 'string') {
const upper = value.toUpperCase();
if (/^[GYX]+$/.test(upper)) return upper;
const chars = [...value].map((c) => TILE[c.toLowerCase()]);
if (chars.every(Boolean)) return chars.join('');
throw new Error('unrecognised pattern string ' + JSON.stringify(value));
}
if (Array.isArray(value)) {
return value
.map((tile) => {
const key = typeof tile === 'string' ? tile : tile?.state ?? tile?.tile ?? tile?.kind ?? tile?.result ?? tile?.status;
const mapped = TILE[String(key).toLowerCase()];
if (!mapped) throw new Error('unrecognised tile ' + JSON.stringify(tile));
return mapped;
})
.join('');
}
throw new Error('unrecognised pattern ' + JSON.stringify(value));
}
let convert: (value: any) => string;
try {
const sample = score(pool[0], pool[0]);
// 21 million conversions: skip the whole normaliser when the engine already
// speaks the reference alphabet, which is the case that actually ships.
convert = typeof sample === 'string' && /^[GYX]+$/.test(sample) ? (v: any) => v : normalise;
const selfScore = convert(sample);
if (!/^G+$/.test(selfScore)) {
say({ error: 'score(w, w) returned "' + selfScore + '"; a word scored against itself must be all-green' });
process.exit(0);
}
} catch (error: any) {
say({ error: String(error?.message ?? error) });
process.exit(0);
}
const hash = createHash('sha256');
const row = new Array<string>(pool.length);
for (let i = 0; i < pool.length; i += 1) {
const answer = pool[i]!;
for (let j = 0; j < pool.length; j += 1) row[j] = convert(score(pool[j]!, answer));
hash.update(row.join(''));
// Only on a TTY: written to a pipe or a CI log, a carriage return is not a
// cursor move and the progress ends up inline with the result.
if (i % 250 === 0 && process.stderr.isTTY) process.stderr.write(' ' + i + '/' + pool.length + ' answers\\r');
}
if (process.stderr.isTTY) process.stderr.write(' '.repeat(40) + '\\r');
say({ digest: hash.digest('hex'), via: 'engine.' + name + '(guess, answer)', pairs: pool.length * pool.length, answers: pool.length });
`;
}
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env node
/**
* `node scripts/new-demo.mjs <slug>` — scaffold a demo.
*
* Copies `src/demos/_template` to `src/demos/<slug>` and `envs/_template` to
* `envs/<package>`, substitutes the name everywhere, and wires NOTHING. There
* is deliberately no registry to edit, no route to add and no import to insert:
* the registry is an `import.meta.glob`, so a demo exists because its directory
* exists. If you ever find yourself adding a line to a shared file to make a new
* demo appear, that is a bug in the registry, not a missing step here.
*
* ── SUBSTITUTIONS ──────────────────────────────────────────────────────────
*
* For `new-demo.mjs wordle-five`, in every copied file's CONTENT and in its
* PATH:
*
* __slug__ wordle-five the directory name, meta.slug, the URL
* __package__ wordle_five the Python package under envs/
* __Title__ Wordle Five a starting point for meta.title
* __Pascal__ WordleFive component and type names
* __camel__ wordleFive variable names
* __SLUG__ WORDLE_FIVE constants
* _template wordle-five (wordle_five under envs/)
*
* Anything still matching `__word__` after the copy is reported, because a
* template token that survives into a real demo compiles fine and ships a
* placeholder to production.
*
* The slug must be kebab-case, must not already exist, and must not collide
* with a top-level route.
*/
import fs from 'node:fs';
import path from 'node:path';
import { abs, cyan, dim, exists, green, red, rel } from './_lib.mjs';
const KEBAB = /^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$/;
/** Top-level paths the router already owns. A demo here would be unreachable. */
const RESERVED = new Set(['gallery', 'demos', 'verticals', 'honesty', 'assets', 'og', 'traces', 'fonts', 'icons']);
const slug = process.argv[2];
if (!slug || slug === '--help' || slug === '-h') {
console.log('usage: node scripts/new-demo.mjs <slug>\n');
console.log(' <slug> kebab-case, e.g. support-refund-triage. Becomes the directory name,');
console.log(' meta.slug and the URL /demos/<slug>.');
process.exit(slug ? 0 : 1);
}
const fail = (message, hint) => {
console.error(`${red('FAIL')} ${message}`);
if (hint) console.error(` ${hint}`);
process.exit(1);
};
if (!KEBAB.test(slug)) {
fail(
`"${slug}" is not kebab-case.`,
'Lower-case letters and digits, single hyphens between words, starting with a letter. ' +
'It is the URL, the directory name and the registry key, so it has exactly one spelling.',
);
}
if (RESERVED.has(slug)) {
fail(`"${slug}" is a reserved path.`, `The router already owns /${slug}. Reserved: ${[...RESERVED].join(', ')}.`);
}
const pkg = slug.replace(/-/g, '_');
const words = slug.split('-');
const substitutions = [
['__package__', pkg],
['__Pascal__', words.map(capitalise).join('')],
['__camel__', words.map((w, i) => (i === 0 ? w : capitalise(w))).join('')],
['__Title__', words.map(capitalise).join(' ')],
['__SLUG__', pkg.toUpperCase()],
['__slug__', slug],
];
const jobs = [
{ from: abs('src', 'demos', '_template'), to: abs('src', 'demos', slug), bareName: slug },
{ from: abs('envs', '_template'), to: abs('envs', pkg), bareName: pkg },
];
for (const job of jobs) {
if (!exists(job.from)) {
fail(
`${rel(job.from)} does not exist.`,
'The template is the contract in worked-example form; scaffolding from nothing would produce a demo ' +
'that satisfies no rule in scripts/check-demos.mjs.',
);
}
if (exists(job.to)) {
fail(
`${rel(job.to)} already exists.`,
'Refusing to overwrite. Pick another slug, or delete the directory yourself if you meant to start over.',
);
}
}
/* -------------------------------------------------------------------- copy */
const written = [];
const leftovers = new Map();
for (const job of jobs) {
copyTree(job.from, job.to, job.bareName);
}
console.log(green(`Created ${written.length} files:`));
for (const file of written) console.log(dim(` ${rel(file)}`));
if (leftovers.size > 0) {
console.log('');
console.log(red('Unsubstituted template tokens remain:'));
for (const [token, files] of leftovers) {
console.log(` ${token} in ${[...files].map(rel).join(', ')}`);
}
console.log('');
console.log('Either add the token to the substitution table in scripts/new-demo.mjs, or fix the template.');
process.exit(1);
}
console.log('');
console.log(`Next, in ${cyan(rel(abs('src', 'demos', slug)))}:`);
console.log(' 1. meta.ts — title, tagline, vertical, persona, rewardLine, the lucide icon name');
console.log(' 2. demo.tsx — the narrative, the anatomy, the reward, and the Surface');
console.log(` 3. ${rel(abs('envs', pkg))} — the environment the reward quotes`);
console.log('');
console.log(`Then ${cyan('pnpm check')}. Nothing else needs editing — the registry finds the demo by existence.`);
console.log(dim(`The demo will 404 until meta.ts, demo.tsx and public/og/${slug}.png all exist.`));
/* ----------------------------------------------------------------- helpers */
function capitalise(word) {
return word.charAt(0).toUpperCase() + word.slice(1);
}
/** Applies the substitution table, longest token first so prefixes cannot win. */
function substitute(text, bareName) {
let out = text;
for (const [token, value] of substitutions) out = out.split(token).join(value);
// `_template` is what the directory is literally called, so it turns up in
// relative imports and in the Python package name inside pyproject.toml.
return out.split('_template').join(bareName);
}
function copyTree(from, to, bareName) {
fs.mkdirSync(to, { recursive: true });
for (const entry of fs.readdirSync(from, { withFileTypes: true })) {
if (entry.name === '__pycache__' || entry.name === 'node_modules') continue;
const source = path.join(from, entry.name);
const target = path.join(to, substitute(entry.name, bareName));
if (entry.isDirectory()) {
copyTree(source, target, bareName);
continue;
}
if (!entry.isFile()) continue;
// Binary files (a fixture PNG in the template, say) are copied byte for
// byte. Running them through the string substitution would corrupt them.
if (isBinary(source)) {
fs.copyFileSync(source, target);
written.push(target);
continue;
}
const body = substitute(fs.readFileSync(source, 'utf8'), bareName);
fs.writeFileSync(target, body, 'utf8');
written.push(target);
for (const match of body.matchAll(/__[A-Za-z][A-Za-z0-9]*__/g)) {
if (!leftovers.has(match[0])) leftovers.set(match[0], new Set());
leftovers.get(match[0]).add(target);
}
}
}
function isBinary(file) {
return /\.(png|jpe?g|gif|webp|avif|ico|woff2?|ttf|otf|pdf|zip|gz)$/i.test(file);
}
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env node
/**
* Renders the social card for every route.
*
* One 1200x630 PNG per route, screenshotted from the real built page, written
* to `public/og/`. Not generated from a template: a card drawn by a separate
* renderer drifts from the page it advertises, and this way a change to the
* board or the palette shows up in the card by construction.
*
* ── ORDERING ───────────────────────────────────────────────────────────────
*
* This reads `dist/` and writes `public/`, which means the cards are always one
* build behind until you build again:
*
* pnpm build && node scripts/og.mjs && pnpm build
*
* That is deliberate. The alternative — screenshotting the dev server — renders
* unminified CSS and a different font-loading path, and the cards came out
* subtly wrong in a way nobody noticed until they were on X.
*
* ── WHERE THIS RUNS ────────────────────────────────────────────────────────
*
* amd-server (x86), where Playwright's chromium is installed. It is NOT part of
* the deploy: the deploy host has no browser, and a deploy that silently
* skipped card generation would ship a site whose every link previews as a
* broken image. Run it here, commit the PNGs, deploy the PNGs.
*
* Env:
* PIG_OG_SCALE device pixel ratio, default 2 (so the file is 2400x1260 for
* a 1200x630 CSS-pixel card; every unfurler downsamples, and
* 1x text on a retina timeline looks like a fax).
*/
import fs from 'node:fs';
import path from 'node:path';
import { abs, die, dim, exists, expandRoutes, green, loadAllMetas, loadVerticals, rel, serveStatic, table } from './_lib.mjs';
const WIDTH = 1200;
const HEIGHT = 630;
const SCALE = Number(process.env.PIG_OG_SCALE ?? 2);
const DIST = abs('dist');
const OUT_DIR = abs('public', 'og');
if (!exists(path.join(DIST, 'index.html'))) {
die(`${rel(DIST)}/index.html does not exist. Run \`pnpm build\` first — the cards are shot from the built site.`);
}
const { chromium } = await import('playwright').catch(() => {
die('playwright is not installed. `pnpm install` first.');
});
const executable = chromium.executablePath();
if (!exists(executable)) {
die(
`Playwright's chromium is not installed at ${executable}.\n` +
' Run this on amd-server (x86), where it is installed, and commit the PNGs. ' +
'The deploy host has no browser, which is exactly why card generation is not part of the deploy.',
);
}
/* ------------------------------------------------------- routes -> filenames */
const { metas } = loadAllMetas();
const verticals = loadVerticals();
if (verticals.error) die(`could not read the verticals: ${verticals.error}`);
const { routes, errors } = expandRoutes({ metas, verticals: verticals.list });
if (errors.length) die(`route enumeration failed:\n - ${errors.join('\n - ')}`);
/**
* A demo's filename comes from its own `meta.ogImage`, not from its slug.
*
* `check-demos` rule 4 asserts that file exists; if this script invented a
* different name, the two would disagree and the check would fail on a card
* that had just been generated.
*/
function outputName(route) {
if (route.kind === 'home') return 'home.png';
if (route.kind === 'demo') {
const declared = metas.get(route.slug)?.ogImage;
return typeof declared === 'string' && declared.trim() !== '' ? path.basename(declared) : `${route.slug}.png`;
}
// Prefixed, because a vertical slug and a demo slug live in the same
// directory and nothing stops them colliding.
if (route.kind === 'vertical') return `vertical-${route.id}.png`;
return `${route.path.replace(/^\/+/, '').replace(/\//g, '-') || 'home'}.png`;
}
const targets = routes.map((route) => ({ route, name: outputName(route) }));
const byName = new Map();
for (const target of targets) {
const clash = byName.get(target.name);
if (clash) {
die(
`${target.route.path} and ${clash.path} would both write public/og/${target.name}. ` +
'Change one of their `meta.ogImage` values; a shared card means one of the two pages advertises the other.',
);
}
byName.set(target.name, target.route);
}
/* --------------------------------------------------------------- screenshot */
fs.mkdirSync(OUT_DIR, { recursive: true });
const server = await serveStatic(DIST);
const browser = await chromium.launch();
const context = await browser.newContext({
viewport: { width: WIDTH, height: HEIGHT },
deviceScaleFactor: SCALE,
colorScheme: 'light',
// Every animation on the site is an entrance. Shooting mid-flight catches
// elements at 40% opacity, which reads as a rendering bug in the card.
reducedMotion: 'reduce',
});
const rows = [];
try {
for (const { route, name } of targets) {
const page = await context.newPage();
await page.goto(`${server.origin}${route.path}`, { waitUntil: 'load', timeout: 30_000 });
await page.waitForSelector('#root > *', { timeout: 30_000 }).catch(() => {});
await page.waitForLoadState('networkidle', { timeout: 30_000 }).catch(() => {});
await page.evaluate(() => document.fonts.ready.then(() => true)).catch(() => {});
const file = path.join(OUT_DIR, name);
// `clip` rather than fullPage: the card is a 1200x630 window onto the top
// of the page, and fullPage would hand the unfurler a 1200x9000 strip that
// every platform crops to something arbitrary.
await page.screenshot({ path: file, clip: { x: 0, y: 0, width: WIDTH, height: HEIGHT } });
await page.close();
rows.push([route.path, `public/og/${name}`, `${(fs.statSync(file).size / 1024).toFixed(0)} kB`]);
}
} finally {
await context.close();
await browser.close();
await server.close();
}
console.log(table(['route', 'card', 'size'], rows));
console.log('');
console.log(green(`og: wrote ${rows.length} cards at ${WIDTH}x${HEIGHT} CSS px (x${SCALE}).`));
console.log(dim('Rebuild before deploying, or dist/ still holds the previous cards.'));
+96
View File
@@ -0,0 +1,96 @@
#!/usr/bin/env node
/**
* `dist/sitemap.xml` and `robots.txt`.
*
* PIG's own site is `noindex` — it is a private tool. This one is the exact
* inverse: it exists to be found, shared and quoted, so the robots file opens
* everything and points at a sitemap listing every route the router actually
* produces.
*
* The route list is the same one `prerender.mjs` bakes, from the same reader,
* so a sitemap entry cannot point at a URL that has no prerendered document.
* Those two drifting apart is how you end up submitting 14 URLs and having 13
* of them indexed as the homepage.
*
* `robots.txt` is written into `public/` (source, committed) AND into `dist/`
* (because this runs after the build that would have copied it). The sitemap is
* a build artifact only — it lists what this build contains.
*/
import fs from 'node:fs';
import path from 'node:path';
import { SITE_ORIGIN, abs, die, exists, expandRoutes, green, loadAllMetas, loadVerticals, rel } from './_lib.mjs';
const DIST = abs('dist');
const PUBLIC = abs('public');
const { metas } = loadAllMetas();
const verticals = loadVerticals();
if (verticals.error) die(`could not read the verticals: ${verticals.error}`);
const { routes, errors } = expandRoutes({ metas, verticals: verticals.list });
if (errors.length) die(`route enumeration failed:\n - ${errors.join('\n - ')}`);
/* ------------------------------------------------------------- robots.txt */
const robots = [
'# demo.primeintellectgrowth.com',
'#',
'# This site is meant to be found. Every page is public, static and safe to',
'# crawl; the source it documents is public too.',
'',
'User-agent: *',
'Allow: /',
'',
`Sitemap: ${SITE_ORIGIN}/sitemap.xml`,
'',
].join('\n');
fs.mkdirSync(PUBLIC, { recursive: true });
fs.writeFileSync(path.join(PUBLIC, 'robots.txt'), robots, 'utf8');
/* ------------------------------------------------------------ sitemap.xml */
if (!exists(path.join(DIST, 'index.html'))) {
die(
`${rel(DIST)}/index.html does not exist, so there is no build to describe. ` +
'Run `pnpm build` first. (public/robots.txt has been written.)',
);
}
// One date for the whole build. Per-file mtimes would claim the vertical pages
// changed whenever anything in the bundle did, which is true of a hash-named
// asset and useless to a crawler.
const lastmod = new Date().toISOString().slice(0, 10);
const xml = [
'<?xml version="1.0" encoding="UTF-8"?>',
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">',
...routes.map((route) =>
[
' <url>',
` <loc>${escapeXml(`${SITE_ORIGIN}${route.path === '/' ? '/' : route.path}`)}</loc>`,
` <lastmod>${lastmod}</lastmod>`,
' </url>',
].join('\n'),
),
'</urlset>',
'',
].join('\n');
fs.writeFileSync(path.join(DIST, 'sitemap.xml'), xml, 'utf8');
fs.writeFileSync(path.join(DIST, 'robots.txt'), robots, 'utf8');
console.log(green(`sitemap: ${routes.length} URLs -> ${rel(path.join(DIST, 'sitemap.xml'))}`));
console.log(green(`robots: ${rel(path.join(PUBLIC, 'robots.txt'))} and ${rel(path.join(DIST, 'robots.txt'))}`));
/** Sitemap URLs are XML text; a bare `&` in a query string invalidates the file. */
function escapeXml(value) {
return value
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&apos;');
}
+700
View File
@@ -0,0 +1,700 @@
import { useCallback, useEffect, useMemo, useState } from 'react';
import { useParams } from 'react-router-dom';
import * as Tabs from '@radix-ui/react-tabs';
import { useUrlState } from '@/lib/url-state';
import type {
DemoEpisode,
DemoModule,
DemoStep,
RunRef,
StoryBeat,
} from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
import { BeatSection } from './BeatSection';
import { BlindCompare } from './BlindCompare';
import { CodeReceipt } from './CodeReceipt';
import { DemoErrorBoundary } from './DemoErrorBoundary';
import { EnvAnatomy } from './EnvAnatomy';
import { LimitsCallout } from './LimitsCallout';
import { MetricMover } from './MetricMover';
import { ModelCallPanel } from './ModelCallPanel';
import { ProvenanceCard } from './ProvenanceCard';
import { ReasoningDrawer } from './ReasoningDrawer';
import { ReasoningPanel } from './ReasoningPanel';
import { RewardBreakdown } from './RewardBreakdown';
import { RewardEditor } from './RewardEditor';
import type { RewardArm } from './RewardEditor';
import { SlotRegion } from './SlotRegion';
import { StatStrip } from './StatStrip';
import type { Stat } from './StatStrip';
import { StepTimeline } from './StepTimeline';
import { RecordedBadge, TracePlayer, useTracePlayback } from './TracePlayer';
import { VerifyBadge } from './VerifyBadge';
import { clampIndex, formatOrDash, useIsDesktop } from './format';
import { scoreReward } from './reward-math';
import { mockDemo, mockEpisodes, mockRuns } from './mock';
const REPO_BLOB = 'https://github.com/karti-ai/PIG-Demo/blob/main/';
const MANIFEST_URL = '/traces/manifest.json';
/**
* Every demo module in the repo, as an unresolved import each.
*
* `import.meta.glob` rather than a generated registry import on purpose: this
* file must compile and render before any demo directory exists, and a glob
* that matches nothing is an empty object rather than a build error.
*/
const DEMO_MODULES = import.meta.glob<Record<string, unknown>>('/src/demos/*/index.{ts,tsx}');
export interface DemoBundle<T = unknown> {
demo: DemoModule<T>;
runs: RunRef[];
episodes: Record<string, DemoEpisode>;
}
type LoadState<T> =
| { status: 'loading' }
| { status: 'ready'; bundle: DemoBundle<T> }
| { status: 'error'; message: string };
function isRunRef(value: unknown): value is RunRef {
if (typeof value !== 'object' || value === null) return false;
const run = value as Record<string, unknown>;
return (
typeof run['id'] === 'string' &&
typeof run['label'] === 'string' &&
typeof run['path'] === 'string' &&
typeof run['model'] === 'string'
);
}
function isEpisode(value: unknown): value is DemoEpisode {
if (typeof value !== 'object' || value === null) return false;
const episode = value as Record<string, unknown>;
return (
typeof episode['runId'] === 'string' &&
Array.isArray(episode['turns']) &&
typeof episode['rewards'] === 'object' &&
episode['rewards'] !== null
);
}
/**
* The manifest is data on disk, not a typed import, so it is validated rather
* than trusted — and three plausible shapes are accepted because the file is
* written by a script in another lane and a keyed map, a nested map and a flat
* list are all reasonable things for that script to have produced.
*/
export function extractRuns(json: unknown, slug: string): RunRef[] {
if (typeof json !== 'object' || json === null) return [];
const root = json as Record<string, unknown>;
const nested = root['demos'];
const keyed =
(Array.isArray(root[slug]) ? root[slug] : undefined) ??
(typeof nested === 'object' && nested !== null
? (nested as Record<string, unknown>)[slug]
: undefined);
if (Array.isArray(keyed)) return keyed.filter(isRunRef);
const flat = Array.isArray(root['runs']) ? root['runs'] : Array.isArray(json) ? json : null;
if (flat) {
return flat.filter(isRunRef).filter((run) => {
const owner = (run as unknown as Record<string, unknown>)['demo'];
return owner === undefined || owner === slug;
});
}
return [];
}
function pickModule(mod: Record<string, unknown>): DemoModule | null {
const candidate = mod['default'] ?? mod['demo'];
if (typeof candidate !== 'object' || candidate === null) return null;
const shape = candidate as Record<string, unknown>;
return typeof shape['adapt'] === 'function' && typeof shape['Surface'] === 'function'
? (candidate as DemoModule)
: null;
}
async function loadBundle(slug: string): Promise<DemoBundle> {
if (slug === '__mock') {
return { demo: mockDemo as unknown as DemoModule, runs: mockRuns, episodes: mockEpisodes };
}
const entry = Object.entries(DEMO_MODULES).find(([path]) =>
path.startsWith(`/src/demos/${slug}/index.`),
);
if (!entry) throw new Error(`No demo is registered under the slug "${slug}".`);
const demo = pickModule(await entry[1]());
if (!demo) {
throw new Error(`The module for "${slug}" does not export a demo that satisfies the contract.`);
}
const manifest = await fetch(MANIFEST_URL, { cache: 'no-cache' })
.then((response) => (response.ok ? response.json() : null))
.catch(() => null);
const runs = extractRuns(manifest, slug);
// One unreadable trace must not blank the page: fetch them all, keep the
// ones that parse, and let the shell report the shortfall.
const loaded = await Promise.all(
runs.map(async (run) => {
try {
const response = await fetch(run.path, { cache: 'no-cache' });
if (!response.ok) return null;
const json: unknown = await response.json();
return isEpisode(json) ? ([run.id, json] as const) : null;
} catch {
return null;
}
}),
);
const episodes: Record<string, DemoEpisode> = {};
for (const item of loaded) {
if (item) episodes[item[0]] = item[1];
}
return { demo, runs: runs.filter((run) => episodes[run.id] !== undefined), episodes };
}
export interface DemoShellProps<T = unknown> {
/** Overrides the route param. Useful for previews and tests. */
slug?: string;
/** Skips loading entirely when the caller already has the bundle. */
bundle?: DemoBundle<T>;
}
/**
* The route component every demo is rendered through.
*
* It owns four things and no more: loading, the narrative beats, the URL state,
* and the page's single polite live region. Everything visual is delegated to
* the surfaces in this directory, and the demo module is never reached into —
* the shell is generic over the demo's board type and only ever calls `adapt`
* and renders `Surface`.
*/
export function DemoShell<T = unknown>({ slug: slugProp, bundle }: DemoShellProps<T>) {
const params = useParams();
const slug = slugProp ?? params['slug'] ?? '';
const [state, setState] = useState<LoadState<T>>(
bundle ? { status: 'ready', bundle } : { status: 'loading' },
);
useEffect(() => {
if (bundle) {
setState({ status: 'ready', bundle });
return;
}
let live = true;
setState({ status: 'loading' });
loadBundle(slug)
.then((loaded) => {
if (live) setState({ status: 'ready', bundle: loaded as DemoBundle<T> });
})
.catch((error: unknown) => {
if (!live) return;
setState({
status: 'error',
message: error instanceof Error ? error.message : String(error),
});
});
return () => {
live = false;
};
}, [slug, bundle]);
if (state.status === 'loading') return <ShellSkeleton />;
if (state.status === 'error') {
return (
<div role="alert" className="card mx-auto my-16 max-w-xl p-6">
<h1 className="text-lg font-semibold">That demo is not here</h1>
<p className="mt-2 text-sm leading-relaxed text-muted">{state.message}</p>
<a
href="/"
className="tap mt-4 inline-flex items-center rounded-lg border border-border px-4 text-sm font-medium hover:bg-surface-2"
>
Back to the gallery
</a>
</div>
);
}
return (
<DemoErrorBoundary demoTitle={state.bundle.demo.meta.title}>
<DemoBody bundle={state.bundle} />
</DemoErrorBoundary>
);
}
function DemoBody<T>({ bundle }: { bundle: DemoBundle<T> }) {
const { demo, runs, episodes } = bundle;
const isDesktop = useIsDesktop();
// The shell writes `?step=` on every advance, including during playback. It
// is `@/lib/url-state`'s job to REPLACE rather than push for these — pushing
// would turn a six-step run into six back-button presses.
const [runParam, setRunParam] = useUrlState('run', '');
const [stepParam, setStepParam] = useUrlState('step', '0');
const [tabParam, setTabParam] = useUrlState('tab', '');
const run = useMemo(
() => runs.find((candidate) => candidate.id === runParam) ?? runs[0],
[runs, runParam],
);
const episode = run ? episodes[run.id] : undefined;
const steps: DemoStep<T>[] = useMemo(
() => (episode ? (demo.adapt(episode) as DemoStep<T>[]) : []),
[demo, episode],
);
const step = clampIndex(Number(stepParam), steps.length);
const setStep = useCallback(
(next: number) => setStepParam(String(clampIndex(next, steps.length))),
[setStepParam, steps.length],
);
const playback = useTracePlayback({ stepCount: steps.length, step, onStepChange: setStep });
const current = steps[step];
const hasBeat = (surface: StoryBeat['surface']) =>
demo.narrative.beats.some((beat) => beat.surface === surface);
const timelineInSplit = !hasBeat('scrubber');
const extras = demo.tabs ?? [];
const extrasInCustom = hasBeat('custom');
const tabIds = useMemo(() => {
const ids = isDesktop ? ['reasoning', 'call'] : ['call'];
if (!extrasInCustom) ids.push(...extras.map((tab) => tab.id));
return ids;
}, [isDesktop, extras, extrasInCustom]);
const activeTab = tabIds.includes(tabParam) ? tabParam : (tabIds[0] ?? 'call');
const arms: RewardArm[] = useMemo(
() =>
runs.map((candidate) => {
const armEpisode = episodes[candidate.id];
const arm: RewardArm = {
id: candidate.id,
label: candidate.label,
values: armEpisode?.rewards ?? {},
};
if (candidate.intervention) arm.note = candidate.intervention;
return arm;
}),
[runs, episodes],
);
const totals = useMemo(
() =>
arms.map((arm) => ({
label: arm.label,
total: scoreReward(demo.reward, arm.values).total,
})),
[arms, demo.reward],
);
const blindPair = useMemo(() => {
for (let i = 0; i < runs.length; i += 1) {
for (let j = i + 1; j < runs.length; j += 1) {
const left = runs[i];
const right = runs[j];
if (!left || !right || left.seed !== right.seed) continue;
const leftEpisode = episodes[left.id];
const rightEpisode = episodes[right.id];
if (!leftEpisode || !rightEpisode) continue;
return { left, right, leftEpisode, rightEpisode };
}
}
return null;
}, [runs, episodes]);
if (!run || !episode || steps.length === 0) {
return (
<div className="mx-auto max-w-canvas px-4 py-16">
<h1 className="text-lg font-semibold">{demo.meta.title}</h1>
<p className="mt-2 max-w-prose text-sm leading-relaxed text-muted">
No recorded run is available for this demo yet. The environment and its grader are in
the repository; the traces are produced by the eval command on the demo's provenance
card.
</p>
<EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} className="mt-8" />
</div>
);
}
const Surface = demo.Surface as unknown as React.ComponentType<{
state: T;
compact?: boolean;
}>;
const heroStats: Stat[] = [
{
label: 'Outcome',
value: episode.outcome,
tone: episode.outcome === 'solved' ? 'positive' : 'warning',
hint: episode.truncated ? 'Truncated before a terminal state' : undefined,
},
{
label: 'Total reward',
value: formatOrDash(scoreReward(demo.reward, episode.rewards).total),
tone: 'brand',
hint: 'Shipped weights',
},
{ label: 'Steps', value: steps.length, hint: 'Model calls in this run' },
{ label: 'Seed', value: episode.seed, hint: 'Same seed reproduces this board' },
];
const reasoningPanel = (
<ReasoningPanel
reasoning={current?.reasoning ?? null}
durationMs={current?.call?.durationMs ?? null}
playing={playback.playing}
speed={playback.speed}
stepIndex={step}
/>
);
const timeline = (
<StepTimeline
steps={steps}
current={step}
onSelect={(next) => {
playback.setPlaying(false);
setStep(next);
}}
Surface={Surface}
onTogglePlay={playback.toggle}
/>
);
const renderSurface = (beat: StoryBeat) => {
switch (beat.surface) {
case 'hero':
return (
<div className="grid gap-4 lg:grid-cols-[auto_minmax(0,1fr)] lg:items-start">
<div className="card w-fit p-4">
<Surface state={(steps[steps.length - 1] as DemoStep<T>).state} />
</div>
<div className="space-y-3">
<StatStrip stats={heroStats} />
<RecordedBadge
model={run.model}
capturedAt={run.capturedAt}
{...(run.intervention ? { intervention: run.intervention } : {})}
className="ml-0 w-fit"
/>
<p className="max-w-prose text-sm leading-relaxed text-muted">
{demo.narrative.thesis}
</p>
</div>
</div>
);
case 'anatomy':
return <EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} />;
case 'split-play':
return (
<div className="space-y-3">
{runs.length > 1 ? (
<RunSwitcher
runs={runs}
activeId={run.id}
onSelect={(id) => {
playback.setPlaying(false);
setRunParam(id);
setStepParam('0');
}}
/>
) : null}
<TracePlayer
playing={playback.playing}
onPlayingChange={playback.setPlaying}
speed={playback.speed}
onSpeedChange={playback.setSpeed}
onRestart={playback.restart}
step={step}
stepCount={steps.length}
onStepChange={(next) => {
playback.setPlaying(false);
setStep(next);
}}
model={run.model}
capturedAt={run.capturedAt}
{...(run.intervention ? { intervention: run.intervention } : {})}
/>
<div className="grid gap-3 lg:grid-cols-[auto_minmax(0,1fr)] lg:items-start">
<div className="card w-fit p-4">
{current ? <Surface state={current.state} /> : null}
</div>
<div className="min-w-0 space-y-3">
{!isDesktop ? (
<ReasoningDrawer
reasoning={current?.reasoning ?? null}
durationMs={current?.call?.durationMs ?? null}
playing={playback.playing}
speed={playback.speed}
stepIndex={step}
/>
) : null}
<Tabs.Root value={activeTab} onValueChange={setTabParam}>
<Tabs.List
aria-label="Details for this step"
className="flex gap-1 overflow-x-auto rounded-lg bg-surface-2 p-1"
>
{isDesktop ? <TabTrigger value="reasoning">Reasoning</TabTrigger> : null}
<TabTrigger value="call">Model call</TabTrigger>
{!extrasInCustom
? extras.map((tab) => (
<TabTrigger key={tab.id} value={tab.id}>
{tab.label}
</TabTrigger>
))
: null}
</Tabs.List>
{isDesktop ? (
<Tabs.Content value="reasoning" className="mt-3 focus-visible:outline-none">
{reasoningPanel}
</Tabs.Content>
) : null}
<Tabs.Content value="call" className="mt-3 focus-visible:outline-none">
<ModelCallPanel call={current?.call ?? null} />
{current?.reply ? (
<div className="card mt-3 p-3">
<h3 className="text-sm font-semibold">Reply</h3>
<p className="mt-1 whitespace-pre-wrap font-mono text-xs leading-relaxed">
{current.reply}
</p>
</div>
) : null}
</Tabs.Content>
{!extrasInCustom
? extras.map((tab) => (
<Tabs.Content
key={tab.id}
value={tab.id}
className="mt-3 focus-visible:outline-none"
>
<tab.Component />
</Tabs.Content>
))
: null}
</Tabs.Root>
</div>
</div>
{timelineInSplit ? timeline : null}
<SlotRegion id="below-board" />
</div>
);
case 'scrubber':
return (
<div className="space-y-3">
{timeline}
<SlotRegion id="below-timeline" />
</div>
);
case 'reward-editor':
return (
<div className="space-y-4">
<RewardBreakdown
spec={demo.reward}
values={episode.rewards}
{...(episode.metrics ? { metrics: episode.metrics } : {})}
/>
<VerifyBadge demo={demo} episode={episode} />
{arms.length > 1 ? <RewardEditor spec={demo.reward} arms={arms} /> : null}
</div>
);
case 'metric': {
const currentTotal = scoreReward(demo.reward, episode.rewards).total;
const first = totals[0];
const series = totals
.filter((entry): entry is { label: string; total: number } => entry.total !== null)
.map((entry) => ({ x: entry.label, y: entry.total }));
return (
<div className="space-y-4">
<MetricMover
label={`Total reward — ${run.label}`}
value={currentTotal ?? 0}
{...(first && first.total !== null && first.label !== run.label
? { baseline: { value: first.total, label: first.label } }
: {})}
series={series}
caption={
'Every point is a recorded run scored by the same grader. Nothing here is a projection.'
}
/>
{blindPair ? (
<BlindCompare
seed={blindPair.left.seed}
Surface={Surface}
a={{
runId: blindPair.left.id,
label: blindPair.left.label,
model: blindPair.left.model,
...(blindPair.left.intervention
? { intervention: blindPair.left.intervention }
: {}),
steps: demo.adapt(blindPair.leftEpisode) as DemoStep<T>[],
total: scoreReward(demo.reward, blindPair.leftEpisode.rewards).total,
}}
b={{
runId: blindPair.right.id,
label: blindPair.right.label,
model: blindPair.right.model,
...(blindPair.right.intervention
? { intervention: blindPair.right.intervention }
: {}),
steps: demo.adapt(blindPair.rightEpisode) as DemoStep<T>[],
total: scoreReward(demo.reward, blindPair.rightEpisode.rewards).total,
}}
/>
) : null}
</div>
);
}
case 'receipt':
return (
<div className="grid gap-4 lg:grid-cols-2 lg:items-start">
<ProvenanceCard provenance={demo.provenance} run={run} />
<CodeReceipt
code={demo.reward.source.code}
path={demo.reward.source.path}
{...(demo.reward.source.marker ? { marker: demo.reward.source.marker } : {})}
href={`${REPO_BLOB}${demo.reward.source.path}`}
/>
<SlotRegion id="after-receipts" className="lg:col-span-2" />
</div>
);
case 'limits':
return <LimitsCallout limits={demo.narrative.limits} />;
case 'custom':
return extras.length > 0 ? (
<div className="space-y-4">
{extras.map((tab) => (
<tab.Component key={tab.id} />
))}
</div>
) : (
<SlotRegion id="before-limits" />
);
default:
return null;
}
};
return (
<div className="mx-auto max-w-canvas px-4 pb-16" style={{ paddingBottom: 'var(--safe-bottom)' }}>
{/*
The page's ONE live region. Every step change lands here and nowhere
else: with reduced motion the tile animation is gone, so this sentence
is the only thing that tells a screen-reader user what just happened.
*/}
<div aria-live="polite" aria-atomic="true" className="sr-only">
{current?.announce ?? ''}
</div>
<header className="pt-8">
<p className="text-xs font-semibold uppercase tracking-wide text-accent-fg">
{demo.meta.vertical.replace(/-/g, ' ')} · for {demo.meta.persona}
</p>
<h1 className="mt-1 text-2xl font-semibold tracking-tight lg:text-3xl">
{demo.meta.title}
</h1>
<p className="mt-2 max-w-prose text-base text-muted">{demo.meta.tagline}</p>
<p className="mt-4 max-w-prose border-l-2 border-brand pl-3 text-sm italic leading-relaxed text-fg">
{demo.narrative.anxiety}
</p>
</header>
<div className="divide-y divide-border">
{demo.narrative.beats.map((beat, index) => (
<BeatSection key={beat.id} beat={beat} number={index + 1}>
{renderSurface(beat)}
</BeatSection>
))}
</div>
</div>
);
}
function TabTrigger({ value, children }: { value: string; children: React.ReactNode }) {
return (
<Tabs.Trigger
value={value}
className="tap flex-1 whitespace-nowrap rounded-md px-3 text-sm font-medium text-muted transition-colors duration-2 ease-enter data-[state=active]:bg-surface data-[state=active]:text-fg data-[state=active]:shadow-sm"
>
{children}
</Tabs.Trigger>
);
}
function RunSwitcher({
runs,
activeId,
onSelect,
}: {
runs: RunRef[];
activeId: string;
onSelect: (id: string) => void;
}) {
return (
<div
role="radiogroup"
aria-label="Recorded run"
className="flex flex-wrap gap-1 rounded-lg bg-surface-2 p-1"
>
{runs.map((run) => {
const active = run.id === activeId;
return (
<button
key={run.id}
type="button"
role="radio"
aria-checked={active}
onClick={() => onSelect(run.id)}
className={cn(
'tap rounded-md px-3 text-sm font-medium transition-colors duration-2 ease-enter',
active ? 'bg-surface text-fg shadow-sm' : 'text-muted hover:text-fg',
)}
>
{run.label}
</button>
);
})}
</div>
);
}
/**
* The loading state. Deliberately shaped like the page it becomes, and with no
* spinner: a spinner on this site would imply a live model call, which is the
* one thing the whole page is at pains to say is not happening.
*/
function ShellSkeleton() {
return (
<div className="mx-auto max-w-canvas px-4 py-10" aria-busy="true">
<p className="sr-only">Loading the recorded run.</p>
<div className="h-8 w-64 rounded-lg bg-surface-2" />
<div className="mt-3 h-4 w-96 max-w-full rounded-lg bg-surface-2" />
<div className="mt-10 grid gap-3 lg:grid-cols-4">
{[0, 1, 2, 3].map((index) => (
<div key={index} className="h-28 rounded-xl bg-surface-2" />
))}
</div>
<div className="mt-6 h-64 rounded-xl bg-surface-2" />
</div>
);
}
+92 -97
View File
@@ -1,9 +1,10 @@
import { Scale, Target, Weight } from 'lucide-react';
import type { LucideIcon } from 'lucide-react';
import { decompose } from '@/lib/demo-kit/reward';
import type { RewardComponent, RewardSpec, RewardValues } from '@/lib/demo-kit/types';
import { Badge } from '@/components/ui/badge';
import { cn } from '@/lib/utils';
import { DASH, formatNumber, formatOrDash } from './format';
import { scoreReward } from './reward-math';
import { EditedChip } from './StatStrip';
const ROLE_META: Record<RewardComponent['role'], { label: string; Icon: LucideIcon }> = {
@@ -17,9 +18,12 @@ export interface RewardBreakdownProps {
values: RewardValues;
/** Unweighted diagnostics. Rendered, never summed — the contract is explicit. */
metrics?: Record<string, number | null>;
/** Overridden weights from the editor. Absent means the shipped weights. */
weights?: Record<string, number>;
/** Set when `weights` came from the visitor rather than the environment. */
/**
* Components carrying the weights in force. Pass the output of `reweight()`
* when the visitor has edited them; omit for the shipped reward.
*/
components?: readonly RewardComponent[];
/** Set when `components` came from the visitor rather than the environment. */
edited?: boolean;
className?: string;
}
@@ -36,111 +40,102 @@ export function RewardBreakdown({
spec,
values,
metrics,
weights,
components,
edited = false,
className,
}: RewardBreakdownProps) {
const { rows, total } = scoreReward(spec, values, weights);
const counterweights = rows.filter((row) => row.component.role === 'counterweight');
const inForce = components ?? spec.components;
const { rows, total } = decompose(values, inForce);
const counterweights = rows.filter((row) => row.role === 'counterweight');
return (
<div className={cn('card overflow-hidden', className)}>
<table className="w-full border-collapse text-sm">
<caption className="sr-only">
Reward components, their weights and their contribution to the total score
</caption>
<thead>
<tr className="border-b border-border text-left text-xs uppercase tracking-wide text-muted">
<th scope="col" className="px-3 py-2 font-medium">
Component
</th>
<th scope="col" className="px-2 py-2 text-right font-medium">
Score
</th>
<th scope="col" className="px-2 py-2 text-right font-medium">
Weight
</th>
<th scope="col" className="px-3 py-2 text-right font-medium">
Value
</th>
</tr>
</thead>
<tbody className="divide-y divide-border">
{rows.map((row) => {
const role = ROLE_META[row.component.role];
const isCounterweight = row.component.role === 'counterweight';
return (
<tr
key={row.component.key}
className={cn(isCounterweight && 'bg-accent-subtle/40')}
>
<th scope="row" className="max-w-0 px-3 py-2.5 text-left font-normal">
<span className="flex flex-wrap items-center gap-1.5">
<span className="font-medium text-fg">{row.component.label}</span>
<span
className={cn(
'inline-flex items-center gap-1 rounded-md px-1.5 py-0.5 text-[10px] font-semibold uppercase tracking-wide',
isCounterweight
? 'bg-brand/15 text-accent-fg'
: 'bg-surface-2 text-muted',
)}
>
<role.Icon className="h-3 w-3" aria-hidden="true" />
{role.label}
<div className="overflow-x-auto">
<table className="w-full border-collapse text-sm">
<caption className="sr-only">
Reward components, their weights and their contribution to the total score
</caption>
<thead>
<tr className="border-b border-border text-left text-xs uppercase tracking-wide text-muted">
<th scope="col" className="px-3 py-2 font-medium">
Component
</th>
<th scope="col" className="px-2 py-2 text-right font-medium">
Score
</th>
<th scope="col" className="px-2 py-2 text-right font-medium">
Weight
</th>
<th scope="col" className="px-3 py-2 text-right font-medium">
Value
</th>
</tr>
</thead>
<tbody className="divide-y divide-border">
{rows.map((row) => {
const role = ROLE_META[row.role];
const isCounterweight = row.role === 'counterweight';
return (
<tr key={row.key} className={cn(isCounterweight && 'bg-accent-subtle/40')}>
<th scope="row" className="max-w-[22rem] px-3 py-2.5 text-left font-normal">
<span className="flex flex-wrap items-center gap-1.5">
<span className="font-medium text-fg">{row.label}</span>
<Badge variant={isCounterweight ? 'default' : 'muted'}>
<role.Icon className="size-3" aria-hidden="true" />
{role.label}
</Badge>
</span>
</span>
<span className="mt-0.5 block text-xs leading-snug text-muted">
{row.component.description}
</span>
<span className="mt-0.5 block font-mono text-[11px] text-muted">
{row.component.key}
</span>
</th>
<td className="nums px-2 py-2.5 text-right align-top font-mono">
{row.score === null ? (
<span className="text-muted" title="The environment did not score this run">
not scored
<span className="mt-0.5 block text-xs leading-snug text-muted">
{row.description}
</span>
) : (
formatNumber(row.score)
)}
</td>
<td className="nums px-2 py-2.5 text-right align-top font-mono">
<span className={cn(edited && 'text-accent-fg')}>
{formatNumber(row.weight, 2)}
</span>
</td>
<td className="nums px-3 py-2.5 text-right align-top font-mono font-semibold">
{row.value === null ? DASH : formatNumber(row.value)}
</td>
</tr>
);
})}
</tbody>
<tfoot>
<tr className="border-t-2 border-border bg-surface-2">
<th scope="row" className="px-3 py-2.5 text-left">
<span className="flex items-center gap-2 font-semibold">
Total reward
{edited ? <EditedChip /> : null}
</span>
</th>
<td colSpan={2} />
<td className="nums px-3 py-2.5 text-right font-mono text-base font-semibold">
{total === null ? (
<span className="text-muted">not scored</span>
) : (
formatOrDash(total)
)}
</td>
</tr>
</tfoot>
</table>
<span className="mt-0.5 block font-mono text-[11px] text-muted">{row.key}</span>
</th>
<td className="nums px-2 py-2.5 text-right align-top font-mono">
{row.score === null ? (
<span className="text-muted" title="The environment did not score this run">
not scored
</span>
) : (
formatNumber(row.score)
)}
</td>
<td className="nums px-2 py-2.5 text-right align-top font-mono">
<span className={cn(edited && 'font-semibold text-accent-fg')}>
{formatNumber(row.weight, 2)}
</span>
</td>
<td className="nums px-3 py-2.5 text-right align-top font-mono font-semibold">
{row.value === null ? DASH : formatNumber(row.value)}
</td>
</tr>
);
})}
</tbody>
<tfoot>
<tr className="border-t-2 border-border bg-surface-2">
<th scope="row" className="px-3 py-2.5 text-left">
<span className="flex items-center gap-2 font-semibold">
Total reward
{edited ? <EditedChip /> : null}
</span>
</th>
<td colSpan={2} />
<td className="nums px-3 py-2.5 text-right font-mono text-base font-semibold">
{total === null ? (
<span className="text-muted">not scored</span>
) : (
formatOrDash(total)
)}
</td>
</tr>
</tfoot>
</table>
</div>
{counterweights.length > 0 ? (
<p className="border-t border-border bg-accent-subtle/40 px-3 py-2.5 text-xs leading-relaxed text-fg">
<span className="font-semibold">
{counterweights.map((row) => row.component.label).join(' and ')}
{counterweights.map((row) => row.label).join(' and ')}
</span>{' '}
{counterweights.length > 1 ? 'are counterweights' : 'is the counterweight'}: without a
term pulling the other way, the cheapest way to maximise the objective is a behaviour
+3 -7
View File
@@ -1,4 +1,5 @@
import type { ReactNode } from 'react';
import { Badge } from '@/components/ui/badge';
import { cn } from '@/lib/utils';
export type StatTone = 'default' | 'positive' | 'warning' | 'danger' | 'info' | 'brand';
@@ -79,13 +80,8 @@ export function StatStrip({ stats, className, live = false }: StatStripProps) {
*/
export function EditedChip({ className }: { className?: string }) {
return (
<span
className={cn(
'rounded-md bg-accent-subtle px-1.5 py-0.5 text-[10px] font-semibold uppercase tracking-wide text-accent-fg',
className,
)}
>
<Badge className={cn('px-1.5 py-0 text-[10px] uppercase tracking-wide', className)}>
edited
</span>
</Badge>
);
}
+118 -190
View File
@@ -1,100 +1,9 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { ChevronLeft, ChevronRight, Circle, Pause, Play, RotateCcw } from 'lucide-react';
import { PLAYBACK_SPEEDS } from '@/lib/demo-kit/player';
import type { PlaybackSpeed } from '@/lib/demo-kit/player';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils';
import { formatDate, usePrefersReducedMotion } from './format';
/** `instant` is not "very fast": it is "do not animate, show me the end". */
export type PlaybackSpeed = 1 | 2 | 4 | 'instant';
export const PLAYBACK_SPEEDS: readonly PlaybackSpeed[] = [1, 2, 4, 'instant'];
/** Wall-clock dwell on a step at 1x. Not the model's real latency — see below. */
const BASE_STEP_MS = 1800;
export interface UseTracePlaybackOptions {
stepCount: number;
step: number;
onStepChange: (next: number) => void;
/**
* Dwell time for one step at 1x, in ms. Defaults to a fixed cadence rather
* than the recorded `durationMs`, and that is deliberate: real calls run from
* 300 ms to half a minute, so replaying at true latency produces a player
* that appears frozen. The recorded latency is still shown, verbatim, in the
* model-call panel — it is reported, just not used as a timeline.
*/
stepDurationMs?: (index: number) => number;
initialSpeed?: PlaybackSpeed;
}
export interface TracePlayback {
playing: boolean;
speed: PlaybackSpeed;
setPlaying: (playing: boolean) => void;
setSpeed: (speed: PlaybackSpeed) => void;
toggle: () => void;
restart: () => void;
atEnd: boolean;
}
export function useTracePlayback({
stepCount,
step,
onStepChange,
stepDurationMs,
initialSpeed = 1,
}: UseTracePlaybackOptions): TracePlayback {
const [playing, setPlaying] = useState(false);
const [speed, setSpeedState] = useState<PlaybackSpeed>(initialSpeed);
const atEnd = step >= stepCount - 1;
// The callback identity changes on every render of the shell; holding it in a
// ref keeps it out of the timer effect's deps, or the timer restarts on every
// render and the step never lands.
const onStepChangeRef = useRef(onStepChange);
onStepChangeRef.current = onStepChange;
const setSpeed = useCallback(
(next: PlaybackSpeed) => {
setSpeedState(next);
if (next === 'instant') {
setPlaying(false);
onStepChangeRef.current(Math.max(stepCount - 1, 0));
}
},
[stepCount],
);
const restart = useCallback(() => {
onStepChangeRef.current(0);
setPlaying(stepCount > 1);
}, [stepCount]);
const toggle = useCallback(() => {
if (stepCount <= 1) return;
setPlaying((was) => {
if (was) return false;
// Pressing play at the end replays from the top rather than doing
// nothing, which is what every visitor expects and nobody says out loud.
if (step >= stepCount - 1) onStepChangeRef.current(0);
return true;
});
}, [step, stepCount]);
useEffect(() => {
if (!playing || speed === 'instant' || stepCount <= 1) return;
if (step >= stepCount - 1) {
setPlaying(false);
return;
}
const base = stepDurationMs?.(step) ?? BASE_STEP_MS;
const timer = window.setTimeout(() => {
onStepChangeRef.current(step + 1);
}, Math.max(base / speed, 120));
return () => window.clearTimeout(timer);
}, [playing, speed, step, stepCount, stepDurationMs]);
return { playing, speed, setPlaying, setSpeed, toggle, restart, atEnd };
}
import { formatDate } from './format';
export interface TracePlayerProps {
playing: boolean;
@@ -105,7 +14,15 @@ export interface TracePlayerProps {
step: number;
stepCount: number;
onStepChange: (next: number) => void;
/** Straight off the run: never a marketing name for the model. */
/** 0..1 through the current step, from the player. Drives the hairline. */
progress?: number;
/**
* False when any step's dwell was invented because the trace carried no
* duration. Surfaced, not hidden: the player's whole claim is that the
* pacing is the model's, and where it is not, it says so.
*/
timingIsReal?: boolean;
/** Straight off the run. Never a marketing name for the model. */
model: string;
capturedAt: string;
/** Present only on an `intervened` run; the contract requires it there. */
@@ -116,11 +33,11 @@ export interface TracePlayerProps {
/**
* Transport controls for a recorded rollout.
*
* There is no spinner anywhere in this component and there never should be. A
* spinner implies a request is in flight; nothing here is live, and an exec who
* There is no spinner in this component and there never should be. A spinner
* implies a request is in flight; nothing here is live, and an exec who
* believes they are watching a model think in real time has been misled by the
* UI rather than the copy. Hence the permanent badge — it is not a disclosure
* we tuck into a footnote, it sits in the transport bar for the whole session.
* UI rather than the copy. Hence the permanent badge — not a disclosure tucked
* into a footnote, but a fixture of the transport bar for the whole session.
*/
export function TracePlayer({
playing,
@@ -131,105 +48,110 @@ export function TracePlayer({
step,
stepCount,
onStepChange,
progress = 0,
timingIsReal = true,
model,
capturedAt,
intervention,
className,
}: TracePlayerProps) {
const reducedMotion = usePrefersReducedMotion();
const canPlay = stepCount > 1;
return (
<div
className={cn(
'card flex flex-wrap items-center gap-x-3 gap-y-2 p-2 sm:gap-x-4',
className,
)}
>
<div className="flex items-center gap-1">
<button
type="button"
className="tap grid place-items-center rounded-lg text-muted transition-colors duration-2 ease-enter hover:bg-surface-2 hover:text-fg disabled:opacity-40"
onClick={() => onStepChange(Math.max(step - 1, 0))}
disabled={step <= 0}
aria-label="Previous step"
>
<ChevronLeft className="h-5 w-5" aria-hidden="true" />
</button>
<button
type="button"
className="tap grid place-items-center rounded-lg bg-primary px-4 text-primary-foreground transition-colors duration-2 ease-enter hover:bg-primary/90 disabled:opacity-40"
onClick={() => onPlayingChange(!playing)}
disabled={!canPlay}
aria-label={playing ? 'Pause the recorded run' : 'Play the recorded run'}
aria-keyshortcuts="Space"
>
{playing ? (
<Pause className="h-5 w-5" aria-hidden="true" />
) : (
<Play className="h-5 w-5" aria-hidden="true" />
)}
</button>
<button
type="button"
className="tap grid place-items-center rounded-lg text-muted transition-colors duration-2 ease-enter hover:bg-surface-2 hover:text-fg disabled:opacity-40"
onClick={() => onStepChange(Math.min(step + 1, Math.max(stepCount - 1, 0)))}
disabled={step >= stepCount - 1}
aria-label="Next step"
>
<ChevronRight className="h-5 w-5" aria-hidden="true" />
</button>
<button
type="button"
className="tap grid place-items-center rounded-lg text-muted transition-colors duration-2 ease-enter hover:bg-surface-2 hover:text-fg"
onClick={onRestart}
aria-label="Restart from the first step"
>
<RotateCcw className="h-4 w-4" aria-hidden="true" />
</button>
</div>
<div className={cn('card overflow-hidden', className)}>
<div className="flex flex-wrap items-center gap-x-3 gap-y-2 p-2 sm:gap-x-4">
<div className="flex items-center gap-1">
<Button
variant="ghost"
size="icon-touch"
onClick={() => onStepChange(Math.max(step - 1, 0))}
disabled={step <= 0}
aria-label="Previous step"
>
<ChevronLeft aria-hidden="true" />
</Button>
<Button
size="touch"
onClick={() => onPlayingChange(!playing)}
disabled={!canPlay}
aria-label={playing ? 'Pause the recorded run' : 'Play the recorded run'}
aria-keyshortcuts="Space"
>
{playing ? <Pause aria-hidden="true" /> : <Play aria-hidden="true" />}
</Button>
<Button
variant="ghost"
size="icon-touch"
onClick={() => onStepChange(Math.min(step + 1, Math.max(stepCount - 1, 0)))}
disabled={step >= stepCount - 1}
aria-label="Next step"
>
<ChevronRight aria-hidden="true" />
</Button>
<Button
variant="ghost"
size="icon-touch"
onClick={onRestart}
aria-label="Restart from the first step"
>
<RotateCcw aria-hidden="true" />
</Button>
</div>
<p className="nums text-sm text-muted">
Step <span className="font-semibold text-fg">{Math.min(step + 1, stepCount)}</span> of{' '}
{stepCount}
</p>
<div
role="radiogroup"
aria-label="Playback speed"
className="flex items-center gap-0.5 rounded-lg bg-surface-2 p-0.5"
>
{PLAYBACK_SPEEDS.map((option) => {
const selected = option === speed;
return (
<button
key={String(option)}
type="button"
role="radio"
aria-checked={selected}
onClick={() => onSpeedChange(option)}
className={cn(
'min-h-9 rounded-md px-2.5 text-xs font-medium transition-colors duration-2 ease-enter',
selected
? 'bg-surface text-fg shadow-sm'
: 'text-muted hover:text-fg',
)}
>
{option === 'instant' ? 'Instant' : `${option}x`}
</button>
);
})}
</div>
<RecordedBadge model={model} capturedAt={capturedAt} intervention={intervention} />
{reducedMotion ? (
// Not an apology — a statement that the page is behaving as asked. The
// steps still advance; only the tile flips and slides are gone.
<p className="sr-only">
Reduced motion is on. Steps still advance and every change is announced.
<p className="nums text-sm text-muted">
Step <span className="font-semibold text-fg">{Math.min(step + 1, stepCount)}</span> of{' '}
{stepCount}
</p>
) : null}
<div
role="radiogroup"
aria-label="Playback speed"
className="flex items-center gap-0.5 rounded-lg bg-surface-2 p-0.5"
>
{PLAYBACK_SPEEDS.map((option) => {
const selected = option === speed;
return (
<button
key={String(option)}
type="button"
role="radio"
aria-checked={selected}
onClick={() => onSpeedChange(option)}
className={cn(
'min-h-9 rounded-md px-2.5 text-xs font-medium transition-colors duration-2 ease-enter',
selected ? 'bg-surface text-fg shadow-sm' : 'text-muted hover:text-fg',
)}
>
{option === 'instant' ? 'Instant' : `${option}x`}
</button>
);
})}
</div>
<RecordedBadge
model={model}
capturedAt={capturedAt}
{...(intervention ? { intervention } : {})}
{...(timingIsReal ? {} : { timingNote: 'pacing approximate' })}
/>
</div>
{/*
A hairline, not a scrubber: the step chips below are the scrubber. It
exists so the pause between steps reads as time passing rather than as
the page having stopped. `transition-[width]` covers the ~15 Hz at which
the player pushes progress — without it the bar visibly ratchets.
*/}
<div className="h-0.5 w-full bg-surface-2" aria-hidden="true">
<div
className="h-full bg-brand transition-[width] duration-1 ease-enter"
style={{
width: `${
stepCount <= 1 ? 0 : ((step + Math.min(Math.max(progress, 0), 1)) / (stepCount - 1)) * 100
}%`,
}}
/>
</div>
</div>
);
}
@@ -238,6 +160,8 @@ export interface RecordedBadgeProps {
model: string;
capturedAt: string;
intervention?: string;
/** Rendered when the playback pacing is not the model's own. */
timingNote?: string;
className?: string;
}
@@ -245,6 +169,7 @@ export function RecordedBadge({
model,
capturedAt,
intervention,
timingNote,
className,
}: RecordedBadgeProps) {
return (
@@ -254,7 +179,7 @@ export function RecordedBadge({
className,
)}
>
<Circle className="h-2 w-2 shrink-0 fill-muted text-muted" aria-hidden="true" />
<Circle className="size-2 shrink-0 fill-muted text-muted" aria-hidden="true" />
<span className="font-medium text-fg">Recorded run</span>
<span aria-hidden="true">·</span>
<span className="nums font-mono">{model}</span>
@@ -265,6 +190,9 @@ export function RecordedBadge({
{intervention}
</span>
) : null}
{timingNote ? (
<span className="rounded-md border border-border px-1.5 py-0.5">{timingNote}</span>
) : null}
</p>
);
}
+87 -160
View File
@@ -1,38 +1,16 @@
import { useMemo, useState } from 'react';
import { CheckCircle2, ChevronDown, HelpCircle, XCircle } from 'lucide-react';
import { verifyEpisode } from '@/lib/demo-kit';
import type { DemoEpisode, DemoModule, RewardValues } from '@/lib/demo-kit/types';
import { VERIFY_TOLERANCE, verifyEpisode } from '@/lib/demo-kit/verify';
import type { AnyDemoModule } from '@/lib/demo-kit/registry';
import type { DemoEpisode } from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
import { DASH, formatDelta, formatNumber, formatOrDash } from './format';
import { rewardDelta, scoreReward } from './reward-math';
/**
* Float tolerance for "the browser agrees with Python".
*
* 1e-9 would be theatre: the two runtimes accumulate a sum in a different
* order, and IEEE-754 does not promise associativity. 1e-6 is well below any
* difference a reward change would produce and well above the noise.
*/
const EPSILON = 1e-6;
type Verdict =
| { kind: 'match'; recomputed: RewardValues; recordedTotal: number | null; recomputedTotal: number | null; maxDelta: number; rows: VerifyRow[] }
| { kind: 'mismatch'; recomputed: RewardValues; recordedTotal: number | null; recomputedTotal: number | null; maxDelta: number; rows: VerifyRow[]; guilty: string[] }
| { kind: 'unverifiable'; reason: string };
interface VerifyRow {
key: string;
label: string;
recorded: number | null;
recomputed: number | null;
delta: number;
}
export interface VerifyBadgeProps<T> {
demo: DemoModule<T>;
export interface VerifyBadgeProps {
demo: AnyDemoModule;
episode: DemoEpisode;
className?: string;
/** Open the receipt on load. The sceptic we are writing for opens it anyway. */
/** Open the receipt on load. The sceptic we wrote this for opens it anyway. */
defaultOpen?: boolean;
}
@@ -40,113 +18,58 @@ export interface VerifyBadgeProps<T> {
* The receipt.
*
* This object exists for one person: the engineer sitting next to the CEO who
* assumes the numbers on a vendor's demo page are hard-coded. It re-runs every
* recorded move through the TypeScript engine in the visitor's own browser,
* rescores it, and prints the comparison — including the delta, to seven
* decimals, because a comparison without a delta is an assertion.
* assumes the numbers on a vendor's demo page are hard-coded. It re-runs the
* recorded trace through the demo's own TypeScript grader, in the visitor's
* browser, and prints the comparison — including the delta to seven decimals,
* because a comparison without a delta is an assertion.
*
* It must therefore be allowed to FAIL loudly. A verifier that silently
* degrades to "verified" when it cannot check anything is worse than no
* verifier: it teaches the sceptic that the badge is decoration.
* It must be allowed to FAIL loudly. A badge that degrades to "verified" when
* it could not check anything is worse than no badge: it teaches the sceptic
* that the whole thing is decoration. The three states come straight from
* `verifyEpisode`, and `unverifiable` is never dressed up as either of the
* other two.
*/
export function VerifyBadge<T>({ demo, episode, className, defaultOpen = false }: VerifyBadgeProps<T>) {
export function VerifyBadge({ demo, episode, className, defaultOpen = false }: VerifyBadgeProps) {
const [open, setOpen] = useState(defaultOpen);
const result = useMemo(() => verifyEpisode(demo, episode), [demo, episode]);
const verdict = useMemo<Verdict>(() => {
if (!demo.verify) {
return {
kind: 'unverifiable',
reason:
'This demo does not ship a browser-side engine, so the recorded scores cannot be re-derived here. The Python that produced them is in the repository and the eval command is below.',
};
}
let recomputed: RewardValues | null;
try {
recomputed = verifyEpisode(demo, episode);
} catch (error) {
// A verifier that throws is a bug on our side, not a failed run. Say so
// rather than showing a red mismatch that blames the recorded numbers.
return {
kind: 'unverifiable',
reason: `The in-browser verifier threw while re-running this episode: ${
error instanceof Error ? error.message : String(error)
}`,
};
}
if (recomputed === null) {
return {
kind: 'unverifiable',
reason: episode.truncated
? 'This run was truncated before the environment reached a terminal state, so there is nothing complete to re-score. The recorded partial numbers are shown as they were captured.'
: 'The environment could not re-derive this episode from the recorded transcript. Nothing here is being asserted as verified.',
};
}
const deltas = rewardDelta(episode.rewards, recomputed);
const labels = new Map(demo.reward.components.map((c) => [c.key, c.label]));
const rows: VerifyRow[] = deltas
.map(({ key, delta }) => ({
key,
label: labels.get(key) ?? key,
recorded: episode.rewards[key] ?? null,
recomputed: recomputed[key] ?? null,
delta,
}))
.sort((a, b) => b.delta - a.delta || a.key.localeCompare(b.key));
const recordedTotal = scoreReward(demo.reward, episode.rewards).total;
const recomputedTotal = scoreReward(demo.reward, recomputed).total;
const totalDelta =
recordedTotal === null || recomputedTotal === null
? recordedTotal === recomputedTotal
? 0
: Number.POSITIVE_INFINITY
: Math.abs(recordedTotal - recomputedTotal);
const maxDelta = rows.reduce((worst, row) => Math.max(worst, row.delta), totalDelta);
const guilty = rows.filter((row) => row.delta > EPSILON).map((row) => row.label);
if (guilty.length === 0 && maxDelta <= EPSILON) {
return { kind: 'match', recomputed, recordedTotal, recomputedTotal, maxDelta, rows };
}
return { kind: 'mismatch', recomputed, recordedTotal, recomputedTotal, maxDelta, rows, guilty };
}, [demo, episode]);
if (verdict.kind === 'unverifiable') {
if (result.status === 'unverifiable') {
return (
<section
aria-label="Verification"
className={cn('card border-border bg-surface-2 p-3', className)}
className={cn('card bg-surface-2 p-3', className)}
>
<p className="flex items-start gap-2 text-sm">
<HelpCircle className="mt-0.5 h-4 w-4 shrink-0 text-muted" aria-hidden="true" />
<HelpCircle className="mt-0.5 size-4 shrink-0 text-muted" aria-hidden="true" />
<span>
<span className="font-semibold">Unverifiable in your browser.</span>{' '}
<span className="text-muted">{verdict.reason}</span>
<span className="text-muted">
{result.reason ?? 'This run cannot be re-computed here.'}
</span>
</span>
</p>
{result.recorded !== null ? (
<p className="nums mt-1.5 pl-6 font-mono text-xs text-muted">
Recorded {formatOrDash(result.recorded)} · recomputed {DASH} · Δ {DASH}
</p>
) : null}
</section>
);
}
const matched = verdict.kind === 'match';
const matched = result.status === 'match';
return (
<section
aria-label="Verification"
className={cn(
'card overflow-hidden',
matched ? 'border-positive/40' : 'border-danger',
className,
)}
className={cn('card overflow-hidden', matched ? 'border-positive/40' : 'border-danger', className)}
>
<div className={cn('p-3', matched ? 'bg-positive/10' : 'bg-danger/10')}>
<p className="flex items-start gap-2 text-sm leading-relaxed">
{matched ? (
<CheckCircle2 className="mt-0.5 h-4 w-4 shrink-0 text-positive" aria-hidden="true" />
<CheckCircle2 className="mt-0.5 size-4 shrink-0 text-positive" aria-hidden="true" />
) : (
<XCircle className="mt-0.5 h-4 w-4 shrink-0 text-danger" aria-hidden="true" />
<XCircle className="mt-0.5 size-4 shrink-0 text-danger" aria-hidden="true" />
)}
<span>
{matched ? (
@@ -159,19 +82,21 @@ export function VerifyBadge<T>({ demo, episode, className, defaultOpen = false }
) : (
<>
<span className="font-semibold text-danger">
Mismatch on {verdict.guilty.join(', ')}
{result.culprit
? `Mismatch on "${result.culprit.label}"`
: 'Mismatch against the recorded score'}
</span>{' '}
<span className="text-fg">
the browser re-run disagrees with the recorded score. Trust the source, not
this page.
the browser re-run disagrees with the published number. Trust the source in
the repository, not this page.
</span>
</>
)}
</span>
</p>
<p className="nums mt-1.5 pl-6 font-mono text-xs text-muted">
Recomputed {formatOrDash(verdict.recomputedTotal)} · recorded{' '}
{formatOrDash(verdict.recordedTotal)} · Δ {formatDelta(verdict.maxDelta)}
Recomputed {formatOrDash(result.recomputed)} · recorded {formatOrDash(result.recorded)} ·
Δ {result.delta === null ? DASH : formatDelta(result.delta)}
</p>
</div>
@@ -182,59 +107,61 @@ export function VerifyBadge<T>({ demo, episode, className, defaultOpen = false }
className="tap flex w-full items-center gap-1.5 border-t border-border px-3 text-left text-xs font-medium text-muted transition-colors duration-2 ease-enter hover:bg-surface-2 hover:text-fg"
>
<ChevronDown
className={cn('h-4 w-4 transition-transform duration-2 ease-enter', open && 'rotate-180')}
className={cn('size-4 transition-transform duration-2 ease-enter', open && 'rotate-180')}
aria-hidden="true"
/>
{open ? 'Hide the component-by-component receipt' : 'Show the component-by-component receipt'}
</button>
{open ? (
<div className="overflow-x-auto border-t border-border">
<table className="nums w-full border-collapse font-mono text-xs">
<thead>
<tr className="text-left text-muted">
<th scope="col" className="px-3 py-1.5 font-medium">
Component
</th>
<th scope="col" className="px-2 py-1.5 text-right font-medium">
Recorded
</th>
<th scope="col" className="px-2 py-1.5 text-right font-medium">
Recomputed
</th>
<th scope="col" className="px-3 py-1.5 text-right font-medium">
Δ
</th>
</tr>
</thead>
<tbody className="divide-y divide-border">
{verdict.rows.map((row) => {
const bad = row.delta > EPSILON;
return (
<tr key={row.key} className={cn(bad && 'bg-danger/10')}>
<th scope="row" className="px-3 py-1.5 text-left font-normal">
{row.label}
</th>
<td className="px-2 py-1.5 text-right">
{row.recorded === null ? DASH : formatNumber(row.recorded, 6)}
</td>
<td className="px-2 py-1.5 text-right">
{row.recomputed === null ? DASH : formatNumber(row.recomputed, 6)}
</td>
<td
className={cn('px-3 py-1.5 text-right', bad ? 'text-danger' : 'text-muted')}
>
{Number.isFinite(row.delta) ? formatDelta(row.delta) : 'not comparable'}
</td>
</tr>
);
})}
</tbody>
</table>
<div className="border-t border-border">
<div className="overflow-x-auto">
<table className="nums w-full border-collapse font-mono text-xs">
<thead>
<tr className="text-left text-muted">
<th scope="col" className="px-3 py-1.5 font-medium">
Component
</th>
<th scope="col" className="px-2 py-1.5 text-right font-medium">
Recorded
</th>
<th scope="col" className="px-2 py-1.5 text-right font-medium">
Recomputed
</th>
<th scope="col" className="px-3 py-1.5 text-right font-medium">
Δ
</th>
</tr>
</thead>
<tbody className="divide-y divide-border">
{result.components.map((row) => {
const bad = row.delta !== null && Math.abs(row.delta) > VERIFY_TOLERANCE;
return (
<tr key={row.key} className={cn(bad && 'bg-danger/10')}>
<th scope="row" className="whitespace-nowrap px-3 py-1.5 text-left font-normal">
{row.label}
</th>
<td className="px-2 py-1.5 text-right">
{row.recorded === null ? DASH : formatNumber(row.recorded, 6)}
</td>
<td className="px-2 py-1.5 text-right">
{row.recomputed === null ? DASH : formatNumber(row.recomputed, 6)}
</td>
<td className={cn('px-3 py-1.5 text-right', bad ? 'text-danger' : 'text-muted')}>
{/* An em dash, not 0.0000000: one side was never scored,
so there is no difference to report. */}
{row.delta === null ? DASH : formatDelta(row.delta)}
</td>
</tr>
);
})}
</tbody>
</table>
</div>
<p className="px-3 py-2 text-[11px] leading-relaxed text-muted">
Tolerance {EPSILON.toExponential()}. The browser engine and the Python environment
can sum the same terms in a different order, and IEEE-754 addition is not
associative, so the comparison is made within a tolerance rather than demanding
Tolerance {VERIFY_TOLERANCE.toExponential()}. The recorded numbers came out of Python
and these came out of JavaScript; both are IEEE-754 doubles summing the same terms in
a different order, so the comparison is made within a tolerance rather than demanding
bit-identical floats.
</p>
</div>
+6 -7
View File
@@ -99,14 +99,13 @@ export function useMediaQuery(query: string, serverValue = false): boolean {
}
/**
* Reduced motion is not only a CSS concern here. The CSS clamps transitions,
* but the trace player and the reasoning stream are JS timers: they have to
* resolve to their final state immediately, or a visitor who asked for no
* motion gets the animation anyway, just without the easing.
* Reduced motion is not only a CSS concern here — the reasoning stream is a JS
* timer and has to resolve to its final state immediately. Re-exported from the
* player rather than reimplemented: two subscriptions to the same media query
* can disagree for a frame, and the frame they disagree on is the one where an
* animation starts.
*/
export function usePrefersReducedMotion(): boolean {
return useMediaQuery('(prefers-reduced-motion: reduce)');
}
export { usePrefersReducedMotion } from '@/lib/demo-kit/player';
/** The one breakpoint the shell branches on: the drawer/panel split. */
export function useIsDesktop(): boolean {
-102
View File
@@ -1,102 +0,0 @@
/**
* Resolving `DemoMeta.icon` — a lucide export NAME — to a component.
*
* The obvious implementations are both wrong, and both were tried:
*
* `import * as lucide from 'lucide-react'` — kills tree-shaking. Every icon
* in the library (~1,500) lands in a chunk to render twelve of them.
*
* `import('lucide-react/dynamicIconImports')` — correct at runtime, but the
* map holds a dynamic import per icon, so Rollup emits ~1,500 chunk files
* into `dist/` for a static site that serves twelve.
*
* So the shell keeps an explicit registry. Adding a demo means adding its icon
* here; that is one line, and in exchange the entry chunk stays honest. An
* unknown name renders the neutral fallback rather than throwing, because a
* typo in a demo's metadata must not take the gallery down.
*/
import type { LucideIcon } from 'lucide-react';
import {
Blocks,
Boxes,
Braces,
Building2,
ClipboardCheck,
Code,
Cpu,
Database,
FileSearch,
Gauge,
Grid3x3,
HeartPulse,
Landmark,
LifeBuoy,
MessagesSquare,
Package,
PhoneCall,
Plug,
Radio,
Receipt,
Scale,
ShieldCheck,
ShoppingCart,
Stethoscope,
Truck,
Wallet,
Workflow,
Zap,
} from 'lucide-react';
const REGISTRY: Record<string, LucideIcon> = {
Blocks,
Boxes,
Braces,
Building2,
ClipboardCheck,
Code,
Cpu,
Database,
FileSearch,
Gauge,
Grid3x3,
HeartPulse,
Landmark,
LifeBuoy,
MessagesSquare,
Package,
PhoneCall,
Plug,
Radio,
Receipt,
Scale,
ShieldCheck,
ShoppingCart,
Stethoscope,
Truck,
Wallet,
Workflow,
Zap,
};
export const FallbackDemoIcon: LucideIcon = Boxes;
/** Every icon name the shell can render, for `check-demos` to assert against. */
export const KNOWN_ICON_NAMES: readonly string[] = Object.keys(REGISTRY);
export function resolveDemoIcon(name: string | undefined): LucideIcon {
if (!name) return FallbackDemoIcon;
return REGISTRY[name] ?? FallbackDemoIcon;
}
export interface DemoIconProps {
/** A lucide export name from `DemoMeta.icon`, e.g. `Grid3x3`. */
name: string | undefined;
className?: string;
/** Icons here are always decorative — the label beside them carries the name. */
strokeWidth?: number;
}
export function DemoIcon({ name, className, strokeWidth = 1.75 }: DemoIconProps) {
const Icon = resolveDemoIcon(name);
return <Icon className={className} strokeWidth={strokeWidth} aria-hidden="true" />;
}
-73
View File
@@ -1,73 +0,0 @@
import type { RewardComponent, RewardSpec, RewardValues } from '@/lib/demo-kit/types';
export interface ScoredRow {
component: RewardComponent;
/** The environment's raw per-component score. `null` means NOT SCORED. */
score: number | null;
/** The weight in force — shipped, or the visitor's edit. */
weight: number;
/** `score x weight`, or null when the component was not scored. */
value: number | null;
}
export interface ScoredReward {
rows: ScoredRow[];
/**
* The weighted sum over components that were actually scored. `null` when
* none of them were: a total of 0 would claim the run scored nothing, which
* is a different and much stronger statement than "we could not score it".
*/
total: number | null;
}
/** The weights the environment ships, as a plain map the editor can copy. */
export function shippedWeights(spec: RewardSpec): Record<string, number> {
const out: Record<string, number> = {};
for (const component of spec.components) out[component.key] = component.weight;
return out;
}
export function scoreReward(
spec: RewardSpec,
values: RewardValues,
weights?: Record<string, number>,
): ScoredReward {
let total = 0;
let anyScored = false;
const rows = spec.components.map((component) => {
const raw = values[component.key];
const score = raw === undefined ? null : raw;
const weight = weights?.[component.key] ?? component.weight;
const value = score === null ? null : score * weight;
if (value !== null) {
total += value;
anyScored = true;
}
return { component, score, weight, value };
});
return { rows, total: anyScored ? total : null };
}
/** True when two reward maps agree to within float noise on every key. */
export function rewardDelta(a: RewardValues, b: RewardValues): { key: string; delta: number }[] {
const keys = new Set([...Object.keys(a), ...Object.keys(b)]);
const out: { key: string; delta: number }[] = [];
for (const key of keys) {
const left = a[key];
const right = b[key];
// Both absent or both explicitly not-scored is agreement, not a zero
// delta on a number nobody produced.
if ((left === null || left === undefined) && (right === null || right === undefined)) {
out.push({ key, delta: 0 });
continue;
}
if (left === null || left === undefined || right === null || right === undefined) {
// One side scored and the other did not. That is a real disagreement and
// it has no numeric magnitude, so flag it as infinite rather than as 0.
out.push({ key, delta: Number.POSITIVE_INFINITY });
continue;
}
out.push({ key, delta: Math.abs(left - right) });
}
return out;
}
+1 -1
View File
@@ -226,7 +226,7 @@ export const VERTICALS: readonly VerticalEntry[] = [
'Escalating everything finds every clause and reviews nothing, so the escalation bucket has a budget and overspending it is penalised.',
plannedForV1: false,
caveat:
'Where this stops being honest: finding the clause is checkable, but whether the replacement language is an acceptable redline is judgment, and grading judgment collapses to an LLM judge — the exact thing a verifiable reward is meant to replace. We would ship the detection half with a real verifier and say plainly that the drafting half is unverified. We would not put a judge behind a bar chart and call it a score.',
'Finding the clause is checkable. Whether the replacement language is an acceptable redline is judgment, and grading judgment collapses to an LLM judge — the exact thing a verifiable reward is meant to replace. We would ship the detection half with a real verifier and say plainly that the drafting half is unverified. We would not put a judge behind a bar chart and call it a score.',
},
{
slug: 'semiconductor-ppa-closure',
+93
View File
@@ -0,0 +1,93 @@
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { createHash } from 'node:crypto';
import { test } from 'node:test';
import { answerForSeed, fnv1a32, hardModeViolation, scoreGuess } from '../engine';
const ANSWERS: string[] = JSON.parse(
readFileSync(new URL('../../../../envs/wordle_five/words/answers.json', import.meta.url), 'utf8'),
);
// The same table as envs/wordle_five/tests/test_engine.py. Both suites carry it
// because a vector that only exists on one side is a vector that can silently
// stop being checked on the other.
const VECTORS: [string, string, string][] = [
['alloy', 'llama', 'YGYXX'],
['speed', 'erase', 'YXYYX'],
['array', 'radar', 'YYYGX'],
['sassy', 'basis', 'YGGXX'],
['eerie', 'rebel', 'YGYXX'],
['level', 'eagle', 'YYXYX'],
['geese', 'these', 'XXGGG'],
['abbey', 'abbot', 'GGGXX'],
['crane', 'plane', 'XXGGG'],
['alloy', 'balmy', 'YXGXG'],
['tares', 'tares', 'GGGGG'],
];
test('scoring vectors', () => {
for (const [guess, answer, expected] of VECTORS) {
assert.equal(scoreGuess(guess, answer), expected, `${guess}/${answer}`);
}
});
test('a letter is never marked more often than it occurs', () => {
for (const answer of ANSWERS.slice(0, 200)) {
for (const guess of ANSWERS.slice(0, 50)) {
const pattern = scoreGuess(guess, answer);
for (const letter of new Set(guess)) {
const marked = [...guess].filter((c, i) => c === letter && pattern[i] !== 'X').length;
const occurs = [...answer].filter((c) => c === letter).length;
assert.ok(marked <= occurs, `${guess}/${answer} marked ${letter} ${marked}x`);
}
}
}
});
// Pinned identically in envs/wordle_five/tests/test_engine.py. If these two
// lists ever diverge, every ?seed= permalink shows a different puzzle than the
// recorded run it claims to be replaying.
const SEED_VECTORS = [
'wants', 'amber', 'spume', 'toady', 'divot', 'filly',
'bobby', 'clews', 'hikes', 'lawns', 'wreak', 'twist',
];
test('seed vectors match the Python engine', () => {
const got = Array.from({ length: 12 }, (_, s) => answerForSeed(s, ANSWERS));
assert.deepEqual(got, SEED_VECTORS);
});
test('fnv1a32 matches known values', () => {
// Computed by envs/wordle_five/wordle_five/engine.py fnv1a32().
assert.equal(fnv1a32('0'), 0x350ca8af);
assert.equal(fnv1a32('7'), 0x320ca3f6);
});
test('hard mode locks greens and counts yellows, but does not ban greys', () => {
assert.equal(hardModeViolation('crown', 'crane', 'GGXXX'), null);
assert.ok(hardModeViolation('blown', 'crane', 'GGXXX'));
const twoEs = scoreGuess('speed', 'erase'); // YXYYX
assert.equal(hardModeViolation('ester', 'speed', twoEs), null);
assert.ok(hardModeViolation('crest', 'speed', twoEs));
// Grey letters carry no constraint at all — the rule most implementations
// add and the real game does not have.
const cIsGrey = scoreGuess('crane', 'tares');
assert.equal(cIsGrey[0], 'X');
assert.equal(hardModeViolation('stare', 'crane', cIsGrey), null);
});
test('conformance digest matches the Python engine', () => {
const committed = readFileSync(
new URL('../../../../envs/wordle_five/CONFORMANCE.txt', import.meta.url),
'utf8',
).split(/\s+/)[0];
const hash = createHash('sha256');
for (const answer of ANSWERS) {
hash.update(ANSWERS.map((g) => scoreGuess(g, answer)).join(''));
}
assert.equal(hash.digest('hex'), committed);
});
+69
View File
@@ -0,0 +1,69 @@
import type { DemoEpisode, DemoStep } from '@/lib/demo-kit';
import { announceRow, scoreGuess, type BoardState, type Pattern } from './engine';
import { parseGuess } from './parse';
/**
* A recorded episode becomes a list of board snapshots.
*
* Snapshots, not deltas: the scrubber lets you jump to any step, and rebuilding
* state by replaying deltas from zero on every seek is both slower and the kind
* of thing that goes subtly wrong when a step is skipped.
*/
export function adapt(episode: DemoEpisode): DemoStep<BoardState>[] {
const answer = (episode as DemoEpisode & { answer?: string }).answer ?? '';
const rows: { guess: string; pattern: Pattern }[] = [];
let rejected = 0;
return episode.turns.map((turn, index) => {
const guess = parseGuess(turn.reply);
let announce: string;
let caption: string | undefined;
const alreadyPlayed = guess !== null && rows.some((r) => r.guess === guess);
const legal = guess !== null && guess.length === 5 && !alreadyPlayed;
if (!legal) {
rejected += 1;
const why =
guess === null
? 'no guess found in the reply'
: alreadyPlayed
? `repeated ${guess.toUpperCase()}`
: `${guess.toUpperCase()} is not five letters`;
announce = `Turn ${index + 1} refused: ${why}.`;
caption = why;
} else {
const pattern = scoreGuess(guess, answer);
rows.push({ guess, pattern });
announce = announceRow(guess, pattern);
caption =
pattern === 'GGGGG'
? 'solved'
: `${[...pattern].filter((t) => t === 'G').length} placed, ${
[...pattern].filter((t) => t === 'Y').length
} present`;
}
const won = rows.length > 0 && rows[rows.length - 1]!.pattern === 'GGGGG';
return {
index,
state: {
seed: episode.seed,
answer,
rows: rows.map((r) => ({ ...r })),
draft: '',
rejected,
status: won ? 'won' : rows.length >= 6 ? 'lost' : 'playing',
invalid: legal ? null : announce,
hardMode: false,
},
reply: turn.reply,
reasoning: turn.reasoning,
call: turn.call,
announce,
caption,
};
});
}
+82
View File
@@ -0,0 +1,82 @@
import { defineDemo, type DemoEpisode, type RewardValues } from '@/lib/demo-kit';
import { adapt } from './adapter';
import { answerForSeed, emptyBoard, type BoardState } from './engine';
import Keyboard from './keyboard';
import meta from './meta';
import { narrative } from './narrative';
import { parseGuess } from './parse';
import { recompute, reward } from './reward';
import Board from './surface';
import { ANSWERS } from './words';
export default defineDemo<BoardState>({
meta,
narrative,
reward,
anatomy: {
task: 'Find a hidden five-letter word in six guesses.',
actions:
'One five-letter word per turn, from a fixed 11,846-word list. Anything else is refused and costs a turn.',
grader:
'Compares the guess to the answer letter by letter and returns green, yellow or grey. It computes; it does not judge.',
score:
'Half for winning, a third for winning quickly, a fifth for never spending a turn on a word that could not have won.',
},
provenance: {
envPackage: 'wordle_five',
tasksetId: 'wordle-five',
verifiersVersion: '0.3.2.dev12',
command: 'uv run python envs/probe.py',
credits: [
{
label: 'prime-rl — Wordle as a starter example',
href: 'https://github.com/PrimeIntellect-ai/prime-rl/tree/main/examples/basic/wordle',
},
{
label: 'verifiers — the wordle environment',
href: 'https://github.com/PrimeIntellect-ai/verifiers/tree/main/environments/wordle',
},
{
label: 'TextArena — the engine those wrap',
href: 'https://github.com/LeonGuertler/TextArena',
},
{
label: 'Our word lists and how they were built',
href: 'https://github.com/karti-ai/PIG-Demo/blob/main/envs/wordle_five/words/PROVENANCE.md',
},
],
},
adapt,
Surface: Board,
interactive: {
init: (seed: number) => emptyBoard(seed, answerForSeed(seed, ANSWERS)),
Controls: Keyboard,
},
/**
* Re-derive the score from the recorded moves.
*
* This is what the verify badge renders. It deliberately reads
* `reference_depth` off the recorded metrics rather than recomputing it — the
* reference depth comes from a search the browser has no business running,
* and a missing one makes the run *unverifiable* rather than wrong.
*/
verify: (episode: DemoEpisode): RewardValues | null => {
const answer = (episode as DemoEpisode & { answer?: string }).answer;
if (!answer) return null;
const guesses: string[] = [];
let rejected = 0;
for (const turn of episode.turns) {
const guess = parseGuess(turn.reply);
if (guess === null || guess.length !== 5 || guesses.includes(guess)) {
rejected += 1;
continue;
}
guesses.push(guess);
}
const depth = episode.metrics?.['reference_depth'];
return recompute(answer, guesses, rejected, typeof depth === 'number' ? depth : null);
},
});
+239
View File
@@ -0,0 +1,239 @@
/**
* The game, in the browser.
*
* This is a port of `envs/wordle_five/wordle_five/engine.py`, and "port" is
* meant strictly: CI scores every (guess, answer) pair in the answer list
* through both implementations and compares a SHA-256 of the result. If these
* two files ever disagree by one tile, the build fails. That gate is what lets
* the page claim it verified a recorded run rather than merely replayed it.
*
* Keep this module pure and dependency-free. It runs on the main thread, in a
* Web Worker, and under `node --test`.
*/
export const WORD_LENGTH = 5;
export const MAX_GUESSES = 6;
/** A tile: correct position, present elsewhere, or absent. */
export type Tile = 'G' | 'Y' | 'X';
/** Five tiles, as a string. `'GYXXY'`. */
export type Pattern = string;
export const ALL_GREEN: Pattern = 'G'.repeat(WORD_LENGTH);
/**
* Green/yellow/grey feedback, as two passes.
*
* The two passes are not stylistic. A letter may be marked non-grey at most as
* many times as it occurs in the answer, and greens have first claim on that
* allocation — so every green in the word must be resolved before any yellow
* is assigned. A single pass marks the first S of SASSY yellow when BASIS has
* already spent both its S's on the greens that come later.
*
* This is the single most common bug in implementations of this game. It is
* also the bug that put a correction video on the most-watched explanation of
* it ever made, so it is worth the extra loop.
*/
export function scoreGuess(guess: string, answer: string): Pattern {
const g = guess.toLowerCase();
const a = answer.toLowerCase();
if (g.length !== a.length) {
throw new Error(`length mismatch: ${guess} vs ${answer}`);
}
const n = a.length;
const pattern: Tile[] = new Array(n).fill('X');
// Counts of each answer letter still available to yellows, keyed by char
// code so this stays allocation-free in the hot loop the solver runs.
const remaining = new Map<string, number>();
// Pass 1 — greens claim their letters out of the pool.
for (let i = 0; i < n; i += 1) {
if (g[i] === a[i]) {
pattern[i] = 'G';
} else {
const c = a[i]!;
remaining.set(c, (remaining.get(c) ?? 0) + 1);
}
}
// Pass 2 — yellows take only what pass 1 left, left to right.
for (let i = 0; i < n; i += 1) {
if (pattern[i] === 'G') continue;
const c = g[i]!;
const left = remaining.get(c) ?? 0;
if (left > 0) {
pattern[i] = 'Y';
remaining.set(c, left - 1);
}
}
return pattern.join('');
}
/**
* Would `candidate` have produced `pattern` for `guess`?
*
* This is the whole of constraint filtering, and it is also how `consistency`
* decides whether a guess contradicted what the player had already been told:
* a guess is consistent exactly when it was still a possible answer.
*/
export function isConsistent(candidate: string, guess: string, pattern: Pattern): boolean {
return scoreGuess(guess, candidate) === pattern;
}
/**
* Hard-mode legality, or null if the guess is legal.
*
* Three details are routinely got wrong and are deliberate here: greens are
* positional and locked; yellows are COUNTED, not merely present, so a guess
* must carry at least as many copies as were revealed; and grey letters are
* not banned at all — hard mode places no constraint on known-absent letters.
*/
export function hardModeViolation(
guess: string,
prevGuess: string,
prevPattern: Pattern,
): string | null {
const g = guess.toLowerCase();
const p = prevGuess.toLowerCase();
for (let i = 0; i < prevPattern.length; i += 1) {
if (prevPattern[i] === 'G' && g[i] !== p[i]) {
return `${p[i]!.toUpperCase()} must stay in position ${i + 1}`;
}
}
const need = new Map<string, number>();
for (let i = 0; i < prevPattern.length; i += 1) {
const tile = prevPattern[i];
if (tile === 'G' || tile === 'Y') {
const c = p[i]!;
need.set(c, (need.get(c) ?? 0) + 1);
}
}
const have = new Map<string, number>();
for (const c of g) have.set(c, (have.get(c) ?? 0) + 1);
for (const [letter, count] of need) {
if ((have.get(letter) ?? 0) < count) {
const copies = count === 1 ? '' : ` ${count} copies of`;
return `guess must contain${copies} ${letter.toUpperCase()}`;
}
}
return null;
}
/**
* FNV-1a, 32-bit. Chosen because it is trivial to reproduce exactly.
*
* A language's built-in RNG is not portable: Python's `random.Random(7)` is a
* Mersenne Twister with no honest one-line equivalent here, so seed 7 would
* pick one word in the environment and a different one in this tab. Every
* permalink would then disagree with the recorded run it claims to show. A
* hash sidesteps it — both sides compute the same integer from the same
* string, and there is nothing to keep in step.
*
* `Math.imul` is what makes the 32-bit multiply exact; a plain `*` overflows
* into a double and silently diverges from Python after the first few bytes.
*/
export function fnv1a32(text: string): number {
let h = 0x811c9dc5;
for (let i = 0; i < text.length; i += 1) {
h ^= text.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
return h >>> 0;
}
/** The hidden word for a seed. Identical in engine.py — see fnv1a32. */
export function answerForSeed(seed: number, pool: readonly string[]): string {
return pool[fnv1a32(String(seed)) % pool.length]!;
}
/** Why a guess would be refused, or null if it is playable. */
export function rejectionReason(
word: string,
history: readonly [string, Pattern][],
allowed: ReadonlySet<string>,
hardMode = false,
): string | null {
const w = word.toLowerCase().trim();
if (w.length !== WORD_LENGTH) return `'${w}' is not ${WORD_LENGTH} letters`;
if (!allowed.has(w)) return `'${w}' is not in the word list`;
if (history.some(([prev]) => prev === w)) return `'${w}' has already been guessed`;
if (hardMode && history.length > 0) {
const last = history[history.length - 1]!;
return hardModeViolation(w, last[0], last[1]);
}
return null;
}
/** The board, as the surface renders it. Snapshots, never deltas. */
export interface BoardState {
seed: number;
answer: string;
/** Completed rows. */
rows: { guess: string; pattern: Pattern }[];
/** What is being typed into the next row, if the board is interactive. */
draft: string;
/** Replies the game refused. They cost a turn of patience, not a row. */
rejected: number;
status: 'playing' | 'won' | 'lost';
/** Set for one render after an illegal guess, to drive the shake. */
invalid: string | null;
hardMode: boolean;
}
export function emptyBoard(seed: number, answer: string, hardMode = false): BoardState {
return {
seed,
answer,
rows: [],
draft: '',
rejected: 0,
status: 'playing',
invalid: null,
hardMode,
};
}
export function isOver(board: BoardState): boolean {
return board.status !== 'playing';
}
/** Play a guess, returning the next board. Pure — never mutates its input. */
export function play(board: BoardState, word: string, allowed: ReadonlySet<string>): BoardState {
if (isOver(board)) return board;
const history = board.rows.map((r) => [r.guess, r.pattern] as [string, Pattern]);
const reason = rejectionReason(word, history, allowed, board.hardMode);
if (reason) {
return { ...board, rejected: board.rejected + 1, invalid: reason, draft: board.draft };
}
const guess = word.toLowerCase().trim();
const pattern = scoreGuess(guess, board.answer);
const rows = [...board.rows, { guess, pattern }];
const status: BoardState['status'] =
pattern === ALL_GREEN ? 'won' : rows.length >= MAX_GUESSES ? 'lost' : 'playing';
return { ...board, rows, draft: '', invalid: null, status };
}
/**
* What a screen reader hears when a row lands.
*
* Not optional decoration: `prefers-reduced-motion` clamps the tile flip to
* nothing, and colour alone is not a result. This sentence IS the feedback for
* anyone who is not looking at the tiles.
*/
export function announceRow(guess: string, pattern: Pattern, remaining?: number): string {
const parts = [...guess].map((letter, i) => {
const tile = pattern[i];
const state = tile === 'G' ? 'placed' : tile === 'Y' ? 'present' : 'absent';
return `${letter.toUpperCase()} ${state}`;
});
const tail = remaining === undefined ? '' : ` ${remaining} words remain.`;
return `${guess.toUpperCase()}: ${parts.join(', ')}.${tail}`;
}
+127
View File
@@ -0,0 +1,127 @@
import { useEffect, useMemo } from 'react';
import { cn } from '@/lib/utils';
import { play, type BoardState } from './engine';
import { allowedNow, primeGuessList } from './words';
const ROWS = ['qwertyuiop', 'asdfghjkl', 'zxcvbnm'];
/** Best-known state of each letter, for tinting the keys. */
function letterStates(board: BoardState): Record<string, 'G' | 'Y' | 'X'> {
const rank = { X: 0, Y: 1, G: 2 } as const;
const out: Record<string, 'G' | 'Y' | 'X'> = {};
for (const { guess, pattern } of board.rows) {
for (let i = 0; i < guess.length; i += 1) {
const letter = guess[i]!;
const tile = pattern[i] as 'G' | 'Y' | 'X';
const current = out[letter];
if (!current || rank[tile] > rank[current]) out[letter] = tile;
}
}
return out;
}
const KEY_TINT: Record<string, string> = {
G: 'bg-tile-exact text-tile-exact-fg',
Y: 'bg-tile-present text-tile-present-fg',
X: 'bg-tile-absent/70 text-tile-absent-fg',
};
export function Keyboard({
state,
onChange,
}: {
state: BoardState;
onChange: (next: BoardState) => void;
}) {
const states = useMemo(() => letterStates(state), [state]);
const done = state.status !== 'playing';
useEffect(() => {
void primeGuessList();
}, []);
const press = (key: string) => {
if (done) return;
if (key === 'enter') {
if (state.draft.length !== 5) {
onChange({ ...state, invalid: 'not enough letters' });
return;
}
onChange(play(state, state.draft, allowedNow()));
return;
}
if (key === 'back') {
onChange({ ...state, draft: state.draft.slice(0, -1), invalid: null });
return;
}
if (state.draft.length < 5) {
onChange({ ...state, draft: state.draft + key, invalid: null });
}
};
// A physical keyboard is how anyone on a laptop will actually play, and
// wiring only the on-screen keys is the most common way that gets forgotten.
useEffect(() => {
const handler = (event: KeyboardEvent) => {
if (event.metaKey || event.ctrlKey || event.altKey) return;
const target = event.target as HTMLElement | null;
if (target && /^(INPUT|TEXTAREA)$/.test(target.tagName)) return;
if (event.key === 'Enter') press('enter');
else if (event.key === 'Backspace') press('back');
else if (/^[a-zA-Z]$/.test(event.key)) press(event.key.toLowerCase());
else return;
event.preventDefault();
};
window.addEventListener('keydown', handler);
return () => window.removeEventListener('keydown', handler);
});
return (
<div className="grid gap-1 sm:gap-1.5" style={{ paddingBottom: 'max(0px, var(--safe-bottom))' }}>
{ROWS.map((row, index) => (
<div key={row} className="flex justify-center gap-1 sm:gap-1.5">
{index === 2 ? (
<button
type="button"
onClick={() => press('enter')}
disabled={done}
className="tap flex-[1.6] rounded-md bg-surface-2 px-1 text-[0.65rem] font-semibold uppercase tracking-wide transition-colors duration-1 hover:bg-accent-subtle disabled:opacity-40"
>
Enter
</button>
) : null}
{[...row].map((letter) => (
<button
key={letter}
type="button"
onClick={() => press(letter)}
disabled={done}
aria-label={letter.toUpperCase()}
className={cn(
'tap min-w-0 flex-1 rounded-md text-sm font-semibold uppercase transition-colors duration-1 disabled:opacity-40',
states[letter] ? KEY_TINT[states[letter]!] : 'bg-surface-2 hover:bg-accent-subtle',
)}
>
{letter}
</button>
))}
{index === 2 ? (
<button
type="button"
onClick={() => press('back')}
disabled={done}
aria-label="Backspace"
className="tap flex-[1.6] rounded-md bg-surface-2 px-1 text-[0.65rem] font-semibold uppercase tracking-wide transition-colors duration-1 hover:bg-accent-subtle disabled:opacity-40"
>
Del
</button>
) : null}
</div>
))}
</div>
);
}
export default Keyboard;
+14
View File
@@ -0,0 +1,14 @@
import { defineMeta } from '@/lib/demo-kit';
export default defineMeta({
slug: 'wordle',
title: 'Word Five',
tagline: 'Guess a hidden five-letter word in six tries, from letter-by-letter feedback.',
vertical: 'reference',
status: 'live',
order: 0,
icon: 'Grid3x3',
persona: 'Anyone signing an AI budget',
rewardLine: 'Win fast, minus wasted guesses',
ogImage: '/og/wordle.png',
});
+95
View File
@@ -0,0 +1,95 @@
import type { Narrative } from '@/lib/demo-kit';
/**
* The six beats, in order. The shell renders them; this file decides what the
* page argues and in what sequence.
*/
export const narrative: Narrative = {
thesis:
'This is the smallest complete reinforcement-learning environment we could find that needs no ' +
'domain knowledge at all. It has everything the ones that matter to your business have: a task, ' +
'a fixed set of legal moves, a grader that cannot be argued with, and a score that moves when ' +
'the model gets better. Learn the machine here, and every demo after this is the same machine ' +
'with a different grader.',
anxiety: 'How would we know it was actually working?',
beats: [
{
id: 'hero',
title: 'Their hello-world, not ours',
claim:
'Prime Intellect ship this exact game as a starter environment in three of their public repositories. We did not pick a game. We picked theirs.',
surface: 'hero',
},
{
id: 'anatomy',
title: 'What an environment actually is',
claim:
'Four parts: a task, the moves that are legal, a grader that computes rather than opines, and a number that moves.',
surface: 'anatomy',
},
{
id: 'play',
title: 'You and the model get the same word',
claim:
'Same hidden word, same six guesses, same rules. Play it, then watch what the model did with it.',
surface: 'split-play',
},
{
id: 'watch',
title: 'Watch it think',
claim:
'This is not a video. It is a recorded attempt replayed at the speed it actually happened, and you can step through it one guess at a time.',
surface: 'scrubber',
},
{
id: 'reward',
title: 'You decide what good means',
claim:
'Move one slider and the winner changes. That is not a trick — it is the product.',
surface: 'reward-editor',
},
{
id: 'metric',
title: 'The number that moves',
claim:
'Out of the box, this model solved none of eight. Letting it think first is the cheapest intervention there is, and you can measure exactly what it bought.',
surface: 'metric',
},
{
id: 'receipt',
title: 'The whole environment, in one screen',
claim:
'The grader is thirty lines of Python. Here it is, and here is the command that runs it.',
surface: 'receipt',
},
{
id: 'limits',
title: 'What this does not teach',
claim:
'A word game is missing four things your business has. Each one is why the next demo exists.',
surface: 'limits',
},
],
limits: [
{
text:
'Nobody is pushing back. There is no counterparty adapting to what the agent does, which is most of what makes fraud and abuse hard.',
answeredBy: 'alert-triage',
},
{
text:
'There is no rule the agent could break. No privacy boundary, no regulator, no policy it must satisfy while it optimises.',
answeredBy: 'denial-appeal',
},
{
text:
'Every guess is objectively scorable. Real decisions have partial credit and honest disagreement about what a good answer even was.',
answeredBy: 'coverage-reserve',
},
{
text:
'The score arrives the moment the game ends. A claim, a bid or a dispatch is graded weeks later, by reality.',
answeredBy: 'day-ahead-bid',
},
],
};
+15
View File
@@ -0,0 +1,15 @@
/**
* Pull the move out of a model reply.
*
* Mirrors `envs/wordle_five/wordle_five/protocol.py`: the first bracketed
* alphabetic token, lowercased, and deliberately no length or dictionary check
* — so "guessed a six-letter word" and "produced no guess at all" stay
* different things in the metrics rather than collapsing into one.
*/
const BRACKETED = /\[([A-Za-z]+)\]/;
export function parseGuess(reply: string | null | undefined): string | null {
if (!reply) return null;
const match = BRACKETED.exec(reply);
return match ? match[1]!.toLowerCase() : null;
}
+110
View File
@@ -0,0 +1,110 @@
import type { RewardSpec, RewardValues } from '@/lib/demo-kit';
import rewardSource from '../../../envs/wordle_five/wordle_five/reward.py?raw';
import { scoreGuess, ALL_GREEN } from './engine';
/**
* The reward, mirrored from `envs/wordle_five/wordle_five/reward.py`.
*
* The labels are the point. `consistency` is a fair variable name and a useless
* thing to put in front of somebody deciding a budget; "guesses that could
* still have won" is the same quantity said in a way that needs no gloss.
*/
export const reward: RewardSpec = {
components: [
{
key: 'solved',
label: 'Found the word',
description: 'Did it win, within six guesses.',
weight: 0.5,
role: 'objective',
},
{
key: 'economy',
label: 'Did it in few guesses',
description:
'Turns used, as a ratio against the best player we ship, on the same hidden word. A ratio rather than a count, so a hard draw is not punished as a bad game.',
weight: 0.3,
role: 'objective',
},
{
key: 'consistency',
label: 'Never wasted a turn',
description:
'The share of attempts spent on a word that could still have been the answer. This one pulls against the other two on purpose.',
weight: 0.2,
role: 'counterweight',
},
],
metrics: [
{ key: 'guesses_used', label: 'Guesses used', description: 'Rows filled on the board.' },
{
key: 'rejected_replies',
label: 'Replies refused',
description: 'Not a word, wrong length, or a repeat. Costs a turn, not a row.',
},
{
key: 'inconsistent_guesses',
label: 'Contradicted itself',
description: 'Guesses ruled out by feedback the model had already been given.',
},
{
key: 'reference_depth',
label: 'Reference took',
description: 'How many guesses our best player needed for this same word.',
},
],
source: {
path: 'envs/wordle_five/wordle_five/reward.py',
code: rewardSource,
marker: 'reward',
},
};
const WEIGHTS: Record<string, number> = { solved: 0.5, economy: 0.3, consistency: 0.2 };
/**
* Re-derive the reward from the moves alone.
*
* This is the browser half of the verification: the page does not display the
* numbers the environment handed it, it recomputes them from the recorded
* guesses and shows the difference. `referenceDepth` cannot be recomputed here
* — it comes from a search the browser has no business running — so it is read
* off the recorded metrics, and its absence makes the run unverifiable rather
* than wrong.
*/
export function recompute(
answer: string,
guesses: readonly string[],
rejected: number,
referenceDepth: number | null,
): RewardValues | null {
if (referenceDepth === null) return null;
const patterns = guesses.map((g) => scoreGuess(g, answer));
const won = patterns.length > 0 && patterns[patterns.length - 1] === ALL_GREEN;
const solved = won ? 1 : 0;
const economy = won ? Math.min(1, referenceDepth / Math.max(1, guesses.length)) : 0;
// Scored over turns SPENT, refusals included. Counting only accepted guesses
// would hand a perfect score to a run that played one word and then jammed
// the parser: one guess, no contradictions, nothing to contradict.
const spent = guesses.length + rejected;
let viable = 0;
for (let i = 0; i < guesses.length; i += 1) {
let ok = true;
for (let k = 0; k < i; k += 1) {
if (scoreGuess(guesses[k]!, guesses[i]!) !== patterns[k]) {
ok = false;
break;
}
}
if (ok) viable += 1;
}
const consistency = spent === 0 ? 0 : viable / spent;
return { solved, economy, consistency };
}
export { WEIGHTS };
+120
View File
@@ -0,0 +1,120 @@
/**
* The entropy solver, in the browser.
*
* This is what makes the page feel alive rather than pre-baked: it answers on
* ANY word the visitor picks, including words no recording covers. It mirrors
* `envs/wordle_five/wordle_five/solver.py` in behaviour — same opener, same
* greedy rule, same tie-break — but not in implementation: Python precomputes
* a 21 MB pattern matrix, which is not a thing to ship to a phone.
*
* Run this in a Web Worker. The opening scan is ~4,600 x 4,600 pattern
* computations and will visibly jank the board on the main thread.
*/
import { scoreGuess, type Pattern } from './engine';
/**
* The opening guess, precomputed.
*
* It never depends on the game state, and computing it in the browser would
* cost the full 21M-pair scan on first paint for an answer that is always the
* same. Regenerate with `uv run python -c "from wordle_five.solver import
* _best_opener; from wordle_five.engine import answers; print(answers()[_best_opener()])"`
* — and if the word list changes, this changes with it.
*/
export const OPENER = 'tares';
/** Expected bits of information from playing `guess` against a candidate set. */
export function entropyOf(guess: string, candidates: readonly string[]): number {
const counts = new Map<Pattern, number>();
for (const candidate of candidates) {
const p = scoreGuess(guess, candidate);
counts.set(p, (counts.get(p) ?? 0) + 1);
}
const total = candidates.length;
let bits = 0;
for (const n of counts.values()) {
const probability = n / total;
bits -= probability * Math.log2(probability);
}
return bits;
}
/** Every answer still viable given the feedback so far. */
export function filterCandidates(
pool: readonly string[],
history: readonly { guess: string; pattern: Pattern }[],
): string[] {
let alive = pool as string[];
for (const { guess, pattern } of history) {
alive = alive.filter((word) => scoreGuess(guess, word) === pattern);
}
return alive;
}
export interface Suggestion {
guess: string;
bits: number;
/** True if this guess could itself be the answer. */
viable: boolean;
candidatesBefore: number;
}
/**
* The solver's move.
*
* Ties break toward a guess that could actually win — free expected value, and
* it costs nothing in `consistency`. When it does NOT break that way, the
* solver is buying information with a word that cannot win, which is exactly
* the trade the reward's counterweight prices. The `viable` flag is surfaced
* so the UI can show the moment it happens.
*/
export function suggest(
pool: readonly string[],
history: readonly { guess: string; pattern: Pattern }[],
/** Cap the guesses considered, for responsiveness on a phone. */
budget = 1500,
): Suggestion {
const candidates = filterCandidates(pool, history);
if (history.length === 0) {
return { guess: OPENER, bits: entropyOf(OPENER, pool), viable: pool.includes(OPENER), candidatesBefore: pool.length };
}
if (candidates.length <= 2) {
const guess = candidates[0] ?? OPENER;
return { guess, bits: candidates.length > 1 ? 1 : 0, viable: true, candidatesBefore: candidates.length };
}
// Score every remaining candidate, plus a slice of the wider pool — a
// non-candidate probe is often the better play, and considering only
// candidates would quietly turn this into the candidate-only policy.
const considered = new Set<string>(candidates);
for (const word of pool) {
if (considered.size >= budget) break;
considered.add(word);
}
let best: Suggestion = { guess: candidates[0]!, bits: -1, viable: true, candidatesBefore: candidates.length };
for (const guess of considered) {
const bits = entropyOf(guess, candidates);
const viable = candidates.includes(guess);
if (bits > best.bits + 1e-12 || (Math.abs(bits - best.bits) <= 1e-12 && viable && !best.viable)) {
best = { guess, bits, viable, candidatesBefore: candidates.length };
}
}
return best;
}
/** Play a whole game against a known answer. Used for the reference depth. */
export function solve(pool: readonly string[], answer: string, maxGuesses = 6): string[] {
const history: { guess: string; pattern: Pattern }[] = [];
const played: string[] = [];
for (let turn = 0; turn < maxGuesses; turn += 1) {
const { guess } = suggest(pool, history);
played.push(guess);
const pattern = scoreGuess(guess, answer);
if (pattern === 'GGGGG') return played;
history.push({ guess, pattern });
}
return played;
}
+38
View File
@@ -0,0 +1,38 @@
/**
* The solver, off the main thread.
*
* The opening scan is ~4,600 x 4,600 pattern computations. On the main thread
* that is a visible freeze on a phone, in the exact moment the visitor first
* touches the board.
*
* Constructed with `new Worker(new URL('./solver.worker.ts', import.meta.url),
* { type: 'module' })`, which produces a same-origin module in the build.
* Never Vite's `?worker&inline`: that yields a blob: URL, and production CSP
* has no `worker-src`, so it falls back to `default-src 'self'` and the worker
* is blocked with no console error at all. The solver would simply never boot,
* in production only.
*/
import type { Pattern } from './engine';
import { suggest } from './solver';
export interface SolverRequest {
id: number;
pool: string[];
history: { guess: string; pattern: Pattern }[];
}
export interface SolverResponse {
id: number;
guess: string;
bits: number;
viable: boolean;
candidatesBefore: number;
}
self.onmessage = (event: MessageEvent<SolverRequest>) => {
const { id, pool, history } = event.data;
const result = suggest(pool, history);
const response: SolverResponse = { id, ...result };
(self as unknown as Worker).postMessage(response);
};
+154
View File
@@ -0,0 +1,154 @@
import { memo } from 'react';
import { cn } from '@/lib/utils';
import { MAX_GUESSES, WORD_LENGTH, type BoardState, type Pattern } from './engine';
/**
* The board. One component for all three jobs — you playing, the replay, and
* the gallery thumbnail — because three near-identical boards is how they drift.
*/
const TILE_CLASS: Record<string, string> = {
G: 'bg-tile-exact text-tile-exact-fg border-tile-exact',
Y: 'bg-tile-present text-tile-present-fg border-tile-present',
X: 'bg-tile-absent text-tile-absent-fg border-tile-absent',
};
/**
* A glyph per state, shown only in high-contrast mode.
*
* Green/yellow/grey is a colour-only distinction, which is exactly why the
* original game ships a high-contrast mode. A second channel means the result
* survives deuteranopia, a projector with the colour balance wrong, and a
* screenshot printed in black and white.
*/
const TILE_GLYPH: Record<string, string> = { G: '●', Y: '◆', X: '' };
function Tile({
letter,
tile,
index,
revealing,
compact,
}: {
letter: string;
tile: Pattern[number] | null;
index: number;
revealing: boolean;
compact?: boolean;
}) {
const filled = letter !== '';
return (
<div
className={cn(
'relative grid place-items-center select-none font-semibold uppercase',
'aspect-square rounded-md border-2 transition-colors',
compact ? 'text-[0.55rem] border' : 'text-xl sm:text-2xl',
tile
? TILE_CLASS[tile]
: filled
? 'border-muted/60 bg-surface text-fg'
: 'border-border bg-surface-2/40 text-fg',
// The flip is a rotation about X with the colour landing at the
// half-way point, staggered along the row. `prefers-reduced-motion`
// clamps it to nothing globally, which is why announceRow() exists.
revealing && tile && 'motion-safe:animate-[tile-flip_520ms_ease-enter_both]',
filled && !tile && 'motion-safe:animate-[tile-pop_120ms_ease-enter]',
)}
style={revealing ? { animationDelay: `${index * 100}ms` } : undefined}
aria-hidden="true"
>
{letter}
{tile ? (
<span className="pointer-events-none absolute bottom-0 right-0.5 text-[0.5em] leading-none opacity-0 [:root[data-contrast='high']_&]:opacity-90">
{TILE_GLYPH[tile]}
</span>
) : null}
</div>
);
}
function Row({
guess,
pattern,
revealing,
shake,
compact,
}: {
guess: string;
pattern: Pattern | null;
revealing: boolean;
shake?: boolean;
compact?: boolean;
}) {
const letters = guess.padEnd(WORD_LENGTH, ' ').slice(0, WORD_LENGTH);
return (
<div
className={cn(
'grid gap-1 sm:gap-1.5',
shake && 'motion-safe:animate-[tile-shake_600ms_ease-enter]',
)}
style={{ gridTemplateColumns: `repeat(${WORD_LENGTH}, minmax(0, 1fr))` }}
>
{[...letters].map((letter, i) => (
<Tile
key={i}
letter={letter.trim()}
tile={pattern ? (pattern[i] as Pattern[number]) : null}
index={i}
revealing={revealing}
compact={compact}
/>
))}
</div>
);
}
export const Board = memo(function Board({
state,
compact,
}: {
state: BoardState;
compact?: boolean;
}) {
const rows = state.rows;
const draftRow = rows.length < MAX_GUESSES && state.status === 'playing' ? state.draft : null;
const blanks = MAX_GUESSES - rows.length - (draftRow === null ? 0 : 1);
return (
<div
className={cn('grid gap-1 sm:gap-1.5', compact ? 'w-full max-w-[7rem]' : 'w-full max-w-sm')}
role="img"
aria-label={
rows.length === 0
? 'Empty board, six guesses remaining.'
: `${rows.length} of ${MAX_GUESSES} guesses played.`
}
>
{rows.map((row, i) => (
<Row
key={`${row.guess}-${i}`}
guess={row.guess}
pattern={row.pattern}
revealing={i === rows.length - 1}
compact={compact}
/>
))}
{draftRow !== null ? (
<Row
guess={draftRow}
pattern={null}
revealing={false}
shake={state.invalid !== null}
compact={compact}
/>
) : null}
{Array.from({ length: Math.max(0, blanks) }, (_, i) => (
<Row key={`blank-${i}`} guess="" pattern={null} revealing={false} compact={compact} />
))}
</div>
);
});
export default Board;
+51
View File
@@ -0,0 +1,51 @@
/**
* The two word lists, loaded the way each is actually used.
*
* `answers` is inlined: the board needs it before first paint to turn a seed
* into a hidden word, and a fetch there would mean a visible empty board on a
* cold cache. At 4,603 words it costs about 14 kB gzipped inside this demo's
* lazy chunk, and never touches the entry chunk.
*
* `guesses` is fetched. It is nearly three times larger, and it is only needed
* the first time somebody presses Enter — by which point it has long arrived.
* Until it does, `isAllowed` falls back to the answer list, which accepts
* strictly fewer words: the failure mode is "your real word was rejected for a
* moment", not "a non-word was accepted", and that is the right way round.
*/
import answersJson from '../../../envs/wordle_five/words/answers.json';
export const ANSWERS: readonly string[] = answersJson;
let guesses: Set<string> | null = null;
let inFlight: Promise<Set<string>> | null = null;
/** Kick off the guess-list fetch. Safe to call more than once. */
export function primeGuessList(): Promise<Set<string>> {
if (guesses) return Promise.resolve(guesses);
if (!inFlight) {
inFlight = fetch('/words/guesses.json')
.then((r) => (r.ok ? r.json() : Promise.reject(new Error(String(r.status)))))
.then((words: string[]) => {
guesses = new Set(words);
return guesses;
})
.catch(() => {
// Degrade to the answer list rather than blocking play. A demo that
// shows an error card because a 106 kB asset was slow would be a worse
// failure than a briefly stricter dictionary.
guesses = new Set(ANSWERS);
return guesses;
});
}
return inFlight;
}
/** The set currently available for validation. Never null. */
export function allowedNow(): ReadonlySet<string> {
return guesses ?? new Set(ANSWERS);
}
export function guessListReady(): boolean {
return guesses !== null;
}
+22
View File
@@ -158,3 +158,25 @@
scroll-behavior: auto !important;
}
}
/*
* Board motion. Declared here rather than in the demo because Tailwind's
* arbitrary `animate-[...]` needs the keyframes to exist in the stylesheet, and
* a demo-local <style> block would be a second place for the token values to
* drift from.
*/
@keyframes tile-flip {
0% { transform: rotateX(0deg); }
50% { transform: rotateX(90deg); }
100% { transform: rotateX(0deg); }
}
@keyframes tile-pop {
0% { transform: scale(1); }
60% { transform: scale(1.06); }
100% { transform: scale(1); }
}
@keyframes tile-shake {
0%, 100% { transform: translateX(0); }
15%, 45%, 75% { transform: translateX(-5px); }
30%, 60%, 90% { transform: translateX(5px); }
}
+15 -7
View File
@@ -99,14 +99,22 @@ export default function Gallery() {
{shown.length === 0 ? (
<div className="card mt-4 p-6">
<p className={s.h3}>Nothing here yet.</p>
<p className={`${s.prose} mt-2`}>
No environment is filed under this vertical. The proposal for it is still on the
verticals page, written out in full.
{/* Two different nothings. Telling a visitor "no environment is filed
under this vertical" when they have not filtered anything reads as
a broken page rather than an empty one. */}
<p className={s.h3}>
{active === ALL ? 'No environments are registered.' : 'Nothing under this vertical.'}
</p>
<button className={`${s.btnSecondary} mt-4`} onClick={() => select(ALL)} type="button">
Show every environment
</button>
<p className={`${s.prose} mt-2`}>
{active === ALL
? 'The registry is empty, which means the site is mid-build rather than hiding something. The lineup below is written either way.'
: 'No environment is filed here yet. The proposal for it is still on its own page, written out in full.'}
</p>
{active === ALL ? null : (
<button className={`${s.btnSecondary} mt-4`} onClick={() => select(ALL)} type="button">
Show every environment
</button>
)}
</div>
) : (
<ul className="mt-4 grid gap-4 sm:grid-cols-2 lg:grid-cols-3">