Landing picks an environment; each environment is a tabbed page opening on Play
ci / web (push) Successful in 3m10s
ci / python (push) Successful in 2m32s

The environment page was a linear scroll of eight narrative beats. That is an
essay, and it is the wrong shape for somebody who has just chosen an
environment and wants to use it. It is now four tabs — Play, Watch, Reward,
Evidence — opening on Play, with the board above the fold at 390x844 and the
anatomy strip directly beneath it. The landing page leads with the picker
instead of burying it under the thesis.

The contract changed rather than layering tabs over beats. `Narrative.beats` is
gone; `claims: Record<DemoTabId, string>` replaces it, one required sentence per
tab. Writing the claim is how an author discovers whether a tab has anything to
say — a tab whose claim is hard to write is usually a tab with nothing in it.
Doing this now costs one migration; doing it after eleven more environments
costs twelve.

Tabs are derived, never declared: Play iff the demo ships an `interactive` mode,
Watch iff it has recorded runs. A demo that could name its own tabs would mean
environment seven inventing a fifth one and the site ceasing to be one product.

One thing the browser caught that no gate would have. The header stat strip
describes the RECORDED RUN, and on Play it sat above the visitor's own empty
board reading "Outcome: failed" — which parses as your game having already
failed before you touch a key. It now renders only on the tabs whose subject is
that run, which also moved the board 54px up the page.

The picker is honest about the shape of the lineup by construction: one built
environment gets its own block and the demo's real board as its thumbnail,
twelve written specifications render dimmed with a Spec badge, and every count
on the page is derived from the data rather than typed.

206 contract checks pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
2026-08-28 18:07:34 -07:00
parent f5f45df224
commit 5bbf913664
23 changed files with 2005 additions and 866 deletions
+137
View File
@@ -0,0 +1,137 @@
---
name: new-environment
description: Build a new RL environment demo for this repo, end to end — specification, Python environment, TypeScript port, captured rollouts, and the demo page. Use when adding an environment to demo.primeintellectgrowth.com, promoting a vertical from spec to live, or when asked to "build environment N" / "add the claims demo" / "make the support one real".
---
# Building an environment
The scarce thing here is not code. It is a **defensible environment**: a task
with a grader that computes rather than opines, and a reward that cannot be
maximised by doing the crude thing. Everything in this repository — the probe
ladder, the conformance digest, the contract validator — exists to stop a
plausible-looking fake from shipping.
Work in this order. Each stage has a gate, and a stage that cannot pass its gate
is telling you the environment is wrong, not that the gate is too strict.
## 0. The specification already exists
`src/content/verticals.ts` carries a reviewed entry for every planned
environment: `task`, `reward`, `counterweight`, `persona`, `anxiety`, and a
`rank` (1 is the one to build next). **Start from that entry.** If you are
building something not in that file, add the entry first and have it reviewed
before writing a line of Python — the spec is the cheapest place to discover
that an environment does not work.
## 1. Write `envs/<pkg>/SPEC.md` before any code
Concrete answers, not restatements of the vertical entry:
- **The episode.** Exactly what one instance is, and what ends it.
- **The data.** Where it comes from. If synthetic, say so on the page — never
imply real customer data. If real, name the licence.
- **The action set.** Legal moves, and what happens to a malformed reply.
(It costs a turn and is recorded. It never crashes the rollout.)
- **The grader.** How the score is computed from the transcript, deterministically.
- **The reward components**, with weights summing to 1.0 and a `counterweight`.
- **The probe ladder**: the policies, and what each must score.
- **The held-out slice.**
⚠️ **If the grader needs an LLM judge, stop.** A judge is the exact thing a
verifiable reward is positioned against, and shipping one here would undercut
every other environment on the site. Either find the computable signal, or ship
the environment as a `spec` with the caveat stated — `legal-playbook-redline`
is in the lineup for precisely this reason and is honest about it.
**Gate:** the spec is reviewed adversarially before implementation. Ask of it:
could a competent sceptic call this grader subjective? Could the reward be
maxed by doing something the buyer would hate?
## 2. The Python environment
`envs/<pkg>/``engine.py` (pure, seeded, never raises on bad input),
`protocol.py` (parse/render; a parse failure is a rejected move, not an
exception), `reward.py` (with `# region: pig-demo/reward` markers around the
part the page quotes), a reference policy, and `tests/`.
Add the policies to `envs/probe.py` and register them for your taskset.
**Gate:** `uv run python envs/probe.py` passes every assertion. Specifically:
doing nothing scores exactly `0.000`; no weighted component is flat across the
ladder; and **your two good policies do not dominate each other** — if one wins
on every component, your counterweight is decorative and the reward editor on
the page would be theatre.
Two mistakes already made here, both worth avoiding twice:
- Score the counterweight over turns **spent**, not moves accepted. Otherwise a
policy that acts once and then jams the parser scores a perfect 1.0 on it.
- The efficiency component's denominator is the depth **your shipped reference
policy** reaches, not a true optimum. Grade against an optimum and your own
oracle rung fails its assertion on some seeds.
## 3. The TypeScript port
`src/demos/<slug>/engine.ts` mirrors the Python. Both sides hash the full
cross-product of the task set and the digest is committed and gated in CI.
⚠️ Never use a language's built-in RNG for seeding. `random.Random(seed)` and
any JS PRNG disagree, so the same seed picks a different instance on each side
and every `?seed=` permalink silently shows a different task than the run it
claims to replay. Use FNV-1a over the decimal seed, as `engine.py` does.
**Gate:** `pnpm conformance` — the two digests match.
## 4. Capture
`uv run python envs/capture.py --arm <arm> --seeds 0-7`
⚠️ **spark-1 serves one model and is single-stream. Capture is a serialised
queue, never parallel.** A thinking-enabled arm can take minutes per seed.
Front-end work does not wait on it — build against `mock.tsx` instead.
Capture at least: a weak arm, a strong arm, and a generated reference policy.
Include at least one run the agent **loses**; a demo where the agent always wins
teaches nothing about the reward. A run that had anything done to it is
`kind: 'intervened'` and must name the `intervention`, so a sampling change can
never be read as a training result.
**Gate:** `uv run python envs/verify_fixtures.py` — every fixture replays through
the engine and reproduces its own recorded rewards.
## 5. The demo module
`pnpm demo:new <slug>` scaffolds it. Fill in `meta.ts`, `narrative.ts`
(`claims` for all four tabs), `surface.tsx`, `adapter.ts`, `reward.ts`, and
`interactive` if the task can be played by a human. Read `src/demos/wordle/` as
the reference implementation.
Where the demo needs something the shell does not offer, that is a contract bug
to fix in `demo-kit` or expose as a `SlotRegion` — never an `if (slug === ...)`
in `src/components/demo/`. `check-demos` fails on it.
**Gate:** `pnpm check && pnpm build`.
## 6. Ship
Add the OG card (`node scripts/og.mjs` — x86 only), rebuild, and
`bash deploy/deploy.sh`. Flip the vertical's status so the picker stops
advertising it as a specification.
## The honesty rules, which are not negotiable
- Rollouts are **recorded, not live**, and the page says so.
- The reward editor **re-scores recorded attempts**; it does not retrain.
- Synthetic data is labelled synthetic, on the page, not in a footnote.
- No customer logos and no named companies as customers, ever.
- Cite someone else's number as theirs, with a link, and never re-quote a figure
across a version change without re-measuring it.
- If the environment is materially easier or harder than the real task it
stands for, say which and by how much.
## Running it as a pipeline
`.claude/workflows/new-environment.js` runs stages 15 as a workflow with
adversarial review between them. Invoke with
`Workflow({ name: 'new-environment', args: { slug, vertical, seeds } })`.
Read this file first regardless — the workflow encodes the sequence, this
encodes the judgement.
+182
View File
@@ -0,0 +1,182 @@
export const meta = {
name: 'new-environment',
description: 'Promote a vertical from published spec to a live environment: specification, Python env, TypeScript port, rollouts, demo page',
whenToUse: 'Adding an environment to demo.primeintellectgrowth.com. Pass { slug, vertical, seeds }.',
phases: [
{ title: 'Specify', detail: 'design the environment, then attack the design' },
{ title: 'Implement', detail: 'Python environment and its probe ladder' },
{ title: 'Port', detail: 'TypeScript engine and the demo module, in parallel' },
{ title: 'Capture', detail: 'real rollouts — serialised, spark-1 is single-stream' },
{ title: 'Verify', detail: 'adversarial review across four lenses' },
],
}
const REPO = '/home/kartios/repos/gitea/PIG-Demo'
const slug = (args && args.slug) || 'support-resolution'
const vertical = (args && args.vertical) || 'customer-support-resolution'
const pkg = slug.replace(/-/g, '_')
const seeds = (args && args.seeds) || '0-7'
const BASE = `You are building environment "${slug}" for the repo at ${REPO}, promoting the vertical "${vertical}" from a published specification to a live, playable demo at demo.primeintellectgrowth.com.
**Read \`.claude/skills/new-environment/SKILL.md\` first, in full.** It carries the judgement; this prompt carries only your stage. Also read \`CONTRACT.md\`, \`AGENTS.md\`, and the reference implementation in \`envs/wordle_five/\` and \`src/demos/wordle/\`.
The specification for this vertical — task, reward, counterweight, persona, anxiety — is already written and reviewed in \`src/content/verticals.ts\`. Start from that entry; do not invent a different task.
Package: \`envs/${pkg}/\`. Demo: \`src/demos/${slug}/\`. Traces: \`public/traces/${slug}/\`.
Do NOT run git. Do NOT edit another stage's files.`
const SPEC_SCHEMA = {
type: 'object', additionalProperties: false,
required: ['episode', 'data', 'actions', 'grader', 'components', 'ladder', 'heldOut', 'risks'],
properties: {
episode: { type: 'string' },
data: { type: 'string', description: 'Source, synthetic-or-real, licence' },
actions: { type: 'string' },
grader: { type: 'string', description: 'How the score is computed, deterministically' },
components: { type: 'array', items: { type: 'object', additionalProperties: false,
required: ['key','label','weight','role','definition'],
properties: { key:{type:'string'}, label:{type:'string'}, weight:{type:'number'},
role:{type:'string', enum:['objective','counterweight','gate']}, definition:{type:'string'} } } },
ladder: { type: 'array', items: { type: 'object', additionalProperties: false,
required: ['policy','behaviour','expected'],
properties: { policy:{type:'string'}, behaviour:{type:'string'}, expected:{type:'string'} } } },
heldOut: { type: 'string' },
risks: { type: 'array', items: { type: 'string' } },
},
}
phase('Specify')
const spec = await agent(`${BASE}
## YOUR STAGE: the specification. No code.
Design the environment concretely and write \`envs/${pkg}/SPEC.md\`. Answer the seven questions in the skill's stage 1 with specifics, not restatements of the vertical entry.
The two decisions that make or break this:
**The grader must compute, not opine.** If you find yourself reaching for an LLM judge, you have not found the environment yet. Look for the signal that already exists in the business: a downstream system's response, a settled number, a later observed outcome. If there genuinely is not one, say so plainly in \`risks\` — shipping it as a spec with the caveat stated is a legitimate outcome and better than faking it.
**The counterweight must be in genuine tension with the objective.** Name the crude policy that maxes the objective, and the component that stops it. Then name a good policy that pays the counterweight's price — if you cannot, the counterweight is a gate wearing the wrong label.
Also decide the data honestly. Synthetic is fine and often the only option; it must be labelled synthetic on the page, and it must not imply real customer records.
Return the structured spec AND write SPEC.md.`, { label: 'spec', phase: 'Specify', schema: SPEC_SCHEMA, effort: 'high' })
const specText = spec ? JSON.stringify(spec, null, 2) : '(the spec stage returned nothing)'
const LENSES = [
'DETERMINISM: is this grader really computable from the transcript, every time, with no human or model judgement anywhere in it? Hunt for a step that quietly needs an opinion.',
'GAMEABILITY: find the policy that maximises this reward while doing something the buyer would be appalled by. If you find one, the counterweight is insufficient. Try at least four distinct cheats.',
'HONESTY: would an executive be misled by this? Check the data provenance, whether the task is materially easier than the real job, and whether anything here implies a customer, a result, or a capability we have not got.',
]
const REVIEW_SCHEMA = {
type: 'object', additionalProperties: false,
required: ['verdict', 'findings'],
properties: {
verdict: { type: 'string', enum: ['sound', 'fixable', 'fatal'] },
findings: { type: 'array', items: { type: 'object', additionalProperties: false,
required: ['problem','fix'], properties: { problem:{type:'string'}, fix:{type:'string'} } } },
},
}
const reviews = (await parallel(LENSES.map((lens, i) => () =>
agent(`Attack this environment specification through ONE lens. Be adversarial: your job is to find the reason it should not ship, not to approve it.
LENS: ${lens}
${BASE}
=== THE SPECIFICATION ===
${specText}`, { label: `attack:${i}`, phase: 'Specify', schema: REVIEW_SCHEMA, effort: 'high' })
))).filter(Boolean)
const reviewText = reviews.map((r, i) => `--- LENS ${i} — verdict: ${r.verdict}\n${r.findings.map(f => ` ! ${f.problem}\n -> ${f.fix}`).join('\n')}`).join('\n\n')
if (reviews.some(r => r.verdict === 'fatal')) {
log('A reviewer called the specification fatal. Stopping before implementation.')
return { stopped: 'fatal specification', spec, reviews }
}
phase('Implement')
const python = await agent(`${BASE}
## YOUR STAGE: the Python environment.
Build \`envs/${pkg}/\` — engine, protocol, reward (with \`# region: pig-demo/reward\` markers), a reference policy, and tests. Register your probe policies in \`envs/probe.py\` for taskset "${slug}".
Implement the specification below, INCLUDING every fix the reviewers demanded. Where a reviewer found a gameable policy, add it to the probe ladder as a rung that must score below the good policies — a cheat you have named and measured is a cheat that cannot come back.
**Your gate:** \`uv run python envs/probe.py\` passes every assertion, including that your two good policies do not dominate each other. Iterate until it does. If it cannot pass, the reward is wrong — fix the reward, not the assertion.
=== SPECIFICATION ===
${specText}
=== WHAT THE REVIEWERS DEMANDED ===
${reviewText}`, { label: 'python', phase: 'Implement', effort: 'high' })
phase('Port')
const ported = (await parallel([
() => agent(`${BASE}
## YOUR STAGE: the TypeScript engine.
Port \`envs/${pkg}/\` 's scorer to \`src/demos/${slug}/engine.ts\`, and extend \`scripts/conformance.mjs\` so this environment's digest is gated the way wordle's is. Seed with FNV-1a over the decimal seed — never a built-in RNG, or the same seed picks a different task on each side and every permalink lies.
**Your gate:** \`pnpm conformance\` — the two digests match.
The Python that landed:
${python}`, { label: 'port:ts', phase: 'Port', effort: 'high' }),
() => agent(`${BASE}
## YOUR STAGE: the demo module.
Build \`src/demos/${slug}/\`\`meta.ts\`, \`narrative.ts\` (\`claims\` for all four tabs: play, watch, reward, evidence), \`surface.tsx\`, \`adapter.ts\`, \`reward.ts\`, and \`interactive\` if a human can play this task. Read \`src/demos/wordle/\` as the reference.
Build against \`src/components/demo/mock.tsx\` — do NOT wait for real rollouts, that is the whole point of the wire format.
Anything the demo needs that the shell does not offer is a contract bug to fix in \`demo-kit\` or expose as a \`SlotRegion\`, never a slug check in \`src/components/demo/\`.
**Your gate:** \`pnpm check\` and \`./node_modules/.bin/tsc --noEmit\` clean for your files.
The specification:
${specText}`, { label: 'port:demo', phase: 'Port', effort: 'high' }),
])).filter(Boolean)
phase('Capture')
const captured = await agent(`${BASE}
## YOUR STAGE: real rollouts.
Extend \`envs/capture.py\` for taskset "${slug}" and capture seeds ${seeds} for at least three arms: a weak one, a strong one, and a generated reference policy.
⚠️ **spark-1 serves one model and is single-stream — run the arms one after another, never concurrently.** Check it is up first: \`curl http://100.127.247.67:8001/v1/models\`. A thinking-enabled arm can take minutes per seed; run it in the background, redirect output to a FILE (piping to head/tail block-buffers Python's stdout and a working run looks hung), and poll the written fixtures rather than the log.
Include at least one run the agent loses. Then \`uv run python envs/build_manifest.py\`.
**Your gate:** \`uv run python envs/verify_fixtures.py\` — every fixture replays and reproduces its own recorded rewards at delta 0.
If spark-1 is unreachable, do NOT invent fixtures. Say so and stop; a fabricated rollout on this site would be worse than a missing one.`, { label: 'capture', phase: 'Capture', effort: 'high' })
phase('Verify')
const CHECKS = [
'Run the full gate chain and make it green: uv run python envs/probe.py, uv run pytest, uv run python envs/verify_fixtures.py, pnpm check, pnpm conformance, pnpm build. Fix whatever is broken.',
'Open the new demo in a real browser at 1280px and 390px (playwright is installed; serve dist). Confirm: it appears in the landing picker, all four tabs render, Play works if it has an interactive mode, ?tab= round-trips, zero console errors, no horizontal overflow at 390px.',
'Audit honesty against the rules in the skill: is synthetic data labelled synthetic ON THE PAGE, are recorded runs labelled recorded, does any claim imply a customer or a result we do not have, and does the page state how this environment differs from the real job? Fix the copy where it does not hold.',
]
const verified = (await parallel(CHECKS.map((c, i) => () =>
agent(`${BASE}\n\n## YOUR STAGE: verification.\n\n${c}\n\nReport precisely what you ran, what failed, and what you changed. Do not report success you did not observe.`,
{ label: `verify:${i}`, phase: 'Verify', effort: 'high' })
))).filter(Boolean)
return {
slug, vertical,
spec, reviews: reviewText,
python, ported, captured,
verified,
}
+13
View File
@@ -1,5 +1,18 @@
# Working in this repository # Working in this repository
## Adding an environment
Read **`.claude/skills/new-environment/SKILL.md`**. It is the recipe and, more
importantly, the judgement: which gate means "iterate" and which means "this
environment does not work". The pipeline that runs it is
`.claude/workflows/new-environment.js`.
The short version: the specification comes from `src/content/verticals.ts`
(`rank: 1` is the one to build next), it gets written up and attacked before any
code, the probe ladder is what proves the reward measures something, and the
conformance digest is what proves the browser and the environment are playing
the same game.
## The three commands ## The three commands
```bash ```bash
+6
View File
@@ -137,6 +137,12 @@ canonical name is the first one and every page says so in its `<link rel=canonic
pnpm demo:new <slug> pnpm demo:new <slug>
``` ```
The full recipe — specification, environment, port, capture, page — is in
`.claude/skills/new-environment/SKILL.md`, and
`.claude/workflows/new-environment.js` runs it as a pipeline with adversarial
review between the stages. The specifications for the eleven environments not
yet built are already written, in `src/content/verticals.ts`.
Copies the templates and wires nothing — the registry finds demos by existence, Copies the templates and wires nothing — the registry finds demos by existence,
so the header, the gallery, the router and the sitemap all pick it up with zero so the header, the gallery, the router and the sitemap all pick it up with zero
edits to shared files. `pnpm check` enforces the contract; see edits to shared files. `pnpm check` enforces the contract; see
Binary file not shown.

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 140 KiB

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 165 KiB

After

Width:  |  Height:  |  Size: 125 KiB

+320 -3
View File
@@ -6,10 +6,11 @@
* shapes; this enforces everything a type cannot: that the directory name * shapes; this enforces everything a type cannot: that the directory name
* matches the slug, that a `spec` demo is a real published specification rather * matches the slug, that a `spec` demo is a real published specification rather
* than a coming-soon card, that a `live` demo's traces exist on disk, that the * than a coming-soon card, that a `live` demo's traces exist on disk, that the
* shared shell has no idea any particular demo exists, and that the reward has * shared shell has no idea any particular demo exists, that every tab on the
* something pulling against its objective. * page asserts something a reader can check, and that the reward has something
* pulling against its objective.
* *
* Thirteen numbered rules, each reported with the file to open. Run it with * Fourteen numbered rules, each reported with the file to open. Run it with
* `pnpm check`. * `pnpm check`.
* *
* Most rules are checked by READING the TypeScript, not by running it — the * Most rules are checked by READING the TypeScript, not by running it — the
@@ -34,12 +35,15 @@ import {
die, die,
exists, exists,
findInDemo, findInDemo,
evalLiteral,
isUnresolved, isUnresolved,
literalAfter,
loadManifest, loadManifest,
loadMeta, loadMeta,
parseStringUnion, parseStringUnion,
read, read,
rel, rel,
segment,
traceFile, traceFile,
walk, walk,
} from './_lib.mjs'; } from './_lib.mjs';
@@ -56,9 +60,50 @@ const typesSrc = read(TYPES_FILE);
// would drift, and it would drift silently in the direction of passing. // would drift, and it would drift silently in the direction of passing.
const VERTICALS = parseStringUnion(typesSrc, 'Vertical'); const VERTICALS = parseStringUnion(typesSrc, 'Vertical');
const STATUSES = parseStringUnion(typesSrc, 'DemoStatus'); const STATUSES = parseStringUnion(typesSrc, 'DemoStatus');
const TAB_IDS = parseStringUnion(typesSrc, 'DemoTabId');
const ROLES = ['objective', 'counterweight', 'gate']; const ROLES = ['objective', 'counterweight', 'gate'];
if (!VERTICALS) die(`could not parse the \`Vertical\` union out of ${rel(TYPES_FILE)}.`); if (!VERTICALS) die(`could not parse the \`Vertical\` union out of ${rel(TYPES_FILE)}.`);
if (!STATUSES) die(`could not parse the \`DemoStatus\` union out of ${rel(TYPES_FILE)}.`); if (!STATUSES) die(`could not parse the \`DemoStatus\` union out of ${rel(TYPES_FILE)}.`);
if (!TAB_IDS) {
die(
`could not parse the \`DemoTabId\` union out of ${rel(TYPES_FILE)}. ` +
'Rule 3 requires one claim per tab, and the list of tabs lives there.',
);
}
/**
* `DEMO_TABS` is the runtime half of `DemoTabId`, and the shell orders the tab
* bar from it. If the two ever disagree, rule 3 would happily pass a demo that
* is missing a claim for a tab the page actually renders — so they are compared
* here, once, before any demo is looked at.
*/
(() => {
let text = null;
try {
text = literalAfter(typesSrc, /(?:export\s+)?const\s+DEMO_TABS\s*(?::\s*[^=]+)?=\s*/, '[');
} catch {
text = null;
}
if (text === null) {
report.fail(
rel(TYPES_FILE),
'rule 3 (one claim per tab)',
'no `DEMO_TABS` array literal found. The union says what a tab id is; the array says what order the ' +
'tabs come in, and the shell reads the array. Both have to exist.',
);
return;
}
const parsed = evalLiteral(text, 'DEMO_TABS');
const listed = Array.isArray(parsed.value) ? parsed.value : null;
report.check(
listed !== null && listed.length === TAB_IDS.length && listed.every((id, i) => id === TAB_IDS[i]),
rel(TYPES_FILE),
'rule 3 (one claim per tab)',
`DEMO_TABS is ${JSON.stringify(listed)} but the DemoTabId union is ${JSON.stringify(TAB_IDS)}. ` +
'They are the same list written twice; a demo cannot be checked against a contract that disagrees ' +
'with itself.',
);
})();
/** /**
* Every icon name lucide-react actually exports, from its own type * Every icon name lucide-react actually exports, from its own type
@@ -97,12 +142,73 @@ const META_FIELDS = {
/* --------------------------------------------------------- literal anchors */ /* --------------------------------------------------------- literal anchors */
const COMPONENT_ANCHORS = [/\bcomponents\s*:\s*/]; const COMPONENT_ANCHORS = [/\bcomponents\s*:\s*/];
const NARRATIVE_ANCHORS = [
/(?:export\s+)?const\s+narrative\s*(?::\s*[^=]+)?=\s*/,
/(?:export\s+)?const\s+[A-Za-z_$][\w$]*Narrative\s*(?::\s*[^=]+)?=\s*/,
/\bnarrative\s*:\s*/,
];
const ANATOMY_ANCHORS = [/(?:export\s+)?const\s+anatomy\s*(?::\s*[^=]+)?=\s*/, /\banatomy\s*:\s*/]; const ANATOMY_ANCHORS = [/(?:export\s+)?const\s+anatomy\s*(?::\s*[^=]+)?=\s*/, /\banatomy\s*:\s*/];
const PROVENANCE_ANCHORS = [ const PROVENANCE_ANCHORS = [
/(?:export\s+)?const\s+provenance\s*(?::\s*[^=]+)?=\s*/, /(?:export\s+)?const\s+provenance\s*(?::\s*[^=]+)?=\s*/,
/\bprovenance\s*:\s*/, /\bprovenance\s*:\s*/,
]; ];
/* --------------------------------------------------------- rule 3 helpers */
/**
* The shortest thing that can still be a claim rather than a heading.
*
* Calibrated against the shortest real one on the site — "Move one slider and
* the winner changes." at 42 characters — with a wide margin, because the point
* is to catch `play: 'Play'`, not to police brevity.
*/
const MIN_CLAIM_CHARS = 20;
const MIN_CLAIM_WORDS = 4;
/** Lower-cased, punctuation stripped, whitespace collapsed. */
const normalise = (text) =>
text
.toLowerCase()
.replace(/[^a-z0-9]+/g, ' ')
.trim();
/**
* Why `claims[tab]` is not a claim, or null if it is one.
*
* A claim is a sentence the tab then has to demonstrate. The three ways one
* arrives broken are: absent, a heading pasted in as prose, and the tab's own
* name — all of which type-check, and all of which render as a page that
* asserts nothing.
*/
function claimProblem(tab, value) {
const label = `narrative.claims.${tab}`;
if (isUnresolved(value)) {
return `${label} is the imported identifier \`${Object.values(value)[0] ?? '?'}\`, not a written-out ` +
'string. The claims are read on the page beside the surface they describe, so they are written here.';
}
if (typeof value !== 'string' || value.trim().length === 0) {
return `${label} is ${value === undefined ? 'missing' : JSON.stringify(value)}. Every tab needs one, ` +
'including a tab this demo may not render — writing the claim is how you find out whether the tab has ' +
'anything in it.';
}
const text = value.trim();
const norm = normalise(text);
// Stripped of the articles and the word "tab", is it just the tab's name?
const bare = norm.replace(/^(?:the|a|an)\s+/, '').replace(/\s+tab$/, '');
if (bare === tab) {
return `${label} is ${JSON.stringify(text)}, which is the tab's own name. The tab bar already says ` +
'that. This is the sentence the tab has to earn.';
}
if (text.length < MIN_CLAIM_CHARS) {
return `${label} is ${text.length} characters (${JSON.stringify(text)}). Under ${MIN_CLAIM_CHARS} it is a ` +
'heading, not a claim — say what the reader will be able to see for themselves on this tab.';
}
if (norm.split(' ').filter(Boolean).length < MIN_CLAIM_WORDS) {
return `${label} is ${JSON.stringify(text)}. A claim is a sentence; ${MIN_CLAIM_WORDS} words is the floor.`;
}
return null;
}
/* ------------------------------------------------------------- the checks */ /* ------------------------------------------------------------- the checks */
const slugs = demoSlugs(); const slugs = demoSlugs();
@@ -142,6 +248,9 @@ for (const slug of slugs) {
report.staticOnly(`${slug}: lucide-react is not installed, so \`icon\` was only checked for PascalCase shape.`); report.staticOnly(`${slug}: lucide-react is not installed, so \`icon\` was only checked for PascalCase shape.`);
} }
/* -- 3. the narrative: a thesis, an anxiety, four claims, a limit ------- */
checkNarrative(slug, report);
/* -- 4. the social card exists ------------------------------------------ */ /* -- 4. the social card exists ------------------------------------------ */
if (nonEmptyString(meta.ogImage)) { if (nonEmptyString(meta.ogImage)) {
const card = abs('public', String(meta.ogImage).replace(/^\/+/, '')); const card = abs('public', String(meta.ogImage).replace(/^\/+/, ''));
@@ -347,6 +456,9 @@ for (const slug of slugs) {
/* -- 13. every step announces itself ------------------------------------ */ /* -- 13. every step announces itself ------------------------------------ */
checkAnnounce(slug, report, manifest); checkAnnounce(slug, report, manifest);
/* -- 14. the page still renders without the interactive half ------------ */
checkEagerHalfStandsAlone(slug, report, metaFile);
} }
/* ---------------------------------------- 8. the shell knows no demo names */ /* ---------------------------------------- 8. the shell knows no demo names */
@@ -412,6 +524,211 @@ report.finish();
/* ------------------------------------------------------------- helpers */ /* ------------------------------------------------------------- helpers */
/**
* Rule 3: the narrative asserts something, tab by tab.
*
* The page is four tabs, not an essay, so the narrative is no longer a list of
* beats with their own surfaces — the shell decides which tabs exist and in
* what order. What the demo owes is the one sentence each tab has to earn, and
* a claim is the field most likely to be left as a heading, because a heading
* type-checks.
*/
function checkNarrative(slug, report) {
const rule = 'rule 3 (one claim per tab)';
const dir = path.join(DEMOS_DIR, slug);
// Lenient, because a narrative may legitimately interpolate a shared
// constant; `claimProblem` then reports the stand-in by name rather than
// letting the whole literal fail to evaluate.
const found = findInDemo(slug, NARRATIVE_ANCHORS, '{', 'narrative', { lenient: true });
if (found.error) {
report.fail(found.file ?? rel(dir), rule, found.error);
return;
}
const narrative = found.value;
const file = found.file;
if (narrative === null || typeof narrative !== 'object') {
report.fail(file, rule, 'the narrative is not an object literal.');
return;
}
report.check(
nonEmptyString(narrative.thesis),
file,
rule,
'narrative.thesis is empty. It is the paragraph that says why this environment is worth the reader\'s ' +
'next five minutes, and no tab supplies it.',
);
report.check(
nonEmptyString(narrative.anxiety),
file,
rule,
'narrative.anxiety is empty. It is the question already in the reader\'s head when they land; the page ' +
'is built to answer it and cannot if it is not written down.',
);
/* The claims. */
if (Array.isArray(narrative.beats) || 'beats' in narrative) {
report.fail(
file,
rule,
'narrative still has `beats`. The page is four tabs now, and which tabs exist is the shell\'s decision, ' +
'not the demo\'s. Replace `beats` with `claims`: one sentence per tab id.',
);
}
const claims = narrative.claims;
if (claims === null || typeof claims !== 'object' || Array.isArray(claims)) {
report.fail(
file,
rule,
`narrative.claims is ${claims === undefined ? 'missing' : JSON.stringify(claims)}. It must be an object ` +
`keyed by tab id: ${TAB_IDS.join(', ')}.`,
);
return;
}
for (const tab of TAB_IDS) {
const problem = claimProblem(tab, claims[tab]);
report.check(problem === null, file, rule, problem ?? '');
}
for (const key of Object.keys(claims)) {
report.check(
TAB_IDS.includes(key),
file,
rule,
`narrative.claims has a key "${key}", which is not a tab. The tabs are ${TAB_IDS.join(', ')}, and they ` +
'are fixed by the contract — a demo that needs a fifth surface adds it through `tabs`, not here.',
);
}
/* The limits. */
const limits = narrative.limits;
if (!Array.isArray(limits) || limits.length === 0) {
report.fail(
file,
rule,
'narrative.limits is empty. Every environment teaches something narrower than the thing it is standing ' +
'in for, and the page that will not say what is the one a reader is right to distrust.',
);
return;
}
limits.forEach((limit, i) => {
report.check(
nonEmptyString(limit?.text),
file,
rule,
`narrative.limits[${i}].text is empty.`,
);
});
}
/**
* Rule 14: the eager half of a demo does not depend on the interactive half.
*
* Play is a tab now, and it is the tab a demo may not have: a demo with no
* `interactive` opens on Watch, and one with no runs has no Watch tab at all.
* That only works if `meta`, `narrative` and `reward` can be rendered on their
* own. `meta.ts` is where it goes wrong first, because it is loaded EAGERLY for
* every demo on every page — an import of `./demo` there drags one demo's board,
* its controls and its word list into the entry chunk for all of them, and does
* it silently.
*
* Only imports that stay inside the demo's own directory are followed. Reaching
* out of it is rule 9's business, and following `@/lib/demo-kit` would walk into
* the shell's own React and report a failure that belongs to nobody.
*/
function checkEagerHalfStandsAlone(slug, report, metaFile) {
const rule = 'rule 14 (the eager half stands alone)';
const dir = path.join(DEMOS_DIR, slug);
const entry = abs(metaFile);
if (!exists(entry)) return;
// Where `interactive` is declared. In practice demo.tsx, but the contract is
// about the value, not the filename.
const interactiveFiles = demoFiles(slug).filter((f) => /\binteractive\s*:/.test(codeOnly(read(f))));
const closure = localClosure(entry, dir);
for (const file of closure) {
if (interactiveFiles.includes(file)) {
report.fail(
rel(entry),
rule,
`reaches ${rel(file)}, which declares \`interactive\`. meta must be renderable with the interactive ` +
'mode absent — it is what the gallery card, the header and the router are built from, and Play is ' +
'the one tab a demo is allowed not to have. Move whatever meta needs into a plain data module.',
);
} else if (/\.tsx$/.test(file)) {
report.fail(
rel(entry),
rule,
`imports ${rel(file)}, a component module. meta is eagerly loaded for EVERY demo on every page, so it ` +
'stays plain serialisable data: no React, no board, no controls. That is also what keeps Play ' +
'optional rather than load-bearing.',
);
} else {
report.passed += 1;
}
}
if (closure.length === 0) report.passed += 1;
// `interactive` present but nothing to play with is the other half of the
// same contract: the shell would render a Play tab over an empty board.
for (const file of interactiveFiles) {
const code = codeOnly(read(file));
report.check(
/\binit\s*:/.test(code) && /\bControls\s*:/.test(code),
rel(file),
rule,
'declares `interactive` without both `init` and `Controls`. The shell shows the Play tab because ' +
'`interactive` exists; a half-declared one is a default tab with nothing in it.',
);
}
}
/** Source with comments and string literals blanked out. */
function codeOnly(src) {
return segment(src)
.map((span) => (span.code ? span.text : ' '))
.join('');
}
/**
* Every file inside `dir` that `entry` imports, transitively. Excludes `entry`.
*/
function localClosure(entry, dir) {
const seen = new Set([entry]);
const queue = [entry];
while (queue.length > 0) {
const file = queue.pop();
for (const spec of importSpecifiers(read(file))) {
const resolved = resolveWithin(spec, file, dir);
if (resolved && !seen.has(resolved)) {
seen.add(resolved);
queue.push(resolved);
}
}
}
seen.delete(entry);
return [...seen].sort();
}
/** Resolve a specifier to a file under `dir`, or null if it leaves it. */
function resolveWithin(spec, fromFile, dir) {
const bare = String(spec).split('?')[0];
let base;
if (bare.startsWith('.')) base = path.resolve(path.dirname(fromFile), bare);
else if (bare.startsWith('@/')) base = abs('src', bare.slice(2));
else return null;
if (path.relative(dir, base).startsWith('..')) return null;
for (const ext of ['', '.ts', '.tsx', '/index.ts', '/index.tsx']) {
const candidate = base + ext;
if (exists(candidate) && fs.statSync(candidate).isFile()) return candidate;
}
return null;
}
/** Every module specifier a file imports, static, dynamic or side-effect. */ /** Every module specifier a file imports, static, dynamic or side-effect. */
function importSpecifiers(src) { function importSpecifiers(src) {
const out = new Set(); const out = new Set();
-49
View File
@@ -1,49 +0,0 @@
import type { ReactNode } from 'react';
import type { StoryBeat } from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
export interface BeatSectionProps {
beat: StoryBeat;
/** 1-based. The narrative is numbered so a reader can be told "see beat 3". */
number: number;
children: ReactNode;
className?: string;
}
/**
* One beat of the exec narrative: a number, a title, a claim, and the surface
* that makes the claim true.
*
* The claim is typeset as an assertion — large, high contrast, above the
* evidence — because the failure mode of a demo site is a visitor watching a
* pretty animation and never learning what it was supposed to prove.
*/
export function BeatSection({ beat, number, children, className }: BeatSectionProps) {
const headingId = `beat-${beat.id}-title`;
return (
<section
id={beat.id}
aria-labelledby={headingId}
data-surface={beat.surface}
className={cn('scroll-mt-[var(--app-header-h)] py-10 lg:py-14', className)}
>
<header className="mb-6 lg:mb-8">
<div className="flex items-baseline gap-3">
<span
aria-hidden="true"
className="nums select-none text-sm font-semibold tabular-nums text-accent-fg"
>
{String(number).padStart(2, '0')}
</span>
<h2 id={headingId} className="text-xl font-semibold tracking-tight lg:text-2xl">
{beat.title}
</h2>
</div>
<p className="mt-3 max-w-2xl text-pretty text-lg leading-snug text-fg lg:text-xl">
{beat.claim}
</p>
</header>
{children}
</section>
);
}
-93
View File
@@ -1,93 +0,0 @@
import type { ComponentType } from 'react';
import { Link } from 'react-router-dom';
import { ArrowRight } from 'lucide-react';
import type { DemoMeta } from '@/lib/demo-kit/types';
import { VERTICAL_LABELS } from '@/lib/demo-kit/registry';
import { Badge } from '@/components/ui/badge';
import { DemoIcon } from '@/components/site/DemoIcon';
import { cn } from '@/lib/utils';
export interface DemoCardProps<T> {
meta: DemoMeta;
/** Defaults to the canonical demo route. */
href?: string;
/**
* The demo's OWN board, drawn compact, as the thumbnail. A screenshot would
* go stale the first time the board changed and nobody would notice; this
* cannot, because it is the same component the demo page renders.
*/
Surface?: ComponentType<{ state: T; compact?: boolean }>;
/** A representative state for the thumbnail — usually a solved board. */
thumbnailState?: T;
className?: string;
}
export function DemoCard<T>({ meta, href, Surface, thumbnailState, className }: DemoCardProps<T>) {
const to = href ?? `/demos/${meta.slug}`;
const isSpec = meta.status === 'spec';
const showSurface = Surface !== undefined && thumbnailState !== undefined;
return (
<article
className={cn(
'card group relative flex flex-col overflow-hidden transition-colors duration-2 ease-enter hover:border-brand/40',
className,
)}
>
<div className="flex items-start gap-3 p-4 pb-3">
<span className="grid size-10 shrink-0 place-items-center rounded-lg bg-accent-subtle text-accent-fg">
<DemoIcon name={meta.icon} className="size-5" />
</span>
<div className="min-w-0 flex-1">
<h3 className="text-base font-semibold leading-tight">
{/* Stretched link: the whole card is the hit target, but there is
still exactly ONE link in the accessibility tree for it. */}
<Link to={to} className="after:absolute after:inset-0 after:content-['']">
{meta.title}
</Link>
</h3>
<p className="mt-0.5 text-sm leading-snug text-muted">{meta.tagline}</p>
</div>
{isSpec ? (
<Badge variant="outline" className="shrink-0 uppercase tracking-wide">
Spec
</Badge>
) : null}
</div>
{showSurface ? (
<div className="mx-4 overflow-hidden rounded-lg bg-surface-2 p-3">
{/* Decorative: the title and tagline already name the demo, and a
board with no run behind it is not information. */}
<div aria-hidden="true">
<Surface state={thumbnailState as T} compact />
</div>
</div>
) : null}
<dl className="mt-3 flex flex-wrap gap-x-4 gap-y-1 px-4 text-xs">
<div className="flex gap-1">
<dt className="text-muted">For</dt>
<dd className="font-medium">{meta.persona}</dd>
</div>
<div className="flex gap-1">
<dt className="text-muted">Vertical</dt>
<dd className="font-medium">{VERTICAL_LABELS[meta.vertical]}</dd>
</div>
</dl>
<p className="mt-2 px-4 pb-4 text-xs leading-relaxed text-muted">
<span className="font-medium text-fg">Reward: </span>
{meta.rewardLine}
</p>
<p className="mt-auto flex items-center gap-1 border-t border-border px-4 py-2.5 text-sm font-medium text-accent-fg">
{isSpec ? 'Read the specification' : 'Open the demo'}
<ArrowRight
className="size-4 transition-transform duration-2 ease-enter group-hover:translate-x-0.5"
aria-hidden="true"
/>
</p>
</article>
);
}
+256 -201
View File
@@ -7,14 +7,14 @@ import { listRuns, loadEpisode, rewardTotal } from '@/lib/demo-kit/episode';
import { usePlayer } from '@/lib/demo-kit/player'; import { usePlayer } from '@/lib/demo-kit/player';
import { loadDemoModule } from '@/lib/demo-kit/registry'; import { loadDemoModule } from '@/lib/demo-kit/registry';
import type { AnyDemoModule } from '@/lib/demo-kit/registry'; import type { AnyDemoModule } from '@/lib/demo-kit/registry';
import type { DemoEpisode, DemoStep, RunRef, StoryBeat } from '@/lib/demo-kit/types'; import type { DemoEpisode, DemoStep, DemoTabId, RunRef } from '@/lib/demo-kit/types';
import { useRunParam, useSpeedParam, useStepParam, useTabParam } from '@/lib/url-state'; import { useRunParam, useSpeedParam, useStepParam, useTabParam } from '@/lib/url-state';
import * as st from '@/content/styles'; import * as st from '@/content/styles';
import { cn } from '@/lib/utils'; import { cn } from '@/lib/utils';
import { BeatSection } from './BeatSection';
import { BlindCompare } from './BlindCompare'; import { BlindCompare } from './BlindCompare';
import { CodeReceipt } from './CodeReceipt'; import { CodeReceipt } from './CodeReceipt';
import { DemoErrorBoundary } from './DemoErrorBoundary'; import { DemoErrorBoundary } from './DemoErrorBoundary';
import { DemoTabBar, TabClaim, resolveTab, visibleTabs } from './DemoTabs';
import { EnvAnatomy } from './EnvAnatomy'; import { EnvAnatomy } from './EnvAnatomy';
import { LimitsCallout } from './LimitsCallout'; import { LimitsCallout } from './LimitsCallout';
import { MetricMover } from './MetricMover'; import { MetricMover } from './MetricMover';
@@ -28,17 +28,24 @@ import { RewardEditor } from './RewardEditor';
import type { RewardArm } from './RewardEditor'; import type { RewardArm } from './RewardEditor';
import { SegmentedControl } from './SegmentedControl'; import { SegmentedControl } from './SegmentedControl';
import { SlotRegion } from './SlotRegion'; import { SlotRegion } from './SlotRegion';
import { StepTimeline } from './StepTimeline';
import { StatStrip } from './StatStrip'; import { StatStrip } from './StatStrip';
import type { Stat } from './StatStrip'; import type { Stat } from './StatStrip';
import { StepTimeline } from './StepTimeline';
import { RecordedBadge, TracePlayer } from './TracePlayer'; import { RecordedBadge, TracePlayer } from './TracePlayer';
import { VerifyBadge } from './VerifyBadge'; import { VerifyBadge } from './VerifyBadge';
import { formatOrDash, useIsDesktop } from './format'; import { formatOrDash, useIsDesktop } from './format';
const REPO_BLOB = 'https://git.karti.ai/PIG/PIG-Demo/src/branch/main/'; const REPO_BLOB = 'https://git.karti.ai/PIG/PIG-Demo/src/branch/main/';
/** The tab the step-detail strip opens on. Kept out of the URL when it is this. */ /**
const DEFAULT_DETAIL_TAB = 'reasoning'; * The panel the step-detail strip inside the Watch tab opens on.
*
* This control is deliberately NOT in the URL. `?tab=` now belongs to the page's
* four top-level tabs, and one param cannot address two nested controls without
* one of them silently winning; a permalink to `?tab=call` would land the reader
* on a page with no such top-level tab.
*/
const DEFAULT_DETAIL_PANEL = 'reasoning';
/** Reserved slug for the shell's own hand-written demo. Dev builds only. */ /** Reserved slug for the shell's own hand-written demo. Dev builds only. */
const MOCK_SLUG = '__mock'; const MOCK_SLUG = '__mock';
@@ -96,7 +103,7 @@ export interface DemoShellProps {
/** /**
* The route component every demo is rendered through. * The route component every demo is rendered through.
* *
* It owns four things and no more: loading, the narrative beats, the URL state, * It owns four things and no more: loading, which tabs exist, the URL state,
* and the page's single polite live region. Everything visual is delegated to * and the page's single polite live region. Everything visual is delegated to
* the surfaces in this directory, and the demo module is never reached into — * the surfaces in this directory, and the demo module is never reached into —
* the shell only ever calls `adapt` and renders `Surface`. * the shell only ever calls `adapt` and renders `Surface`.
@@ -162,7 +169,6 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
const [runParam, setRunParam] = useRunParam(); const [runParam, setRunParam] = useRunParam();
const [stepParam, setStepParam] = useStepParam(); const [stepParam, setStepParam] = useStepParam();
const [tabParam, setTabParam] = useTabParam(DEFAULT_DETAIL_TAB);
const [speedParam, setSpeedParam] = useSpeedParam(); const [speedParam, setSpeedParam] = useSpeedParam();
const run = useMemo( const run = useMemo(
@@ -192,11 +198,22 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
// re-run the effect on the player's own advance and fight it. // re-run the effect on the player's own advance and fight it.
}, [stepParam, seek]); }, [stepParam, seek]);
const hasBeat = (surface: StoryBeat['surface']) => // Derived, never declared. A demo that ships no interactive mode has no Play
demo.narrative.beats.some((beat) => beat.surface === surface); // tab and opens on Watch; one whose traces failed to load has no Watch tab
const timelineInSplit = !hasBeat('scrubber'); // and opens on Reward.
const extras = demo.tabs ?? []; const hasRecording = Boolean(run && episode && steps.length > 0);
const extrasInCustomBeat = hasBeat('custom'); const tabs = useMemo(
() => visibleTabs({ play: Boolean(demo.interactive), watch: hasRecording }),
[demo.interactive, hasRecording],
);
// `visibleTabs` always keeps `reward` and `evidence`, so index 0 exists; the
// fallback is here only so the type does not need an assertion.
const defaultTab: DemoTabId = tabs[0] ?? 'evidence';
const [tabParam, setTabParam] = useTabParam(defaultTab);
const activeTab = resolveTab(tabParam, tabs, defaultTab);
// React-only, not a URL param. See DEFAULT_DETAIL_PANEL.
const [detailPanel, setDetailPanel] = useState(DEFAULT_DETAIL_PANEL);
const arms = useMemo<RewardArm[]>( const arms = useMemo<RewardArm[]>(
() => () =>
@@ -230,54 +247,39 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
return null; return null;
}, [runs, episodes]); }, [runs, episodes]);
if (!run || !episode || steps.length === 0) {
return (
<main className={cn(st.shell, 'py-16')}>
<h1 className={st.h2}>{demo.meta.title}</h1>
<p className={cn(st.prose, 'mt-3 max-w-prose')}>
No recorded run is available for this demo yet. The environment and its grader are in
the repository; the traces come from the eval command on the provenance card.
</p>
<EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} className="mt-8" />
</main>
);
}
const Surface = demo.Surface as ComponentType<{ state: unknown; compact?: boolean }>; const Surface = demo.Surface as ComponentType<{ state: unknown; compact?: boolean }>;
const current = steps[player.index]; const current = steps[player.index];
const lastStep = steps[steps.length - 1]; const claims = demo.narrative.claims;
const extras = demo.tabs ?? [];
const heroStats: Stat[] = [ // The seed is the shared coordinate between the visitor's board and the
{ // agent's. With no recording to match, the demo's own first board will do.
label: 'Outcome', const playSeed = run?.seed ?? 0;
value: episode.outcome,
tone: episode.outcome === 'solved' ? 'positive' : 'warning',
...(episode.truncated ? { hint: 'Truncated before a terminal state' } : {}),
},
{
label: 'Total reward',
value: formatOrDash(rewardTotal(episode.rewards, demo.reward.components)),
tone: 'brand',
hint: 'Shipped weights',
},
{ label: 'Steps', value: steps.length, hint: 'Model calls in this run' },
{ label: 'Seed', value: episode.seed, hint: 'The same seed reproduces this board' },
];
const timeline = ( const headerStats: Stat[] = episode
<StepTimeline ? [
steps={steps} {
current={player.index} label: 'Outcome',
onSelect={(next) => { value: episode.outcome,
player.pause(); tone: episode.outcome === 'solved' ? 'positive' : 'warning',
player.seek(next); ...(episode.truncated ? { title: 'Truncated before a terminal state' } : {}),
}} },
Surface={Surface} {
onTogglePlay={player.toggle} label: 'Reward',
/> value: formatOrDash(rewardTotal(episode.rewards, demo.reward.components)),
); tone: 'brand',
title: 'Total under the shipped weights',
},
{ label: 'Steps', value: steps.length, title: 'Model calls in this run' },
{
label: 'Seed',
value: episode.seed,
title: 'The same seed reproduces this board',
},
]
: [];
const detailTabs: { id: string; label: string; content: ReactNode }[] = [ const detailPanels: { id: string; label: string; content: ReactNode }[] = [
{ {
id: 'reasoning', id: 'reasoning',
label: 'Reasoning', label: 'Reasoning',
@@ -319,51 +321,100 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
</div> </div>
), ),
}, },
...(extrasInCustomBeat // A demo's own extra panels ride alongside the step detail, where they sit
? [] // next to the step they are almost always about. With no recording there is
: extras.map((tab) => ({ id: tab.id, label: tab.label, content: <tab.Component /> }))), // no step detail, so the Evidence tab picks them up instead.
...(hasRecording
? extras.map((tab) => ({ id: tab.id, label: tab.label, content: <tab.Component /> }))
: []),
]; ];
const activeTab = detailTabs.some((tab) => tab.id === tabParam) ? tabParam : DEFAULT_DETAIL_TAB; const activeDetail = detailPanels.some((panel) => panel.id === detailPanel)
? detailPanel
: DEFAULT_DETAIL_PANEL;
const renderSurface = (beat: StoryBeat): ReactNode => { const timeline = (
switch (beat.surface) { <StepTimeline
case 'hero': steps={steps}
return ( current={player.index}
<div className="grid grid-cols-1 gap-4 lg:grid-cols-[auto_minmax(0,1fr)] lg:items-start"> onSelect={(next) => {
<div className="card w-fit p-4"> player.pause();
{lastStep ? <Surface state={lastStep.state} /> : null} player.seek(next);
</div> }}
<div className="space-y-3"> Surface={Surface}
<StatStrip stats={heroStats} /> onTogglePlay={player.toggle}
<RecordedBadge />
model={run.model} );
capturedAt={run.capturedAt}
{...(run.intervention ? { intervention: run.intervention } : {})} return (
className="ml-0 w-fit" <main className={cn(st.shell, 'pb-16')}>
/> {/*
<p className={cn(st.prose, 'max-w-prose')}>{demo.narrative.thesis}</p> The page's ONE live region. Every step change lands here and nowhere
</div> else: with reduced motion the board animation is gone, so this sentence
is the only thing that tells a screen-reader user what just happened.
*/}
<div aria-live="polite" aria-atomic="true" className="sr-only">
{current?.announce ?? ''}
</div>
<header className="pt-6 sm:pt-8">
<p className={cn(st.eyebrow, 'text-accent-fg')}>For {demo.meta.persona}</p>
<h1 className={cn(st.h2, 'mt-1')}>{demo.meta.title}</h1>
<p className={cn(st.lede, 'mt-2 max-w-prose')}>{demo.meta.tagline}</p>
{/* The buyer's question, kept quiet on purpose: it is the thing they
walked in with, not the thing this page is asserting. */}
<p className="mt-2 max-w-prose text-sm italic leading-relaxed text-muted">
{demo.narrative.anxiety}
</p>
{/*
The stats describe the RECORDED RUN, so they only belong on the tabs
whose subject is that run. On Play they sat above the visitor's own
empty board reading "Outcome: failed", which parses as *your* game
having already failed before you have touched a key.
*/}
{run && headerStats.length > 0 && activeTab !== 'play' ? (
// One line that scrolls itself rather than a block that wraps: every
// row this header spends is a row of the interactive board pushed
// below the fold on a 390px phone.
<div className="mt-3 flex items-center gap-2 overflow-x-auto pb-1 sm:flex-wrap sm:overflow-visible sm:pb-0">
<StatStrip stats={headerStats} />
<RecordedBadge
model={run.model}
capturedAt={run.capturedAt}
{...(run.intervention ? { intervention: run.intervention } : {})}
className="ml-0 shrink-0 flex-nowrap"
/>
</div> </div>
); ) : null}
<SlotRegion id="hero-aside" />
</header>
case 'anatomy': <Tabs value={activeTab} onValueChange={setTabParam} className="mt-4 sm:mt-5">
return <EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} />; <DemoTabBar tabs={tabs} />
case 'split-play': {tabs.includes('play') ? (
return ( // `forceMount` keeps the visitor's half-finished board alive while
<div className="space-y-3"> // they read the other tabs, so a game in progress survives a trip to
{demo.interactive ? ( // Reward and back. Radix leaves the hiding to the author under
<> // `forceMount`, which is what the `data-[state=inactive]` class does —
<PlayYourself demo={demo} seed={run.seed} /> // it is load-bearing, not belt-and-braces.
<div className="space-y-1 pt-4"> <TabsContent
<h3 className={st.h3}>What the model did</h3> value="play"
<p className={cn(st.prose, 'max-w-prose text-sm')}> forceMount
Same hidden answer, same rules, same budget replayed at the className="mt-5 space-y-5 data-[state=inactive]:hidden sm:mt-6 sm:space-y-6"
speed it actually happened. >
</p> <TabClaim>{claims.play}</TabClaim>
</div> <PlayYourself demo={demo} seed={playSeed} />
</> <SlotRegion id="below-board" />
) : null} <div className="space-y-2">
<h2 className={st.h3}>The machine you are inside</h2>
<EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} compact />
</div>
</TabsContent>
) : null}
{tabs.includes('watch') && run && episode ? (
<TabsContent value="watch" className="mt-6 space-y-4">
<TabClaim>{claims.watch}</TabClaim>
{runs.length > 1 ? ( {runs.length > 1 ? (
<RunSwitcher <RunSwitcher
@@ -411,31 +462,23 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
</div> </div>
<div className="min-w-0"> <div className="min-w-0">
<Tabs value={activeTab} onValueChange={setTabParam}> <Tabs value={activeDetail} onValueChange={setDetailPanel}>
<TabsList aria-label="Details for this step" className="w-full overflow-x-auto"> <TabsList aria-label="Details for this step" className="w-full overflow-x-auto">
{detailTabs.map((tab) => ( {detailPanels.map((panel) => (
<TabsTrigger key={tab.id} value={tab.id}> <TabsTrigger key={panel.id} value={panel.id}>
{tab.label} {panel.label}
</TabsTrigger> </TabsTrigger>
))} ))}
</TabsList> </TabsList>
{detailTabs.map((tab) => ( {detailPanels.map((panel) => (
<TabsContent key={tab.id} value={tab.id}> <TabsContent key={panel.id} value={panel.id}>
{tab.content} {panel.content}
</TabsContent> </TabsContent>
))} ))}
</Tabs> </Tabs>
</div> </div>
</div> </div>
{timelineInSplit ? timeline : null}
<SlotRegion id="below-board" />
</div>
);
case 'scrubber':
return (
<div className="space-y-3">
{timeline} {timeline}
{player.timingIsReal ? null : ( {player.timingIsReal ? null : (
<p className="text-xs text-muted"> <p className="text-xs text-muted">
@@ -444,42 +487,7 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
</p> </p>
)} )}
<SlotRegion id="below-timeline" /> <SlotRegion id="below-timeline" />
</div>
);
case 'reward-editor':
return (
<div className="space-y-4">
<RewardBreakdown
spec={demo.reward}
values={episode.rewards}
{...(episode.metrics ? { metrics: episode.metrics } : {})}
/>
<VerifyBadge demo={demo} episode={episode} />
{arms.length > 1 ? <RewardEditor spec={demo.reward} arms={arms} /> : null}
</div>
);
case 'metric': {
const currentTotal = rewardTotal(episode.rewards, demo.reward.components);
const points = arms
.map((arm) => ({ x: arm.label, y: rewardTotal(arm.values, demo.reward.components) }))
.filter((point): point is { x: string; y: number } => point.y !== null);
const baselineArm = arms[0];
const baselineValue = baselineArm
? rewardTotal(baselineArm.values, demo.reward.components)
: null;
return (
<div className="space-y-4">
<MetricMover
label={`Total reward — ${run.label}`}
value={currentTotal ?? 0}
{...(baselineArm && baselineValue !== null && arms.length > 1
? { baseline: { value: baselineValue, label: baselineArm.label } }
: {})}
series={points}
caption="Every point is a recorded run scored by the same grader. Nothing here is a projection."
/>
{blindPair ? ( {blindPair ? (
<BlindCompare <BlindCompare
seed={blindPair.left.seed} seed={blindPair.left.seed}
@@ -506,14 +514,63 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
}} }}
/> />
) : null} ) : null}
</div> </TabsContent>
); ) : null}
}
<TabsContent value="reward" className="mt-6 space-y-4">
<TabClaim>{claims.reward}</TabClaim>
{episode ? (
<RewardBreakdown
spec={demo.reward}
values={episode.rewards}
{...(episode.metrics ? { metrics: episode.metrics } : {})}
/>
) : (
<>
<p className={cn(st.prose, 'max-w-prose')}>
No recorded run has been scored for this environment yet, so every term below
reads as not scored rather than as zero. The weights are the ones the environment
ships.
</p>
<RewardBreakdown spec={demo.reward} values={{}} />
</>
)}
{/* The editor carries the ranking it re-orders in its own right-hand
column, so the two are never on screen apart. */}
{arms.length > 1 ? <RewardEditor spec={demo.reward} arms={arms} /> : null}
{episode ? <VerifyBadge demo={demo} episode={episode} /> : null}
{run && episode ? (
<HeadlineMetric
label={`Total reward — ${run.label}`}
arms={arms}
components={demo.reward.components}
currentRewards={episode.rewards}
/>
) : null}
<SlotRegion id="beside-reward" />
</TabsContent>
<TabsContent value="evidence" className="mt-6 space-y-6">
<TabClaim>{claims.evidence}</TabClaim>
{/*
`narrative.thesis` is a required field of the contract and the only
paragraph on a demo that argues for the environment as a whole
rather than for one tab. It has to be SOMEWHERE, and this is the
tab a visitor opens to read rather than to do — Play stays a board
above the fold, which is the one thing a paragraph here would cost.
*/}
<p className={cn(st.prose, 'max-w-prose')}>{demo.narrative.thesis}</p>
<EnvAnatomy anatomy={demo.anatomy} rewardLine={demo.meta.rewardLine} />
case 'receipt':
return (
<div className="grid grid-cols-1 gap-4 lg:grid-cols-2 lg:items-start"> <div className="grid grid-cols-1 gap-4 lg:grid-cols-2 lg:items-start">
<ProvenanceCard provenance={demo.provenance} run={run} /> <ProvenanceCard provenance={demo.provenance} {...(run ? { run } : {})} />
<CodeReceipt <CodeReceipt
code={demo.reward.source.code} code={demo.reward.source.code}
path={demo.reward.source.path} path={demo.reward.source.path}
@@ -522,55 +579,57 @@ function DemoBody({ bundle }: { bundle: DemoBundle }) {
/> />
<SlotRegion id="after-receipts" className="lg:col-span-2" /> <SlotRegion id="after-receipts" className="lg:col-span-2" />
</div> </div>
);
case 'limits': {!hasRecording && extras.length > 0 ? (
return <LimitsCallout limits={demo.narrative.limits} />; <div className="space-y-4">
{extras.map((tab) => (
<tab.Component key={tab.id} />
))}
</div>
) : null}
case 'custom':
return extras.length > 0 ? (
<div className="space-y-4">
{extras.map((tab) => (
<tab.Component key={tab.id} />
))}
</div>
) : (
<SlotRegion id="before-limits" /> <SlotRegion id="before-limits" />
); <LimitsCallout limits={demo.narrative.limits} />
</TabsContent>
</Tabs>
</main>
);
}
default: /**
return null; * The headline metric, with every recorded arm on the same line.
} *
}; * Arms that were never scored are dropped rather than plotted at zero — the
* difference between "scored badly" and "not scored" is the site's whole
* argument, and a chart is the easiest place in the world to lose it.
*/
function HeadlineMetric({
label,
arms,
components,
currentRewards,
}: {
label: string;
arms: RewardArm[];
components: AnyDemoModule['reward']['components'];
currentRewards: DemoEpisode['rewards'];
}) {
const points = arms
.map((arm) => ({ x: arm.label, y: rewardTotal(arm.values, components) }))
.filter((point): point is { x: string; y: number } => point.y !== null);
const baselineArm = arms[0];
const baselineValue = baselineArm ? rewardTotal(baselineArm.values, components) : null;
return ( return (
<main className={cn(st.shell, 'pb-16')}> <MetricMover
{/* label={label}
The page's ONE live region. Every step change lands here and nowhere value={rewardTotal(currentRewards, components) ?? 0}
else: with reduced motion the board animation is gone, so this sentence {...(baselineArm && baselineValue !== null && arms.length > 1
is the only thing that tells a screen-reader user what just happened. ? { baseline: { value: baselineValue, label: baselineArm.label } }
*/} : {})}
<div aria-live="polite" aria-atomic="true" className="sr-only"> series={points}
{current?.announce ?? ''} caption="Every point is a recorded run scored by the same grader. Nothing here is a projection."
</div> />
<header className="pt-8">
<p className={cn(st.eyebrow, 'text-accent-fg')}>For {demo.meta.persona}</p>
<h1 className={cn(st.h2, 'mt-1')}>{demo.meta.title}</h1>
<p className={cn(st.lede, 'mt-2 max-w-prose')}>{demo.meta.tagline}</p>
<p className="mt-4 max-w-prose border-l-2 border-brand pl-3 text-sm italic leading-relaxed text-fg">
{demo.narrative.anxiety}
</p>
</header>
<div className="divide-y divide-border">
{demo.narrative.beats.map((beat, index) => (
<BeatSection key={beat.id} beat={beat} number={index + 1}>
{renderSurface(beat)}
</BeatSection>
))}
</div>
</main>
); );
} }
@@ -647,7 +706,6 @@ function RunSwitcher({
); );
} }
/** /**
* The loading state. Shaped like the page it becomes, and with no spinner: a * The loading state. Shaped like the page it becomes, and with no spinner: a
* spinner here would imply a live model call, which is the one thing the whole * spinner here would imply a live model call, which is the one thing the whole
@@ -659,12 +717,9 @@ function ShellSkeleton() {
<p className="sr-only">Loading the recorded run.</p> <p className="sr-only">Loading the recorded run.</p>
<Skeleton className="h-8 w-64" /> <Skeleton className="h-8 w-64" />
<Skeleton className="mt-3 h-4 w-full max-w-md" /> <Skeleton className="mt-3 h-4 w-full max-w-md" />
<div className="mt-10 grid grid-cols-1 gap-3 sm:grid-cols-2 lg:grid-cols-4"> <Skeleton className="mt-4 h-10 w-full max-w-sm" />
{[0, 1, 2, 3].map((index) => ( <Skeleton className="mt-6 h-11 w-full max-w-md" />
<Skeleton key={index} className="h-28" /> <Skeleton className="mt-6 h-80" />
))}
</div>
<Skeleton className="mt-6 h-64" />
</div> </div>
); );
} }
+126
View File
@@ -0,0 +1,126 @@
/**
* The demo page's tab list, and the rules that decide which tabs exist.
*
* A demo never declares its tabs. The shell derives them from what the demo
* actually has, which is the only way the four labels can mean the same thing
* on every page: `Play` is always the visitor doing the task, `Watch` is always
* a recorded attempt, and the absence of one is a fact about the demo rather
* than an authoring choice someone forgot to make.
*/
import type { ReactNode } from 'react';
import { TabsList, TabsTrigger } from '@/components/ui/tabs';
import { DEMO_TABS } from '@/lib/demo-kit/types';
import type { DemoTabId } from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
/** What a person reads. One word each, so four of them fit at 390px. */
export const TAB_LABELS: Readonly<Record<DemoTabId, string>> = {
play: 'Play',
watch: 'Watch',
reward: 'Reward',
evidence: 'Evidence',
};
/** Read to a screen reader, where a one-word label is not enough context. */
const TAB_DESCRIPTIONS: Readonly<Record<DemoTabId, string>> = {
play: 'Play the environment yourself',
watch: 'Watch a recorded agent attempt',
reward: 'The reward, and what happens when you change it',
evidence: 'The environment, its provenance and its limits',
};
export interface TabAvailability {
/** The demo ships an interactive mode. */
play: boolean;
/** At least one recorded run adapted to at least one step. */
watch: boolean;
}
/**
* The tabs this demo has, in contract order.
*
* `reward` and `evidence` are unconditional: a demo with neither an interactive
* mode nor a recorded run is still an environment with a grader you can read,
* and that is the one thing the site refuses to leave out.
*/
export function visibleTabs(has: TabAvailability): DemoTabId[] {
return DEMO_TABS.filter((id) => {
if (id === 'play') return has.play;
if (id === 'watch') return has.watch;
return true;
});
}
/**
* Coerce whatever the URL says into a tab that exists.
*
* An unknown or unavailable `?tab=` lands on the default rather than rendering
* an empty page — a stale permalink to `?tab=watch` on a demo whose traces were
* pulled must still show something.
*/
export function resolveTab(
raw: string,
visible: readonly DemoTabId[],
fallback: DemoTabId,
): DemoTabId {
return visible.find((id) => id === raw) ?? fallback;
}
/**
* The tab strip.
*
* Full width and equal columns below `sm` so four tabs land inside 390px
* without the list becoming a horizontal scroller — a tab bar you have to
* scroll hides the tabs, which is the one thing it exists to advertise. From
* `sm` up it shrinks back to its natural width and sits left.
*/
export function DemoTabBar({
tabs,
className,
}: {
tabs: readonly DemoTabId[];
className?: string;
}) {
return (
<TabsList
aria-label="How to explore this environment"
className={cn('flex w-full sm:inline-flex sm:w-auto', className)}
>
{tabs.map((id) => (
<TabsTrigger
key={id}
value={id}
title={TAB_DESCRIPTIONS[id]}
className="min-w-0 flex-1 px-2 text-[0.8125rem] sm:flex-none sm:px-3 sm:text-sm"
>
{TAB_LABELS[id]}
</TabsTrigger>
))}
</TabsList>
);
}
/**
* The sentence a tab has to earn, typeset as an assertion rather than a
* heading.
*
* The failure mode of a demo site is a visitor watching something pretty and
* never learning what it was supposed to prove. The claim is the first thing in
* every panel for that reason, and it is deliberately not an `<h2>`: the tab
* trigger is already this panel's accessible name, and a heading here would
* make the claim navigable furniture instead of a thing someone reads.
*/
export function TabClaim({ children, className }: { children: ReactNode; className?: string }) {
return (
<p
className={cn(
'max-w-2xl text-pretty text-base leading-snug text-fg sm:text-lg lg:text-xl',
className,
)}
>
{children}
</p>
);
}
+21 -33
View File
@@ -5,22 +5,12 @@ import { cn } from '@/lib/utils';
export type StatTone = 'default' | 'positive' | 'warning' | 'danger' | 'info' | 'brand'; export type StatTone = 'default' | 'positive' | 'warning' | 'danger' | 'info' | 'brand';
export interface Stat { export interface Stat {
/** Short. Two or three words; it sits above the number. */ /** Short. Two or three words; it sits beside the number. */
label: string; label: string;
value: ReactNode; value: ReactNode;
/** One clarifying line, shown under the number at a smaller size. */
hint?: string;
tone?: StatTone; tone?: StatTone;
/** Set when this number was derived under an edited reward, not recorded. */ /** One clarifying line. A tooltip here, because the strip is one line high. */
edited?: boolean; title?: string;
}
export interface StatStripProps {
stats: Stat[];
className?: string;
/** Announce changes as they happen. Off by default — the shell owns the
* page's single live region and two competing ones talk over each other. */
live?: boolean;
} }
const TONE: Record<StatTone, string> = { const TONE: Record<StatTone, string> = {
@@ -33,40 +23,38 @@ const TONE: Record<StatTone, string> = {
}; };
/** /**
* A row of headline numbers. Scrolls horizontally on a phone rather than * The run's headline numbers, one line high.
* wrapping into a ragged grid: four stats reflowing to 2x2 at 390px puts the *
* least important number in the most prominent corner. * This used to be a row of cards, which is the right thing in the middle of a
* page and the wrong thing in a header: four stat cards push the interactive
* board below the fold on a 390px phone, and the board arriving above the fold
* is what the whole page is now organised around. So the cards are gone and
* this is the only stat strip — a second, compact variant living beside the
* card version is how the two drift into disagreeing about what a stat looks
* like.
*
* It scrolls itself rather than wrapping: every row this header spends is a row
* of the board pushed down.
*/ */
export function StatStrip({ stats, className, live = false }: StatStripProps) { export function StatStrip({ stats, className }: { stats: Stat[]; className?: string }) {
if (stats.length === 0) return null; if (stats.length === 0) return null;
return ( return (
<dl <dl
className={cn( className={cn(
'flex snap-x snap-mandatory gap-3 overflow-x-auto pb-1', 'flex shrink-0 items-baseline gap-x-4 rounded-lg border border-border bg-surface-2 px-3 py-2',
'sm:grid sm:snap-none sm:overflow-visible sm:pb-0',
stats.length <= 2 ? 'sm:grid-cols-2' : 'sm:grid-cols-3 lg:grid-cols-4',
className, className,
)} )}
{...(live ? { 'aria-live': 'polite' as const } : {})}
> >
{stats.map((stat) => ( {stats.map((stat) => (
<div <div
key={stat.label} key={stat.label}
className="card min-w-[9.5rem] flex-1 shrink-0 snap-start px-4 py-3" className="flex shrink-0 items-baseline gap-1.5"
{...(stat.title ? { title: stat.title } : {})}
> >
<dt className="flex items-center gap-1.5 text-xs font-medium uppercase tracking-wide text-muted"> <dt className="text-xs font-medium uppercase tracking-wide text-muted">{stat.label}</dt>
<span className="truncate">{stat.label}</span> <dd className={cn('nums text-sm font-semibold', TONE[stat.tone ?? 'default'])}>
{stat.edited ? <EditedChip /> : null}
</dt>
<dd
className={cn(
'nums mt-1 text-2xl font-semibold leading-tight',
TONE[stat.tone ?? 'default'],
)}
>
{stat.value} {stat.value}
</dd> </dd>
{stat.hint ? <dd className="mt-0.5 text-xs text-muted">{stat.hint}</dd> : null}
</div> </div>
))} ))}
</dl> </dl>
+121 -10
View File
@@ -8,7 +8,13 @@
* has something complete to render — including the awkward cases a real trace * has something complete to render — including the awkward cases a real trace
* eventually produces: a step with no reasoning, a null model call, a * eventually produces: a step with no reasoning, a null model call, a
* not-scored reward component, and a truncated run. * not-scored reward component, and a truncated run.
*
* It ships an `interactive` mode for the same reason. Play is the tab the page
* opens on, so a fixture without one would leave the shell's default tab the
* only surface here with nothing to render against.
*/ */
import { useCallback, useId, useState } from 'react';
import type { import type {
DemoEpisode, DemoEpisode,
DemoModule, DemoModule,
@@ -16,6 +22,7 @@ import type {
RewardValues, RewardValues,
RunRef, RunRef,
} from '@/lib/demo-kit/types'; } from '@/lib/demo-kit/types';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils'; import { cn } from '@/lib/utils';
export type MarkKind = 'exact' | 'present' | 'absent'; export type MarkKind = 'exact' | 'present' | 'absent';
@@ -104,6 +111,112 @@ function MockSurface({ state, compact = false }: { state: MockState; compact?: b
); );
} }
/* ------------------------------------------------------------- interactive */
/**
* Answers the playable board can draw. Small on purpose: this is a fixture.
*
* Ordered so that seed 7 — the seed the recorded runs below use — draws the
* same answer they were recorded against, which keeps Play and Watch showing
* the same puzzle while the shell is being built.
*/
const PLAYABLE = ['SLATE', 'PRIZE', 'MOUND', ANSWER] as const;
/** Deterministic in the seed, as the contract requires. */
function initMock(seed: number): MockState {
const answer = PLAYABLE[Math.abs(Math.trunc(seed)) % PLAYABLE.length] as string;
return { answer, guesses: [], solved: false };
}
const LETTERS = /^[A-Za-z]*$/;
/**
* The playable controls: type a five-letter word, submit, see it marked.
*
* Deliberately an input rather than an on-screen keyboard. The shell only ever
* sees `onChange(next)`, so the fixture's job is to produce every state a real
* demo's controls can — mid-word, illegal, solved, out of guesses — with as
* little of its own machinery as possible.
*/
function MockControls({
state,
onChange,
seed,
}: {
state: MockState;
onChange: (next: MockState) => void;
seed: number;
}) {
const [draft, setDraft] = useState('');
const fieldId = useId();
const over = state.solved || state.guesses.length >= MAX_GUESSES;
const ready = draft.length === 5 && !over;
const submit = useCallback(() => {
if (!ready) return;
const word = draft.toUpperCase();
const guesses = [...state.guesses, { word, marks: mark(word, state.answer) }];
onChange({ ...state, guesses, solved: word === state.answer });
setDraft('');
}, [draft, onChange, ready, state]);
const status = state.solved
? `Solved in ${state.guesses.length} ${state.guesses.length === 1 ? 'guess' : 'guesses'}.`
: over
? `Out of guesses. The word was ${state.answer}.`
: `${MAX_GUESSES - state.guesses.length} guesses left.`;
return (
<div className="flex flex-col gap-3">
<form
className="flex flex-wrap items-center gap-2"
onSubmit={(event) => {
event.preventDefault();
submit();
}}
>
<label className="sr-only" htmlFor={fieldId}>
Your five-letter guess
</label>
<input
id={fieldId}
value={draft}
disabled={over}
onChange={(event) => {
const next = event.target.value.slice(0, 5);
if (LETTERS.test(next)) setDraft(next.toUpperCase());
}}
autoComplete="off"
autoCapitalize="characters"
spellCheck={false}
inputMode="text"
placeholder="GUESS"
className="tap min-w-0 flex-1 rounded-md border border-border bg-surface px-3 font-mono text-base uppercase tracking-[0.3em] text-fg placeholder:tracking-normal placeholder:text-muted focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-brand disabled:opacity-50"
/>
<Button type="submit" size="touch" disabled={!ready}>
Guess
</Button>
<Button
type="button"
variant="outline"
size="touch"
onClick={() => {
setDraft('');
onChange(initMock(seed));
}}
>
Reset
</Button>
</form>
{/* The board is an image to a screen reader, so the outcome has to be
said in words somewhere that announces itself. */}
<p aria-live="polite" className="text-sm text-muted">
{status}
</p>
</div>
);
}
const REWARD_SOURCE = `import verifiers as vf const REWARD_SOURCE = `import verifiers as vf
@@ -192,15 +305,12 @@ export const mockDemo: DemoModule<MockState> = {
thesis: thesis:
'An environment is an eval you can take the gradient of. The task, the legal moves and the grader are all code — so you can change what "good" means and watch the number move.', 'An environment is an eval you can take the gradient of. The task, the legal moves and the grader are all code — so you can change what "good" means and watch the number move.',
anxiety: 'Is this a benchmark I read, or a thing I can actually change?', anxiety: 'Is this a benchmark I read, or a thing I can actually change?',
beats: [ claims: {
{ id: 'hero', title: 'The run', claim: 'This is a recorded rollout, not a live request.', surface: 'hero' }, play: 'Play a round yourself, because the rest of this page is about what happened when a model played the same one.',
{ id: 'anatomy', title: 'The machine', claim: 'Four boxes: task, legal actions, grader, score.', surface: 'anatomy' }, watch: 'This is a recorded attempt, replayed one turn at a time, with the reasoning and the token cost of every turn attached.',
{ id: 'play', title: 'Watch it think', claim: 'Every move has a reason and a cost, both recorded.', surface: 'split-play' }, reward: 'Move a single weight and the ranking of the runs re-orders underneath it. That is the whole product.',
{ id: 'reward', title: 'Change what good means', claim: 'Move a weight and the ranking moves with it.', surface: 'reward-editor' }, evidence: 'The grader is thirty lines of Python, printed here beside the command that ran it and the runs it produced.',
{ id: 'metric', title: 'The number that moves', claim: 'The score is a measurement, not a claim.', surface: 'metric' }, },
{ id: 'receipt', title: 'The receipts', claim: 'Every number here has a command that reproduces it.', surface: 'receipt' },
{ id: 'limits', title: 'What this does not teach', claim: 'A word game is not your business process.', surface: 'limits' },
],
limits: [ limits: [
{ {
text: 'A five-letter word has one right answer. Most business decisions do not, and a grader that pretends otherwise scores confidence rather than correctness.', text: 'A five-letter word has one right answer. Most business decisions do not, and a grader that pretends otherwise scores confidence rather than correctness.',
@@ -239,7 +349,7 @@ export const mockDemo: DemoModule<MockState> = {
{ key: 'guesses_used', label: 'Guesses used', description: 'How many of the six were spent.' }, { key: 'guesses_used', label: 'Guesses used', description: 'How many of the six were spent.' },
{ key: 'unique_letters', label: 'Unique letters tried', description: 'Breadth of the search.' }, { key: 'unique_letters', label: 'Unique letters tried', description: 'Breadth of the search.' },
], ],
source: { path: 'envs/wordle_five/wordle_five/rewards.py', code: REWARD_SOURCE, marker: '--8<-- efficiency' }, source: { path: 'envs/mock_five/mock_five/rewards.py', code: REWARD_SOURCE, marker: '--8<-- efficiency' },
}, },
provenance: { provenance: {
envPackage: 'wordle_five', envPackage: 'wordle_five',
@@ -258,6 +368,7 @@ export const mockDemo: DemoModule<MockState> = {
}, },
adapt, adapt,
Surface: MockSurface, Surface: MockSurface,
interactive: { init: initMock, Controls: MockControls },
verify, verify,
}; };
+441
View File
@@ -0,0 +1,441 @@
/**
* The environment picker: one card shape, two pages.
*
* Home leads with it and Gallery filters it, so the card lives here rather than
* twice. The list is built at module scope from the registry and the vertical
* lineup — creating `src/demos/<slug>/` adds a card with no edit to this file,
* which is the same discovery-by-existence property the registry itself has.
*
* The asymmetry is the honest part. Almost everything in the lineup is written
* rather than built, and a wall of identical tiles would imply a wall of
* working demos on a site whose entire argument is that its claims are
* checkable. So a built environment gets its own board, its own block and the
* only filled call to action, and the written ones read as a published roadmap
* underneath it: dimmed, badged, and each linking to a real specification
* rather than to nothing. Both pages count them from the data, never from a
* number typed into the copy.
*/
import { useEffect, useState, type ReactNode } from 'react';
import { Link } from 'react-router-dom';
import { ArrowRight, FileText, Play } from 'lucide-react';
import { DemoIcon } from '@/components/site/DemoIcon';
import { Badge } from '@/components/ui/badge';
import { Skeleton } from '@/components/ui/skeleton';
import { lineup, routes } from '@/content/lineup';
import * as s from '@/content/styles';
import {
listDemos,
listVerticals,
loadDemoModule,
VERTICAL_LABELS,
VERTICAL_ORDER,
} from '@/lib/demo-kit/registry';
import type { DemoStatus, Vertical } from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
/** One card's worth of environment, whether it is built or only written. */
export interface EnvironmentEntry {
/** React key. Namespaced, because a demo and a vertical can share a slug. */
id: string;
title: string;
/** One sentence on what the agent DOES. */
tagline: string;
/** The job title that owns the budget for it. */
persona: string;
/** What the reward pays for, and what it takes away. */
rewardLine: string;
/** A lucide icon name, resolved by `DemoIcon`. */
icon: string;
status: DemoStatus;
href: string;
/** Exec-facing vertical name. */
verticalLabel: string | null;
/** Filter identity. `null` matches no filter and shows only under "All". */
verticalKey: Vertical | null;
/** Set when the entry is a built demo; drives the board thumbnail. */
slug: string | null;
/** A standing qualifier printed on the card, e.g. "Not in the first set". */
note: string | null;
}
/**
* The card wants one sentence. `verticals.ts` writes two or three, because the
* vertical page needs the whole argument. Trimming here rather than adding a
* `tagline` field to the data is what stops a specification and its summary
* drifting apart — there is only ever one copy of the sentence.
*/
function firstSentence(text: string): string {
const trimmed = text.trim();
const match = /^[\s\S]*?[.!?](?=\s|$)/.exec(trimmed);
const first = match?.[0]?.trim();
// A very short first fragment means the split found an abbreviation rather
// than a sentence end, and half a clause on a card is worse than three lines.
return first !== undefined && first.length >= 24 ? first : trimmed;
}
/** Verticals a demo already exists for. Those are shown as the demo, not as a proposal. */
const builtVerticals = new Set<Vertical>(listVerticals().map((group) => group.vertical));
const demoEntries: readonly EnvironmentEntry[] = listDemos().map((meta) => ({
id: `demo:${meta.slug}`,
title: meta.title,
tagline: meta.tagline,
persona: meta.persona,
rewardLine: meta.rewardLine,
icon: meta.icon,
status: meta.status,
/*
* `?tab=play` is written out even though `DEFAULT_TAB` in
* `src/lib/url-state.ts` is already `play` and the param would be stripped
* from a permalink. The promise this card makes is that one click puts you on
* the board, and that promise should not quietly depend on which tab the
* shell happens to default to next quarter.
*/
href: meta.status === 'live' ? `${routes.demo(meta.slug)}?tab=play` : routes.demo(meta.slug),
verticalLabel: VERTICAL_LABELS[meta.vertical],
verticalKey: meta.vertical,
slug: meta.slug,
note: null,
}));
const proposalEntries: readonly EnvironmentEntry[] = lineup
.filter((vertical) => vertical.key === null || !builtVerticals.has(vertical.key))
.map((vertical) => ({
id: `vertical:${vertical.slug}`,
title: vertical.title,
tagline: firstSentence(vertical.task),
persona: vertical.persona,
rewardLine: firstSentence(vertical.reward),
icon: vertical.icon,
status: 'spec' as const,
// Its own page, with the task, the reward, the counterweight and the
// caveat written out. A dimmed card is only honest if it goes somewhere.
href: routes.vertical(vertical.slug),
verticalLabel: vertical.key === null ? null : VERTICAL_LABELS[vertical.key],
verticalKey: vertical.key,
slug: null,
note: vertical.plannedForV1 ? null : 'Not in the first set',
}));
/** Everything, built first. `lineup` is already ranked, so proposals keep that order. */
export const environments: readonly EnvironmentEntry[] = [
...demoEntries.filter((entry) => entry.status === 'live'),
...demoEntries.filter((entry) => entry.status === 'spec'),
...proposalEntries,
];
/** Playable in this tab. */
export const builtEnvironments: readonly EnvironmentEntry[] = environments.filter(
(entry) => entry.status === 'live',
);
/** Published as a specification, with no interactive surface yet. */
export const writtenEnvironments: readonly EnvironmentEntry[] = environments.filter(
(entry) => entry.status === 'spec',
);
/**
* Filter options, in the contract's own order.
*
* An entry whose `verticalKey` is null — a proposal the contract has no key for
* — is reachable only under "All". That is deliberate: such a proposal is on
* the site as the strongest form of the argument, not as something we would
* build, so it belongs to no filterable industry.
*/
export const filterKeys: readonly Vertical[] = VERTICAL_ORDER.filter((key) =>
environments.some((entry) => entry.verticalKey === key),
);
/* ------------------------------------------------------------------ cards */
function StatusBadge({ status }: { status: DemoStatus }) {
if (status === 'live') {
return (
<Badge variant="positive" className="uppercase tracking-wide">
Live
</Badge>
);
}
return (
<Badge variant="outline" className="uppercase tracking-wide">
<FileText aria-hidden="true" className="size-3.5" />
Spec
</Badge>
);
}
/**
* The two lines an executive actually scans for: whose budget this is, and
* what the number pays for. The vertical is not repeated here — both card
* layouts already print it beside the icon.
*/
function MetaRows({ entry }: { entry: EnvironmentEntry }) {
return (
<dl className="mt-4 space-y-1.5 text-sm">
<div className="flex gap-2">
<dt className="w-16 shrink-0 text-muted">For</dt>
<dd className="min-w-0 text-fg">{entry.persona}</dd>
</div>
<div className="flex gap-2">
<dt className="w-16 shrink-0 text-muted">Reward</dt>
<dd className="min-w-0 text-fg">{entry.rewardLine}</dd>
</div>
</dl>
);
}
/**
* One environment, as a card.
*
* The whole rectangle is the link — one target, one focus ring, no nested
* interactive elements to trap a keyboard in.
*/
export function EnvironmentCard({
entry,
className,
}: {
entry: EnvironmentEntry;
className?: string;
}) {
const isSpec = entry.status === 'spec';
return (
<Link
aria-label={`${entry.title}${isSpec ? 'read the specification' : 'play it'}`}
className={cn(
s.cardLink,
'h-full',
// Dimmed, never disabled. It goes to a real specification, which is the
// only thing that makes dimming it honest rather than teasing.
isSpec && 'opacity-75 hover:opacity-100 focus-visible:opacity-100',
className,
)}
to={entry.href}
>
<span className="flex items-start justify-between gap-3">
<span className="flex min-w-0 items-center gap-2">
<span
className={cn(
'grid size-10 shrink-0 place-items-center rounded-lg',
isSpec ? 'bg-surface-2 text-muted' : 'bg-accent-subtle text-accent-fg',
)}
>
<DemoIcon className="size-5" name={entry.icon} />
</span>
{entry.verticalLabel === null ? null : (
<span className="truncate text-xs text-muted">{entry.verticalLabel}</span>
)}
</span>
<StatusBadge status={entry.status} />
</span>
<h3 className="mt-4 text-base font-semibold leading-snug tracking-tight text-fg">
{entry.title}
</h3>
<p className={`${s.prose} mt-1.5 text-sm`}>{entry.tagline}</p>
<MetaRows entry={entry} />
{/* `mt-auto` so the call to action sits on the same line in every card of
a row, however long the tagline above it ran. */}
<span className="mt-auto flex flex-wrap items-center justify-between gap-x-3 gap-y-1 pt-4">
<span className="inline-flex items-center gap-1.5 text-sm font-semibold text-accent-fg">
{isSpec ? 'Read the specification' : 'Play it'}
<ArrowRight
aria-hidden="true"
className="size-4 transition-transform duration-2 ease-enter group-hover:translate-x-0.5"
/>
</span>
{entry.note === null ? null : <span className="text-xs text-muted">{entry.note}</span>}
</span>
</Link>
);
}
/**
* The board, drawn by the demo's own `Surface`.
*
* A screenshot would go stale the first time the board changed and nobody would
* notice. This cannot: it is the same component the demo page renders, in the
* state `interactive.init` hands a new player.
*/
const THUMBNAIL_SEED = 0;
function BoardThumbnail({ slug }: { slug: string }) {
const [board, setBoard] = useState<ReactNode>(null);
useEffect(() => {
let live = true;
/*
* Deliberately after paint, and deliberately not budgeted as a picture.
* This pulls the demo's own lazy chunk — the one the visitor is a click
* away from — so the cost is a prefetch of the destination, and the board
* arriving is proof the destination is already loaded.
*/
void loadDemoModule(slug)
.then((module) => {
if (!live || !module.interactive) return;
const { Surface } = module;
setBoard(<Surface state={module.interactive.init(THUMBNAIL_SEED)} />);
})
.catch(() => {
// A card without a board is still a complete card. A demo whose chunk
// fails to load has its own error page to say so; this is not it.
});
return () => {
live = false;
};
}, [slug]);
return (
// Decorative: the title, the tagline and the reward already say everything
// this board says, and an unplayed board is a shape, not information.
<div aria-hidden="true">
<div className="rounded-lg border border-border bg-surface-2 p-3">
{board ?? <Skeleton className="aspect-[5/6] w-full" />}
</div>
{/* An unplayed board is easy to read as a broken one. One line stops
that, and it is true of any demo's Surface, not just this one. */}
<p className="mt-2 text-xs text-muted">The environments own board, before a move.</p>
</div>
);
}
/**
* The built environment, given the weight it has earned.
*
* Wider than a card, with the real board in it and the only filled call to
* action on the page. If a second demo ever ships, this renders for that one
* too and the picker stays honest without an edit.
*/
export function FeaturedEnvironment({
entry,
className,
}: {
entry: EnvironmentEntry;
className?: string;
}) {
return (
<Link
aria-label={`${entry.title} — play it`}
className={cn(s.cardLink, 'p-5 sm:p-7', className)}
to={entry.href}
>
<div className="grid gap-6 lg:grid-cols-[minmax(0,17rem)_minmax(0,1fr)] lg:gap-10">
{/*
The copy is FIRST in the source and the board is pulled left only from
`lg`. On a phone a full-width board above the title pushes the name of
the environment and its call to action below the fold, which is the
one thing this card exists to avoid.
*/}
<div className="flex flex-col">
<span className="flex flex-wrap items-center gap-2">
<StatusBadge status={entry.status} />
{entry.verticalLabel === null ? null : (
<span className="inline-flex items-center gap-1.5 text-xs text-muted">
<DemoIcon className="size-3.5" name={entry.icon} />
{entry.verticalLabel}
</span>
)}
</span>
<h3 className="mt-3 text-2xl font-bold tracking-tight text-fg sm:text-3xl">
{entry.title}
</h3>
<p className={`${s.lede} mt-2 text-base sm:text-lg`}>{entry.tagline}</p>
<MetaRows entry={entry} />
<p className={`${s.prose} mt-4 text-sm`}>
Four tabs. Play the board, watch a recorded model play the same board, move the reward
weights and see the ranking change, then read the Python that scored it.
</p>
<span className="mt-5 flex flex-wrap items-center gap-x-4 gap-y-2">
{/* Styled as the button it behaves as. It is not a <button>: the
whole card is already the one link, and a control inside a link
is a keyboard trap dressed up as a call to action. */}
<span aria-hidden="true" className={s.btnPrimary}>
<Play className="size-4" />
Play it
</span>
<span className="text-xs text-muted">Opens on the board. No sign-up, no sales call.</span>
</span>
</div>
{entry.slug === null ? null : (
<div className="max-w-[17rem] lg:order-first">
<BoardThumbnail slug={entry.slug} />
</div>
)}
</div>
</Link>
);
}
/* ----------------------------------------------------------------- filter */
/** The "no vertical chosen" value. Not a `Vertical`, so it cannot collide with one. */
export const ALL_VERTICALS = 'all';
export type VerticalFilterValue = Vertical | typeof ALL_VERTICALS;
/**
* Vertical chips. Wraps on a phone rather than scrolling sideways — a chip
* hidden off the right edge is a filter nobody knows exists.
*/
export function VerticalFilter({
active,
onSelect,
className,
}: {
active: VerticalFilterValue;
onSelect: (next: VerticalFilterValue) => void;
className?: string;
}) {
const options: readonly VerticalFilterValue[] = [ALL_VERTICALS, ...filterKeys];
return (
<ul aria-label="Filter environments by vertical" className={cn('flex flex-wrap gap-2', className)}>
{options.map((key) => {
const selected = key === active;
return (
<li key={key}>
<button
aria-pressed={selected}
className={cn(
'tap inline-flex items-center rounded-lg border px-4 py-2 text-sm font-medium transition-colors duration-2 ease-enter',
selected
? 'border-brand bg-accent-subtle text-accent-fg'
: 'border-border bg-surface text-muted hover:bg-surface-2 hover:text-fg',
)}
onClick={() => onSelect(key)}
type="button"
>
{key === ALL_VERTICALS ? 'All' : VERTICAL_LABELS[key]}
</button>
</li>
);
})}
</ul>
);
}
/** Cards in a responsive grid. One list, so a screen reader gets the count. */
export function EnvironmentGrid({
entries,
className,
}: {
entries: readonly EnvironmentEntry[];
className?: string;
}) {
return (
<ul className={cn('grid gap-4 sm:grid-cols-2 lg:grid-cols-3', className)}>
{entries.map((entry) => (
<li className="flex" key={entry.id}>
<EnvironmentCard entry={entry} />
</li>
))}
</ul>
);
}
+74 -8
View File
@@ -15,6 +15,10 @@
* · Every step sets a non-empty `announce`. Reduced motion clamps the * · Every step sets a non-empty `announce`. Reduced motion clamps the
* animation to nothing, so for a screen-reader user the announcement IS * animation to nothing, so for a screen-reader user the announcement IS
* the result, not a courtesy (rule 13). * the result, not a courtesy (rule 13).
* · `meta.ts` never imports this file. The demo page is four tabs and Play
* is the one a demo is allowed not to have, so the eager half — meta,
* narrative, reward — has to render with the interactive half absent
* (rule 14).
*/ */
import { defineDemo, type DemoEpisode, type DemoStep, type RewardValues } from '@/lib/demo-kit'; import { defineDemo, type DemoEpisode, type DemoStep, type RewardValues } from '@/lib/demo-kit';
@@ -128,14 +132,53 @@ export default defineDemo<__Pascal__State>({
'The queue is not the problem. Deciding which three of four hundred items are worth a person is the ' + 'The queue is not the problem. Deciding which three of four hundred items are worth a person is the ' +
'problem, and that decision is exactly the kind of judgement a reward can be written down for.', 'problem, and that decision is exactly the kind of judgement a reward can be written down for.',
anxiety: 'What stops it escalating everything so it never misses one?', anxiety: 'What stops it escalating everything so it never misses one?',
beats: [
{ id: 'hero', title: 'One queue, one decision', claim: 'Every item is either worth a person or it is not.', surface: 'hero' }, /**
{ id: 'anatomy', title: 'What the environment is', claim: 'A task, a fixed action set, a grader, and a score that moves.', surface: 'anatomy' }, * ── THE FOUR CLAIMS ──────────────────────────────────────────────────
{ id: 'play', title: 'Watch a recorded run', claim: 'These are recorded turns, not a scripted animation.', surface: 'split-play' }, *
{ id: 'reward', title: 'Move the weights yourself', claim: 'Pay only for catches and the queue gets escalated whole.', surface: 'reward-editor' }, * The demo page is four tabs — Play, Watch, Reward, Evidence — and this is
{ id: 'receipt', title: 'The code that scored it', claim: 'The number on this page came out of the function below it.', surface: 'receipt' }, * the one sentence each of them has to earn. You do not choose the tabs or
{ id: 'limits', title: 'What this does not show', claim: 'One queue, one grader, and no cost of being wrong.', surface: 'limits' }, * their order; the shell does, and it drops Play when there is no
], * `interactive` and Watch when there are no recorded runs. You only write
* what each one asserts.
*
* Write all four even if this demo will not render all four. A tab whose
* claim you cannot write is a tab with nothing in it, and finding that out
* here is cheaper than finding it out in review.
*
* A good claim is:
*
* · a SENTENCE, not a label. "The reward" is a heading. "Move one weight
* and the ranking of the runs re-orders under it" is a claim.
* · falsifiable BY THE TAB IT SITS ON. The reader should be able to look
* at the surface below it and agree or disagree within a few seconds.
* · about this environment, not about reinforcement learning. The thesis
* above is where the general argument goes.
* · addressed to the person in `meta.persona`, in their words.
*
* The four below are real claims for this worked example. Replace them —
* do not delete the shape.
*/
claims: {
// Play: what the visitor learns by doing the task themselves, and why
// doing it first makes the other three tabs mean something.
play:
'Work the queue yourself for thirty seconds and you will feel the trade the agent is being scored on: ' +
'every item you escalate has to be worth someone opening it.',
// Watch: what the recorded run proves that a claim about the run cannot.
watch:
'This is a real recorded run, replayed one turn at a time — the reasoning, the model call and the cost ' +
'of each decision are exactly what came off the wire.',
// Reward: what changes on screen when the reader changes what "good"
// means. Name the thing that moves.
reward:
'Take the weight off restraint and the run that escalated everything climbs to the top of the ranking. ' +
'That is not a bug in the score; it is the score doing what you asked.',
// Evidence: what the reader can go and check, and what this does not show.
evidence:
'The grader is a short Python function, printed here with the command that ran it, so you can disagree ' +
'with the number by reading the code rather than by trusting us.',
},
limits: [ limits: [
{ {
text: 'The grader knows which items needed a person because the dataset says so. A real queue has no such column, and building one is most of the work.', text: 'The grader knows which items needed a person because the dataset says so. A real queue has no such column, and building one is most of the work.',
@@ -213,6 +256,29 @@ export default defineDemo<__Pascal__State>({
adapt, adapt,
Surface: Board, Surface: Board,
/**
* ── ADDING PLAY ──────────────────────────────────────────────────────────
*
* There is deliberately no `interactive` here, because a scaffolded demo
* starts as `spec`. Add one and the shell grows a Play tab and OPENS ON IT —
* that is the whole shape of the page, so it is worth doing:
*
* interactive: {
* init: (seed: number) => empty__Pascal__(seed), // pure in the seed
* Controls: __Pascal__Controls, // its own module
* },
*
* `init` must be deterministic in the seed: the shell re-inits on reset and
* on a shared link, and a board that comes back different has quietly told
* the visitor the environment is not reproducible.
*
* `Controls` gets `{ state, onChange, seed }` and nothing else. It owns no
* state the board does not — hand the next board to `onChange` and let the
* shell re-render, or Play and the replay will drift apart. Put it in its own
* file and import it here, never from `meta.ts` (rule 14).
*/
verify: recompute, verify: recompute,
}); });
+17 -62
View File
@@ -1,75 +1,30 @@
import type { Narrative } from '@/lib/demo-kit'; import type { Narrative } from '@/lib/demo-kit';
/** /**
* The six beats, in order. The shell renders them; this file decides what the * What this environment argues, tab by tab.
* page argues and in what sequence. *
* The page is a set of tabs rather than an essay because an executive who has
* chosen an environment wants to be doing the task, not reading a case for it.
* Each claim is the one sentence that tab has to earn.
*/ */
export const narrative: Narrative = { export const narrative: Narrative = {
thesis: thesis:
'This is the smallest complete reinforcement-learning environment we could find that needs no ' + 'This is the smallest complete reinforcement-learning environment we could find that needs no ' +
'domain knowledge at all. It has everything the ones that matter to your business have: a task, ' + 'domain knowledge at all. It has everything the ones that matter to your business have: a task, ' +
'a fixed set of legal moves, a grader that cannot be argued with, and a score that moves when ' + 'a fixed set of legal moves, a grader that cannot be argued with, and a score that moves when ' +
'the model gets better. Learn the machine here, and every demo after this is the same machine ' + 'the model gets better. Learn the machine here, and every environment after this is the same ' +
'with a different grader.', 'machine with a different grader.',
anxiety: 'How would we know it was actually working?', anxiety: 'How would we know it was actually working?',
beats: [ claims: {
{ play:
id: 'hero', 'Play it yourself first. Everything else on this page is about what happened when a model tried the same thing.',
title: 'Their hello-world, not ours', watch:
claim: 'This is not a video. It is a recorded attempt replayed at the speed it actually happened, and you can step through it one guess at a time.',
'Prime Intellect ship this exact game as a starter environment in three of their public repositories. We did not pick a game. We picked theirs.', reward:
surface: 'hero', 'Move one slider and the winner changes. That is not a trick — it is the product.',
}, evidence:
{ 'The grader is thirty lines of Python. Here it is, here is the command that runs it, and here is what this environment does not teach you.',
id: 'anatomy', },
title: 'What an environment actually is',
claim:
'Four parts: a task, the moves that are legal, a grader that computes rather than opines, and a number that moves.',
surface: 'anatomy',
},
{
id: 'play',
title: 'You and the model get the same word',
claim:
'Same hidden word, same six guesses, same rules. Play it, then watch what the model did with it.',
surface: 'split-play',
},
{
id: 'watch',
title: 'Watch it think',
claim:
'This is not a video. It is a recorded attempt replayed at the speed it actually happened, and you can step through it one guess at a time.',
surface: 'scrubber',
},
{
id: 'reward',
title: 'You decide what good means',
claim:
'Move one slider and the winner changes. That is not a trick — it is the product.',
surface: 'reward-editor',
},
{
id: 'metric',
title: 'The number that moves',
claim:
'Out of the box, this model solved none of eight. Letting it think first is the cheapest intervention there is, and you can measure exactly what it bought.',
surface: 'metric',
},
{
id: 'receipt',
title: 'The whole environment, in one screen',
claim:
'The grader is thirty lines of Python. Here it is, and here is the command that runs it.',
surface: 'receipt',
},
{
id: 'limits',
title: 'What this does not teach',
claim:
'A word game is missing four things your business has. Each one is why the next demo exists.',
surface: 'limits',
},
],
limits: [ limits: [
{ {
text: text:
+2 -1
View File
@@ -33,10 +33,11 @@ export type {
RewardSpec, RewardSpec,
RewardValues, RewardValues,
RunRef, RunRef,
StoryBeat, DemoTabId,
Vertical, Vertical,
} from './types'; } from './types';
export { DEMO_TABS } from './types';
export { defineDemo, defineMeta } from './define'; export { defineDemo, defineMeta } from './define';
/** `null` is "not scored", never 0.0. Every absence goes through these two. */ /** `null` is "not scored", never 0.0. Every absence goes through these two. */
+28 -20
View File
@@ -170,24 +170,19 @@ export interface DemoEpisode {
}[]; }[];
} }
/** One beat of the exec narrative. The shell renders these in order. */ /**
export interface StoryBeat { * The four tabs every demo page has, in order.
id: string; *
title: string; * `play` is the landing tab and the reason the page exists: a visitor should be
/** One sentence, asserted as a claim the page then demonstrates. */ * doing the task within one click of choosing an environment, not reading about
claim: string; * it. The other three are what they reach for once they have felt it.
/** Which shared surface renders it. */ *
surface: * A demo with no `interactive` mode has no `play` tab and opens on `watch`;
| 'hero' * the shell works that out, not the demo.
| 'anatomy' */
| 'split-play' export type DemoTabId = 'play' | 'watch' | 'reward' | 'evidence';
| 'scrubber'
| 'reward-editor' export const DEMO_TABS: readonly DemoTabId[] = ['play', 'watch', 'reward', 'evidence'];
| 'metric'
| 'receipt'
| 'limits'
| 'custom';
}
/** What this demo deliberately does not teach, and which demo answers it. */ /** What this demo deliberately does not teach, and which demo answers it. */
export interface Limit { export interface Limit {
@@ -201,7 +196,14 @@ export interface Narrative {
thesis: string; thesis: string;
/** The question in the buyer's head when they land. */ /** The question in the buyer's head when they land. */
anxiety: string; anxiety: string;
beats: StoryBeat[]; /**
* One sentence per tab, asserted as a claim the tab then demonstrates.
*
* Required for every tab, including ones this demo may not render — writing
* the claim is how an author works out whether the tab has anything to say.
* A tab whose claim is hard to write is usually a tab with nothing in it.
*/
claims: Record<DemoTabId, string>;
limits: Limit[]; limits: Limit[];
} }
@@ -254,6 +256,12 @@ export interface DemoModule<TState = unknown> {
* means 'unverifiable' — a truncated trace — and must never render as zero. * means 'unverifiable' — a truncated trace — and must never render as zero.
*/ */
verify?: (episode: DemoEpisode) => RewardValues | null; verify?: (episode: DemoEpisode) => RewardValues | null;
/** Extra tabs beside the default ones. */ /**
* Extra panels this demo adds. NOT extra top-level tabs: the page's four
* tabs are fixed and derived, so these ride in the step-detail strip inside
* `watch`, beside Reasoning and Model call, where they sit next to the step
* they are almost always about. A demo with no recording has no step-detail
* strip, and they fall to the bottom of `evidence` instead.
*/
tabs?: { id: string; label: string; Component: React.ComponentType }[]; tabs?: { id: string; label: string; Component: React.ComponentType }[];
} }
+1 -1
View File
@@ -3,7 +3,7 @@
* *
* It owns almost nothing on purpose. The router's loader has already validated * It owns almost nothing on purpose. The router's loader has already validated
* the slug against the registry and started the demo's chunk, and `DemoShell` * the slug against the registry and started the demo's chunk, and `DemoShell`
* owns the loading, the beats and the URL state, so all that is left here is * owns the loading, the tabs and the URL state, so all that is left here is
* the document head — the half of SEO that `scripts/prerender.mjs` cannot do, * the document head — the half of SEO that `scripts/prerender.mjs` cannot do,
* because a visitor who lands on `/` and clicks through never fetches a new * because a visitor who lands on `/` and clicks through never fetches a new
* document and would otherwise keep the home page's title and canonical link. * document and would otherwise keep the home page's title and canonical link.
+99 -135
View File
@@ -1,30 +1,41 @@
import { useMemo } from 'react'; import { useMemo } from 'react';
import { Link, useSearchParams } from 'react-router-dom'; import { Link, useSearchParams } from 'react-router-dom';
import { ArrowRight, FileText } from 'lucide-react';
import { iconFor } from '@/content/icons';
import { import {
allDemos, ALL_VERTICALS,
demosForVertical, EnvironmentGrid,
lineup, FeaturedEnvironment,
routes, environments,
verticalForDemo, filterKeys,
verticalKeysInUse, VerticalFilter,
} from '@/content/lineup'; type VerticalFilterValue,
} from '@/components/site/EnvironmentPicker';
import { routes } from '@/content/lineup';
import { PROPOSAL_NOTICE } from '@/content/verticals'; import { PROPOSAL_NOTICE } from '@/content/verticals';
// The registry owns the taxonomy's exec-facing names. The lineup's own titles
// are longer marketing headings ("Customer Support Resolution") and would wrap
// two lines inside a filter chip on a phone, so chips use the registry label.
import { VERTICAL_LABELS } from '@/lib/demo-kit/registry'; import { VERTICAL_LABELS } from '@/lib/demo-kit/registry';
import type { Vertical } from '@/lib/demo-kit/types'; import type { Vertical } from '@/lib/demo-kit/types';
import * as s from '@/content/styles'; import * as s from '@/content/styles';
import { pageTitle, useSeo } from '@/lib/seo'; import { pageTitle, useSeo } from '@/lib/seo';
const ALL = 'all'; /**
* The fuller list.
*
* Home leads with the picker; this is where you go to see everything and cut it
* by industry. The cards are the same component, so the two pages cannot drift
* into describing the same environment two different ways — which is how an
* honest site quietly becomes a dishonest one.
*/
function verticalLabel(key: Vertical): string { /** What a `spec` has to contain before it is allowed on this page. Contract rule 12. */
return VERTICAL_LABELS[key]; const SPEC_PARTS: readonly { label: string; body: string }[] = [
} { label: 'A task', body: 'One unit of work with a beginning and an end.' },
{ label: 'Legal moves', body: 'What the agent is allowed to do, and what is refused.' },
{ label: 'A grader', body: 'Deterministic code that marks the attempt. No judge, no rubric.' },
{
label: 'A counterweight',
body: 'The term that stops the objective being maximised the crude way.',
},
];
export default function Gallery() { export default function Gallery() {
useSeo({ useSeo({
@@ -42,58 +53,41 @@ export default function Gallery() {
*/ */
const [params, setParams] = useSearchParams(); const [params, setParams] = useSearchParams();
const raw = params.get('vertical'); const raw = params.get('vertical');
const active: Vertical | typeof ALL = const active: VerticalFilterValue =
raw && verticalKeysInUse.includes(raw as Vertical) ? (raw as Vertical) : ALL; raw && filterKeys.includes(raw as Vertical) ? (raw as Vertical) : ALL_VERTICALS;
const shown = useMemo( const shown = useMemo(
() => (active === ALL ? allDemos : allDemos.filter((d) => d.vertical === active)), () =>
active === ALL_VERTICALS
? environments
: environments.filter((entry) => entry.verticalKey === active),
[active], [active],
); );
function select(next: Vertical | typeof ALL) { const built = shown.filter((entry) => entry.status === 'live');
const written = shown.filter((entry) => entry.status === 'spec');
function select(next: VerticalFilterValue) {
// `replace` so a run of filter taps leaves one entry in history, not eight. // `replace` so a run of filter taps leaves one entry in history, not eight.
if (next === ALL) setParams({}, { replace: true }); if (next === ALL_VERTICALS) setParams({}, { replace: true });
else setParams({ vertical: next }, { replace: true }); else setParams({ vertical: next }, { replace: true });
} }
const filters: readonly (Vertical | typeof ALL)[] = [ALL, ...verticalKeysInUse];
const unbuilt = lineup.filter((v) => demosForVertical(v.key).length === 0).length;
return ( return (
<main className={`${s.shell} py-10 sm:py-16`}> <main className={`${s.shell} py-10 sm:py-16`}>
<p className={s.eyebrow}>Gallery</p> <p className={s.eyebrow}>Gallery</p>
<h1 className={`${s.h1} mt-3 max-w-3xl`}>Every environment we have built or specified.</h1> <h1 className={`${s.h1} mt-3 max-w-3xl`}>Every environment we have built or specified.</h1>
<p className={`${s.lede} mt-5 max-w-2xl`}> <p className={`${s.lede} mt-5 max-w-2xl`}>
A live demo is playable in this tab. A spec is a written environment task, action set, A built environment is playable in this tab, on the same code the repository ships. A spec is
grader, counterweight and the command that evaluates it published in full, with no a written environment task, action set, grader, counterweight and the command that
interactive surface yet. There are no coming-soon cards here. evaluates it published in full, with no interactive surface yet. There are no coming-soon
cards here.
</p> </p>
{filters.length > 2 ? ( {filterKeys.length > 1 ? (
<div className="mt-8"> <div className="mt-8">
<h2 className="sr-only">Filter by vertical</h2> <h2 className="sr-only">Filter by vertical</h2>
<ul aria-label="Filter demos by vertical" className="flex flex-wrap gap-2"> <VerticalFilter active={active} onSelect={select} />
{filters.map((key) => {
const selected = key === active;
return (
<li key={key}>
<button
aria-pressed={selected}
className={`tap inline-flex items-center rounded-lg border px-4 py-2 text-sm font-medium transition-colors duration-2 ease-enter ${
selected
? 'border-brand bg-accent-subtle text-accent-fg'
: 'border-border bg-surface text-muted hover:bg-surface-2 hover:text-fg'
}`}
onClick={() => select(key)}
type="button"
>
{key === ALL ? 'All' : verticalLabel(key)}
</button>
</li>
);
})}
</ul>
</div> </div>
) : null} ) : null}
@@ -103,7 +97,8 @@ export default function Gallery() {
*/} */}
<p aria-live="polite" className="mt-6 text-sm text-muted"> <p aria-live="polite" className="mt-6 text-sm text-muted">
{shown.length === 1 ? '1 environment' : `${shown.length} environments`} {shown.length === 1 ? '1 environment' : `${shown.length} environments`}
{active === ALL ? '' : ` in ${verticalLabel(active)}`} {active === ALL_VERTICALS ? '' : ` in ${VERTICAL_LABELS[active]}`}
{shown.length === 0 ? '' : ` · ${built.length} built · ${written.length} written`}
</p> </p>
{shown.length === 0 ? ( {shown.length === 0 ? (
@@ -111,102 +106,71 @@ export default function Gallery() {
{/* Two different nothings. Telling a visitor "no environment is filed {/* Two different nothings. Telling a visitor "no environment is filed
under this vertical" when they have not filtered anything reads as under this vertical" when they have not filtered anything reads as
a broken page rather than an empty one. */} a broken page rather than an empty one. */}
<p className={s.h3}> <h2 className={s.h3}>
{active === ALL ? 'No environments are registered.' : 'Nothing under this vertical.'} {active === ALL_VERTICALS
</p> ? 'No environments are registered.'
: 'Nothing under this vertical.'}
</h2>
<p className={`${s.prose} mt-2`}> <p className={`${s.prose} mt-2`}>
{active === ALL {active === ALL_VERTICALS
? 'The registry is empty, which means the site is mid-build rather than hiding something. The lineup below is written either way.' ? 'The registry is empty and the lineup is empty with it, which means the site is mid-build rather than hiding something.'
: 'No environment is filed here yet. The proposal for it is still on its own page, written out in full.'} : 'No environment is filed here yet, built or written.'}
</p> </p>
{active === ALL ? null : ( {active === ALL_VERTICALS ? null : (
<button className={`${s.btnSecondary} mt-4`} onClick={() => select(ALL)} type="button"> <button
className={`${s.btnSecondary} mt-4`}
onClick={() => select(ALL_VERTICALS)}
type="button"
>
Show every environment Show every environment
</button> </button>
)} )}
</div> </div>
) : ( ) : null}
<ul className="mt-4 grid gap-4 sm:grid-cols-2 lg:grid-cols-3">
{shown.map((demo) => {
const Icon = iconFor(demo.icon);
const isSpec = demo.status === 'spec';
const vertical = verticalForDemo(demo);
return (
<li className="flex" key={demo.slug}>
<article className="flex w-full flex-col">
<Link
// A spec is dimmed but never disabled: it goes to a real
// page with a real specification on it, which is the only
// thing that makes dimming it honest rather than teasing.
aria-label={`${demo.title}${isSpec ? 'read the specification' : 'play it'}`}
className={`${s.cardLink} h-full ${isSpec ? 'opacity-70 hover:opacity-100' : ''}`}
to={routes.demo(demo.slug)}
>
<span className="flex items-start justify-between gap-3">
<Icon aria-hidden="true" className="size-5 shrink-0 text-brand" />
{isSpec ? (
<span className={`${s.pill} gap-1`}>
<FileText aria-hidden="true" className="size-3.5" />
Spec
</span>
) : (
<span className={`${s.pill} border-positive/30 bg-positive/10 text-positive`}>
Live
</span>
)}
</span>
<h3 className="mt-3 text-lg font-bold tracking-tight text-fg">{demo.title}</h3> {built.length > 0 ? (
<p className={`${s.prose} mt-1.5 text-sm`}>{demo.tagline}</p> <section aria-labelledby="built" className="mt-4">
<h2 className={s.h2} id="built">
{built.length === 1 ? 'The one you can play' : 'The ones you can play'}
</h2>
<div className="mt-4 space-y-4">
{built.map((entry) => (
<FeaturedEnvironment entry={entry} key={entry.id} />
))}
</div>
</section>
) : null}
<dl className="mt-4 space-y-1.5 text-sm"> {written.length > 0 ? (
<div className="flex gap-2"> <section aria-labelledby="written" className="mt-10">
<dt className="shrink-0 text-muted">For</dt> <div className="flex flex-wrap items-end justify-between gap-3">
<dd className="text-fg">{demo.persona}</dd> <h2 className={s.h2} id="written">
</div> {written.length === 1 ? 'The one that is written' : `The ${written.length} that are written`}
<div className="flex gap-2"> </h2>
<dt className="shrink-0 text-muted">Reward</dt> <span className={s.proposalPill}>{PROPOSAL_NOTICE}</span>
<dd className="text-fg">{demo.rewardLine}</dd> </div>
</div> <EnvironmentGrid className="mt-4" entries={written} />
</dl> </section>
) : null}
<span className="mt-4 inline-flex items-center gap-1.5 text-sm font-semibold text-accent-fg">
{isSpec ? 'Read the specification' : 'Play it'}
<ArrowRight
aria-hidden="true"
className="size-4 transition-transform duration-2 ease-enter group-hover:translate-x-0.5"
/>
</span>
</Link>
{vertical ? (
<p className="mt-2 px-1 text-xs text-muted">
<Link className={s.link} to={routes.vertical(vertical.slug)}>
{vertical.title}
</Link>{' '}
· {PROPOSAL_NOTICE}
</p>
) : null}
</article>
</li>
);
})}
</ul>
)}
<div className="card mt-12 p-5 sm:p-7"> <div className="card mt-12 p-5 sm:p-7">
{/* Counted, not typed. A hard-coded "eleven" on a site about checkable <h2 className={s.h2}>What a spec has to contain</h2>
numbers goes stale the first time a demo ships. */}
<h2 className={s.h2}>
The {unbuilt} we have not built
</h2>
<p className={`${s.prose} mt-3 max-w-2xl`}> <p className={`${s.prose} mt-3 max-w-2xl`}>
The lineup is a set of proposals, written to the same four-part shape as the live one. A card that says a demo is coming is not a specification, and the build refuses one. Every
Reading one takes a minute and tells you whether the idea survives contact with your own written environment above states all four of these, and the reward weights that go with
numbers. them, before it is allowed on this page.
</p> </p>
<Link className={`${s.btnPrimary} mt-5`} to={routes.home}> <ol className="mt-5 grid gap-3 sm:grid-cols-2 lg:grid-cols-4">
See the lineup {SPEC_PARTS.map((part, i) => (
<li className="card bg-surface-2 p-4" key={part.label}>
<span className="nums text-xs font-semibold text-brand">0{i + 1}</span>
<h3 className={`${s.h3} mt-1.5`}>{part.label}</h3>
<p className={`${s.prose} mt-1.5 text-sm`}>{part.body}</p>
</li>
))}
</ol>
<Link className={`${s.btnSecondary} mt-6`} to={routes.honesty}>
What we measured, and what we didnt
</Link> </Link>
</div> </div>
</main> </main>
+161 -250
View File
@@ -1,38 +1,31 @@
import { Link } from 'react-router-dom'; import { Link } from 'react-router-dom';
import { ArrowRight, ArrowUpRight, Play } from 'lucide-react'; import { ArrowRight, ArrowUpRight } from 'lucide-react';
import { helloWorldCitations, reproduce, trainingResult } from '@/content/evidence'; import {
import { iconFor } from '@/content/icons'; EnvironmentGrid,
import { featuredDemo, lineup, routes } from '@/content/lineup'; FeaturedEnvironment,
builtEnvironments,
writtenEnvironments,
} from '@/components/site/EnvironmentPicker';
import { helloWorldCitations, trainingResult } from '@/content/evidence';
import { routes } from '@/content/lineup';
import { PROPOSAL_NOTICE } from '@/content/verticals'; import { PROPOSAL_NOTICE } from '@/content/verticals';
import * as s from '@/content/styles'; import * as s from '@/content/styles';
import { pageTitle, useSeo } from '@/lib/seo'; import { pageTitle, useSeo } from '@/lib/seo';
/** /**
* The four boxes. This is the definition the whole site rests on, so it is * The landing page: a line of thesis, then the picker.
* written once, here, in the order a person who has never heard the words *
* "reinforcement learning" can read it: what the job is, what you are allowed * The page used to open with the argument and bury the demos below it. That is
* to do, who marks it, what the mark is. * an essay, and an executive who has read the first sentence has already
* decided whether to click something. So the picker is the primary object here
* and everything that used to lead — the credential, the measured number — sits
* underneath it as support for a visitor who wants it before they click.
*
* Counts in the copy are derived from the registry and the lineup, never typed.
* A hard-coded "eleven" on a site whose whole argument is that its claims are
* checkable goes stale the first time a demo ships.
*/ */
const ANATOMY: readonly { label: string; body: string }[] = [
{
label: 'A task',
body: 'One unit of work with a beginning and an end. Guess a five-letter word in six tries.',
},
{
label: 'Legal moves',
body: 'What the player is allowed to do. Any word on the list, once, five letters.',
},
{
label: 'A grader',
body: 'Code that marks the attempt. It runs the same way every time and there is nobody to appeal to.',
},
{
label: 'A score that moves',
body: 'One number per attempt. Train against it and it goes up, or it does not and you found that out cheaply.',
},
];
export default function Home() { export default function Home() {
// The head is set here as well as baked by `scripts/prerender.mjs`, and the // The head is set here as well as baked by `scripts/prerender.mjs`, and the
// two are not redundant: prerender covers the crawler that fetches the // two are not redundant: prerender covers the crawler that fetches the
@@ -42,265 +35,183 @@ export default function Home() {
useSeo({ useSeo({
title: pageTitle(), title: pageTitle(),
description: description:
'Interactive demos of reinforcement-learning environments for executives. Real verifiers environments, real recorded rollouts, and a reward you can change to see the ranking flip.', 'Interactive demos of reinforcement-learning environments for executives. Play the environment, watch a recorded model play the same board, then move the reward and watch the ranking change.',
canonical: routes.home, canonical: routes.home,
ogImage: '/og/home.png', ogImage: '/og/home.png',
}); });
const Featured = featuredDemo; const [featured, ...otherBuilt] = builtEnvironments;
const builtCount = builtEnvironments.length;
const writtenCount = writtenEnvironments.length;
// Written out rather than interpolated inline so the zero and the singular
// both read as English. Both numbers come from the data — `VERTICALS` has
// twelve entries and none of them is keyed `reference`, so the one live demo
// suppresses no proposal and the lineup reads one and twelve today. It will
// not stay that way, and the sentence has to survive the day it changes.
const builtSentence =
builtCount === 1
? 'One environment is built and playable in this tab.'
: `${builtCount} environments are built and playable in this tab.`;
const writtenSentence =
writtenCount === 1
? 'One more is a written specification:'
: `The other ${writtenCount} are written specifications:`;
return ( return (
<main> <main>
{/* ── The thesis ─────────────────────────────────────────────────── */} {/* ── The thesis, in two sentences ───────────────────────────────── */}
<section className={`${s.shell} pt-10 sm:pt-16`}> <section className={`${s.shell} pt-10 sm:pt-16`}>
<p className={s.eyebrow}>Environments, demonstrated</p> <p className={s.eyebrow}>Environments, demonstrated</p>
<h1 className={`${s.h1} mt-3 max-w-4xl`}> <h1 className={`${s.h1} mt-3 max-w-4xl`}>
An environment is an eval you can take the gradient of. An environment is an eval you can take the gradient of.
</h1> </h1>
<p className={`${s.lede} mt-5 max-w-2xl`}> <p className={`${s.lede} mt-5 max-w-2xl`}>
You write down what good means, in code. A model attempts the work. The grader scores it You write down what good means, in code, and then you train against that number and watch
and cannot be argued with. Then you train against that score and watch the number move it move or watch it sit still, which you found out in an afternoon instead of a quarter.
or watch it not move, which you found out in an afternoon instead of a quarter. </p>
</section>
{/* ── The picker: the built one ──────────────────────────────────── */}
<section aria-labelledby="pick" className={`${s.shell} mt-10 sm:mt-14`}>
<h2 className={s.h2} id="pick">
Pick one. You are playing it in a click.
</h2>
<p className={`${s.prose} mt-3 max-w-3xl`}>
{builtSentence}{' '}
{writtenCount === 0 ? null : (
<>
{writtenSentence} the task, the grader, the counterweight that stops the grader being
farmed, and the command that evaluates it. Nothing here is a coming-soon card.
</>
)}
</p> </p>
<div className="mt-7 flex flex-col gap-3 sm:flex-row sm:items-center"> {featured ? <FeaturedEnvironment className="mt-6" entry={featured} /> : null}
{Featured ? ( {otherBuilt.length > 0 ? <EnvironmentGrid className="mt-4" entries={otherBuilt} /> : null}
<Link className={s.btnPrimary} to={routes.demo(Featured.slug)}> </section>
<Play aria-hidden="true" className="size-4" />
Play the environment {/* ── The picker: the written ones ───────────────────────────────── */}
</Link> {/* Dropped entirely rather than rendered as "The 0 we have written": the
) : null} day every proposal becomes a demo, this section should disappear. */}
<Link className={s.btnSecondary} to={routes.honesty}> {writtenCount === 0 ? null : (
What we measured, and what we didnt <section aria-labelledby="written" className={`${s.shell} mt-10 sm:mt-14`}>
</Link> <div className="flex flex-wrap items-end justify-between gap-3">
<h2 className={s.h2} id="written">
The {writtenCount} we have written but not built.
</h2>
<span className={s.proposalPill}>{PROPOSAL_NOTICE}</span>
</div> </div>
</section> <p className={`${s.prose} mt-3 max-w-3xl`}>
Each one is a task an environment could run, a reward stated in a number your board already
reads, and the counterweight that stops that reward being maximised the crude way. Every
card opens the whole argument, including where it stops being honest.
</p>
{/* ── The credential, before anything else we say ────────────────── */} <EnvironmentGrid className="mt-6" entries={writtenEnvironments} />
<section className={`${s.shell} ${s.section}`}>
<div className="card p-5 sm:p-7">
<p className={s.eyebrow}>Why a word game</p>
<h2 className={`${s.h2} mt-2`}>We didnt pick a game. We picked theirs.</h2>
<p className={`${s.prose} mt-3 max-w-3xl`}>
Wordle is Prime Intellects own hello-world. It is one of five basic end-to-end examples
in their trainer, a shipped environment in their library, and the environment their
official tutorial optimises prompts against. A demo of their idea should start where
they start.
</p>
<ul className="mt-5 grid gap-3 sm:grid-cols-3">
{helloWorldCitations.map((c) => (
<li key={c.href}>
<a
aria-label={c.label}
className={`${s.cardLink} h-full bg-surface-2`}
href={c.href}
rel="noreferrer noopener"
target="_blank"
>
<span className={`${s.h3} inline-flex items-start gap-1.5`}>
<span className="font-mono text-[0.8125rem] leading-6">{c.label}</span>
<ArrowUpRight aria-hidden="true" className="mt-1 size-4 shrink-0 text-muted" />
</span>
<span className={`${s.prose} mt-2 text-sm`}>{c.claim}</span>
<span className="sr-only">(opens in a new tab)</span>
</a>
</li>
))}
</ul>
</div>
</section>
{/* ── What an environment is, in four boxes ──────────────────────── */} <Link className={`${s.btnSecondary} mt-6`} to={routes.gallery}>
<section className={`${s.shell} pb-12 sm:pb-16`}> Filter the lineup by industry
<h2 className={s.h2}>Four parts. That is the whole of it.</h2> <ArrowRight aria-hidden="true" className="size-4" />
<ol className="mt-6 grid gap-4 sm:grid-cols-2 lg:grid-cols-4"> </Link>
{ANATOMY.map((box, i) => (
<li className="card flex flex-col p-5" key={box.label}>
<span className="nums text-xs font-semibold text-brand">0{i + 1}</span>
<h3 className={`${s.h3} mt-2`}>{box.label}</h3>
<p className={`${s.prose} mt-2 text-sm`}>{box.body}</p>
</li>
))}
</ol>
</section> </section>
)}
{/* ── The one measured number ────────────────────────────────────── */} {/* ── The support: why this game, and the one measured number ────── */}
<section className={`${s.shell} pb-12 sm:pb-16`}> <section aria-labelledby="evidence" className={`${s.shell} ${s.section}`}>
<div className="card overflow-hidden"> <h2 className="sr-only" id="evidence">
<div className="grid gap-6 p-5 sm:p-7 lg:grid-cols-[minmax(0,22rem)_minmax(0,1fr)] lg:gap-10"> Why this environment, and what has been measured on it
<div> </h2>
<p className={s.eyebrow}>Published by Prime Intellect</p> <div className="grid gap-4 lg:grid-cols-2">
<p className="nums mt-3 flex flex-wrap items-baseline gap-x-3 gap-y-1"> <div className="card p-5 sm:p-7">
<span className="text-4xl font-extrabold tracking-tight text-muted sm:text-5xl"> <p className={s.eyebrow}>Why a word game</p>
{trainingResult.before} <h3 className="mt-2 text-xl font-bold tracking-tight text-fg sm:text-2xl">
</span> We didnt pick a game. We picked theirs.
<ArrowRight aria-hidden="true" className="size-6 shrink-0 text-muted" /> </h3>
<span className="text-4xl font-extrabold tracking-tight text-positive sm:text-5xl"> <p className={`${s.prose} mt-3`}>
{trainingResult.after} Wordle is Prime Intellects own hello-world: a basic example in their trainer, a
</span> shipped environment in their library, and the environment their official tutorial
</p> optimises prompts against.
<p className={`${s.prose} mt-2 text-sm`}> </p>
{trainingResult.model} {trainingResult.metric} on this task, before and after <ul className="mt-4 space-y-2">
training. {helloWorldCitations.map((c) => (
</p> <li key={c.href}>
</div>
<div className="flex flex-col justify-center">
<p className={s.prose}>
Out of the box, a 1.7-billion-parameter model never once guesses the word. After an{' '}
{trainingResult.method}, it wins about six games in ten. Measured on{' '}
{trainingResult.evalDescription}. Both checkpoints are public, so the claim is
checkable rather than quotable.
</p>
<p className="mt-4 flex flex-wrap gap-2">
<a
className={s.pill}
href={trainingResult.source.href}
rel="noreferrer noopener"
target="_blank"
>
The write-up
<ArrowUpRight aria-hidden="true" className="size-3.5" />
</a>
{trainingResult.checkpoints.map((c) => (
<a <a
className={s.pill} className="tap flex items-start gap-2 rounded-md py-1 text-sm text-muted transition-colors duration-1 ease-enter hover:text-fg"
href={c.href} href={c.href}
key={c.href}
rel="noreferrer noopener" rel="noreferrer noopener"
target="_blank" target="_blank"
> >
<span className="font-mono">{c.label.replace('PrimeIntellect/', '')}</span> <ArrowUpRight aria-hidden="true" className="mt-0.5 size-4 shrink-0 text-brand" />
<ArrowUpRight aria-hidden="true" className="size-3.5" /> <span>
<span className="font-mono text-[0.8125rem] text-fg">{c.label}</span>
<span className="block">{c.claim}</span>
<span className="sr-only">(opens in a new tab)</span>
</span>
</a> </a>
))} </li>
</p> ))}
{/* </ul>
The same write-up publishes average-reward figures for these
runs. They are deliberately not on this page — see /honesty.
*/}
<p className="mt-3 text-xs text-muted">
We quote the win rate only. The reward numbers in that write-up span versions of the
environment and were never re-measured together.{' '}
<Link className={s.link} to={routes.honesty}>
Why that matters
</Link>
.
</p>
</div>
</div> </div>
</div>
</section>
{/* ── The live demo ──────────────────────────────────────────────── */} <div className="card p-5 sm:p-7">
{Featured ? ( <p className={s.eyebrow}>Published by Prime Intellect</p>
<section className={`${s.shell} pb-12 sm:pb-16`}> <p className="nums mt-3 flex flex-wrap items-baseline gap-x-3 gap-y-1">
<p className={s.eyebrow}>The live one</p> <span className="text-4xl font-extrabold tracking-tight text-muted sm:text-5xl">
<h2 className={`${s.h2} mt-2`}>Play it, then change what counts as good.</h2> {trainingResult.before}
<p className={`${s.prose} mt-3 max-w-2xl`}>
The demo runs the same environment the repository ships. You can play a board yourself,
watch a recorded model play the same board, read the Python that scored it, and then
move the reward weights and watch the ranking of two recorded runs change under you.
</p>
<Link
aria-label={`Open the ${Featured.title} demo`}
className={`${s.cardLink} mt-6 sm:p-7`}
to={routes.demo(Featured.slug)}
>
<span className="flex flex-wrap items-center gap-2">
<span className={`${s.pill} border-positive/30 bg-positive/10 text-positive`}>
Live
</span> </span>
<span className="text-xs text-muted">For the {Featured.persona}</span> <ArrowRight aria-hidden="true" className="size-6 shrink-0 text-muted" />
</span> <span className="text-4xl font-extrabold tracking-tight text-positive sm:text-5xl">
<span className="mt-3 text-xl font-bold tracking-tight text-fg sm:text-2xl"> {trainingResult.after}
{Featured.title}
</span>
<span className={`${s.prose} mt-2`}>{Featured.tagline}</span>
<span className="mt-4 flex flex-wrap items-center justify-between gap-3">
<span className="text-sm text-muted">
Reward: <span className="text-fg">{Featured.rewardLine}</span>
</span> </span>
<span className="inline-flex items-center gap-1.5 text-sm font-semibold text-accent-fg"> </p>
Open the demo <p className={`${s.prose} mt-3`}>
<ArrowRight {trainingResult.model} {trainingResult.metric} on this task, before and after training.
aria-hidden="true" Out of the box it never once guesses the word; after an {trainingResult.method}, it
className="size-4 transition-transform duration-2 ease-enter group-hover:translate-x-0.5" wins about six games in ten. Measured on {trainingResult.evalDescription}, and both
/> checkpoints are public, so the claim is checkable rather than quotable.
</span> </p>
</span> <p className="mt-4 flex flex-wrap gap-2">
</Link> <a
<div className="mt-4 grid gap-3 sm:grid-cols-2"> className={s.pill}
<div> href={trainingResult.source.href}
<p className="text-xs font-semibold uppercase tracking-wider text-muted"> rel="noreferrer noopener"
Or skip the browser target="_blank"
</p> >
<pre className={`${s.codeBlock} mt-2`}> The write-up
<code> <ArrowUpRight aria-hidden="true" className="size-3.5" />
{reproduce.clone} </a>
{'\n'} {trainingResult.checkpoints.map((c) => (
{reproduce.install} <a
{'\n'} className={s.pill}
{reproduce.evaluate} href={c.href}
</code> key={c.href}
</pre> rel="noreferrer noopener"
</div> target="_blank"
<p className={`${s.prose} self-end text-sm`}> >
Three commands and you have the environment on your own machine, scoring your own <span className="font-mono">{c.label.replace('PrimeIntellect/', '')}</span>
model. Nothing on this page needs our servers to be up. <ArrowUpRight aria-hidden="true" className="size-3.5" />
</a>
))}
</p>
{/*
The same write-up publishes average-reward figures for these runs.
They are deliberately not on this page — see /honesty.
*/}
<p className="mt-3 text-xs text-muted">
We quote the win rate only. The reward numbers in that write-up span versions of the
environment and were never re-measured together.
</p> </p>
</div> </div>
</section>
) : null}
{/* ── The lineup ─────────────────────────────────────────────────── */}
<section className={`${s.shell} pb-16 sm:pb-24`}>
<div className="flex flex-wrap items-end justify-between gap-3">
<div>
<p className={s.eyebrow}>The lineup</p>
<h2 className={`${s.h2} mt-2`}>Twelve of these, ranked.</h2>
</div>
<span className={s.proposalPill}>{PROPOSAL_NOTICE}</span>
</div> </div>
<p className={`${s.prose} mt-3 max-w-2xl`}>
Each one is a task an environment could run, a reward in a number your board already
reads, and the counterweight that stops that reward being farmed the crude way. They are
our proposals. Nobodys roadmap, nobodys customer list.
</p>
<ul className="mt-6 grid gap-3 sm:grid-cols-2 lg:grid-cols-3"> <div className="mt-6 flex flex-col gap-3 sm:flex-row sm:items-center">
{lineup.map((v) => {
const Icon = iconFor(v.icon);
return (
<li key={v.slug}>
<Link
aria-label={v.title}
className={`${s.cardLink} h-full`}
to={routes.vertical(v.slug)}
>
<span className="flex items-start justify-between gap-3">
<Icon aria-hidden="true" className="size-5 shrink-0 text-brand" />
<span className="nums text-xs font-semibold text-muted">
{String(v.rank).padStart(2, '0')}
</span>
</span>
<span className={`${s.h3} mt-3`}>{v.title}</span>
<span className={`${s.prose} mt-1.5 text-sm`}>{v.reward}</span>
{!v.plannedForV1 ? (
<span className="mt-3 text-xs text-muted">Not in the first set</span>
) : null}
</Link>
</li>
);
})}
</ul>
<div className="mt-8 flex flex-col gap-3 sm:flex-row">
<Link className={s.btnSecondary} to={routes.gallery}>
See what is built
</Link>
<Link className={s.btnSecondary} to={routes.honesty}> <Link className={s.btnSecondary} to={routes.honesty}>
Read the honesty page first What we measured, and what we didnt
</Link> </Link>
<p className="text-sm text-muted">
Every number on this site, with the one that is ours and the ones that are not.
</p>
</div> </div>
</section> </section>
</main> </main>