Document the contract and the traps

CONTRACT.md states the twelve rules and, for each, the failure it prevents. AGENTS.md lists the three directories a demo agent may touch and the seven traps that have already cost time here — pnpm 11's settings move, the non-portable RNG, the two-pass rule, the two reward-denominator subtleties, the silent CSP worker block, and Caddy's silent bind. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
2026-08-28 15:48:20 -07:00
parent 408ce4a525
commit 2c2dcad9fd
4 changed files with 491 additions and 7 deletions
@@ -0,0 +1,64 @@
+# Working in this repository
+
+## The three commands
+
+```bash
+uv sync --all-packages && uv run pytest envs/wordle_five/tests && uv run python envs/probe.py
+pnpm install && pnpm check && pnpm build
+uv run python envs/verify_fixtures.py
+```
+
+## Directories an agent adding a demo may touch
+
+Exactly three, plus one line of content:
+
+- `src/demos/<slug>/`
+- `envs/<pkg>/`
+- `public/traces/<slug>/`
+- one entry in `src/content/verticals.ts`
+
+Anything else is a shared file and a merge conflict waiting to happen. If you
+believe you need to edit the shell, the header or the router, you have found a
+contract bug — read `CONTRACT.md`, rule 6.
+
+## Traps that have already cost time here
+
+**pnpm 11 does not read the `pnpm` field in `package.json`.** Settings live in
+`pnpm-workspace.yaml`, and an unapproved build script makes `pnpm install`
+*exit 1* rather than warn.
+
+**A language's built-in RNG is not portable.** `random.Random(seed)` and any
+JavaScript PRNG will disagree, so the same seed picks different words on the two
+sides and every permalink silently shows a different puzzle than the run it
+claims to replay. Both sides use FNV-1a over the decimal seed. `Math.imul` on
+the JS side is load-bearing — a plain multiply overflows into a double.
+
+**Scoring must be two passes.** Every green resolves before any yellow. One
+pass gets `SASSY`/`BASIS` wrong, and 21.2 million pairs are hashed in CI
+precisely so that cannot ship.
+
+**`consistency` is scored over turns SPENT, not guesses accepted.** Counting
+only legal guesses hands a free 1.0 to a policy that plays one word and jams the
+parser five times: one guess, no contradictions, perfect score.
+
+**`economy`'s denominator is the shipped solver's depth, not a depth-optimal
+search.** Entropy-greedy is not depth-optimal, so grading it against a true
+optimum makes the oracle rung fail its own probe assertion on some seeds.
+
+**The production CSP has no `worker-src`,** so it falls back to
+`default-src 'self'`. A blob-backed worker (Vite's `?worker&inline`) is blocked
+in production only, with no console error — the solver simply never boots. CI
+greps the bundle for `blob:` for this reason.
+
+**Caddy's `bind 10.0.0.2` on cloud-2 is mandatory and its absence is silent.**
+Without it the site answers an empty 200 behind a valid certificate, and a
+`--resolve` check from cloud-2 itself still passes. Verify from a third machine
+against the real hostname.
+
+**spark-1 serves one model and is single-stream.** Capture is a serialised
+queue, not a parallel one. Book it before starting a long sweep.
+
+## Style
+
+Comments explain *why*, and only where the reason is not evident. Never narrate
+what the code says. Prefer one honest number over three adjectives.
@@ -0,0 +1,104 @@
+# The demo contract
+
+Every demo is one directory under `src/demos/<slug>/` plus one Python package
+under `envs/`. The shell renders anything that satisfies the interfaces in
+`src/lib/demo-kit/types.ts`. Nothing else in the repository needs editing —
+the header, the gallery, the router and the sitemap are all generated from the
+registry, and the registry finds demos by existence.
+
+That is the whole design goal: **demo number nine must not be able to break
+demo number one.**
+
+## The rules, and why each exists
+
+1. **The directory name is the slug.** Two sources of truth for a URL is one
+   too many.
+
+2. **`meta.ts` is eager, serialisable and React-free.** It is imported for
+   every demo on every page load, because the header needs all of them to
+   render. `icon` is a lucide icon *name*, not a component — importing the
+   component would pull lucide into the entry chunk on behalf of a demo nobody
+   opened.
+
+3. **`demo.tsx` is lazy.** Everything expensive lives behind it.
+
+4. **A demo imports from `@/lib/demo-kit` and nowhere deeper.** Reaching into
+   `@/lib/demo-kit/player` couples a demo to an implementation detail; the
+   barrel is the contract's surface.
+
+5. **A demo never imports from `@/components/demo/`.** The shell renders demos;
+   demos do not reach into the shell.
+
+6. **The shell never mentions a slug.** `if (slug === 'wordle')` in
+   `src/components/demo/` is a contract bug — either fix the contract or expose
+   a `SlotRegion`. `check-demos` fails on it.
+
+7. **Reward weights sum to 1.0, and at least one component is a
+   `counterweight`.** A reward with only objectives teaches the crude version of
+   what you asked for.
+
+8. **A counterweight must be in genuine tension with the objective.** If every
+   good policy also scores 1.0 on it, it is a `gate` — declare it as one. The
+   probe ladder is where you prove the difference: two good policies, neither
+   dominating.
+
+9. **`null` is "not scored", never `0.0`.** A zero is a claim that the policy
+   did badly. A null is an admission that we do not know. Rendering the second
+   as the first is the quiet way a demo starts lying.
+
+10. **Every `DemoStep` sets `announce`.** `prefers-reduced-motion` clamps every
+    animation to nothing, so colour alone carries the result — and colour alone
+    is not a result. The announcement *is* the feedback for anyone not looking
+    at the screen.
+
+11. **A `RunRef` with `kind: 'intervened'` must name its `intervention`.** This
+    exists so a prompt change can never be presented as a training result by
+    omission.
+
+12. **A `spec` demo ships a real specification** — task, legal actions,
+    deterministic grader, counterweight, and the eval command that would run
+    it. A coming-soon card is not a specification, and `check-demos` refuses one.
+
+## Adding a demo
+
+```bash
+pnpm demo:new claims          # copies src/demos/_template and envs/_template
+```
+
+Then, in order:
+
+1. **Write the environment first**, in `envs/<pkg>/`. The engine, the reward,
+   and a probe ladder. If the reward cannot be probed, it cannot be trusted, and
+   nothing downstream is worth building.
+2. **Port the scorer to TypeScript** and add it to the conformance gate. The
+   browser must be able to re-derive what the environment recorded, or the
+   verify badge is decoration.
+3. **Capture rollouts** with `envs/capture.py`. Include at least one run the
+   agent loses; a demo where the agent always wins teaches nothing about the
+   reward.
+4. **Write `meta.ts`, `narrative.ts`, `surface.tsx`, `adapter.ts`** and the
+   demo module. The narrative's `limits` must name which demo answers each gap.
+5. `pnpm check && pnpm build`.
+
+## The marker syntax for code receipts
+
+`RewardSpec.source.marker` selects a region of a source file to highlight:
+
+```python
+# region: pig-demo/reward
+...
+# endregion: pig-demo/reward
+```
+
+Exclusive of the marker lines, **exactly one pair per file**. Zero or two or
+more is a fatal error in `check-receipts.mjs`, because a silently-wrong region
+would quote the wrong code under a claim that it is the code that ran.
+
+## What to expect to change
+
+The contract was written against a turn-based game. The first vertical demo is
+deliberately a one-shot classifier, so it stresses the weakest axis — no engine,
+no interactive driver, possibly a single run. **Budget one demo-kit refactor
+there and treat it as expected rather than as a failure.** If more than a slot
+is needed, widen the contract once, deliberately, updating `CONTRACT.md` and
+`_template` in the same commit.
@@ -0,0 +1,298 @@
+#!/usr/bin/env node
+/**
+ * Bakes a real `<head>` and real markup into one HTML file per route.
+ *
+ * A client-only SPA ships ONE document. Every route gets the homepage's title,
+ * the homepage's description and the homepage's og:image, because the tags the
+ * app sets live in JavaScript that a crawler or a link unfurler never runs. So
+ * every shared link previews as the homepage, and the per-route OG cards that
+ * `scripts/og.mjs` spent two minutes rendering are dead weight on disk.
+ *
+ * This fixes that the only way that actually works without a server: run the
+ * real app in a real browser against the built output, let it render and set
+ * its own head, then write what the browser ended up with to
+ * `dist/<route>/index.html`. Static hosting serves those directly; the SPA
+ * boots on top and takes over as normal.
+ *
+ * Three things it is careful about, each of which has broken this before:
+ *
+ *   - The app is mounted with `createRoot`, not `hydrateRoot`, so React
+ *     discards the baked markup and re-renders. That is FINE and deliberate:
+ *     there is no hydration mismatch to worry about, and the markup exists for
+ *     the crawler, not for the first paint.
+ *   - next-themes stamps `data-theme` and `color-scheme` onto <html> from the
+ *     viewer's OS preference. Capturing that bakes ONE viewer's theme into the
+ *     file every other viewer downloads, so both are reset before writing.
+ *   - A route that never calls `useSeo` silently keeps the homepage's head.
+ *     That is the exact bug this script exists to fix, so it is a failure, not
+ *     a warning.
+ *
+ * Routes come from the router source and the demo/vertical data, never from a
+ * list in here — see `expandRoutes` in `_lib.mjs`.
+ *
+ * Usage: `node scripts/prerender.mjs` (runs as part of `pnpm build`).
+ * Env: `PIG_DEMO_ORIGIN` overrides the public origin baked into canonical/og:url.
+ */
+
+import fs from 'node:fs';
+import path from 'node:path';
+
+import { SITE_ORIGIN, abs, die, exists, expandRoutes, green, dim, loadAllMetas, loadVerticals, read, rel, serveStatic } from './_lib.mjs';
+
+const DIST = abs('dist');
+const INDEX = path.join(DIST, 'index.html');
+
+if (!exists(INDEX)) die(`${rel(INDEX)} does not exist. Run \`vite build\` before prerendering.`);
+
+const { chromium } = await import('playwright').catch(() => {
+  die('playwright is not installed. `pnpm install` first; prerender needs a real browser to render the app.');
+});
+
+/* ------------------------------------------------------------- the routes */
+
+const { metas, errors: metaErrors } = loadAllMetas();
+for (const e of metaErrors) console.warn(`warn  ${e.file}: ${e.error}`);
+
+const verticals = loadVerticals();
+if (verticals.error) die(`could not read the verticals: ${verticals.error}`);
+
+const { routes, errors } = expandRoutes({ metas, verticals: verticals.list });
+if (errors.length) die(`route enumeration failed:\n  - ${errors.join('\n  - ')}`);
+
+const verticalById = new Map(verticals.list.map((v) => [v.id, v]));
+
+/* --------------------------------------------- the homepage's own defaults */
+
+const indexHtml = read(INDEX);
+const attr = (re) => indexHtml.match(re)?.[1] ?? '';
+const BASE = {
+  title: attr(/<title>([^<]*)<\/title>/i),
+  description: attr(/<meta[^>]+name=["']description["'][^>]*content=["']([^"']*)["']/i),
+  siteName: attr(/<meta[^>]+property=["']og:site_name["'][^>]*content=["']([^"']*)["']/i) || 'PIG Demo',
+  image: attr(/<meta[^>]+property=["']og:image["'][^>]*content=["']([^"']*)["']/i) || `${SITE_ORIGIN}/og/home.png`,
+};
+if (!BASE.title) die(`${rel(INDEX)} has no <title>. The baked head is built from it; there is nothing to start from.`);
+
+const absoluteUrl = (value) => (/^https?:\/\//i.test(value) ? value : `${SITE_ORIGIN}${value.startsWith('/') ? '' : '/'}${value}`);
+
+/** What this route's head SHOULD say, from the data, where we know. */
+function expected(route) {
+  if (route.kind === 'demo') {
+    const meta = metas.get(route.slug);
+    if (!meta) return null;
+    return {
+      title: `${meta.title} — ${BASE.siteName}`,
+      description: meta.tagline,
+      image: meta.ogImage ?? BASE.image,
+    };
+  }
+  if (route.kind === 'vertical') {
+    const vertical = verticalById.get(route.id);
+    if (!vertical) return null;
+    return {
+      title: `${vertical.title} — ${BASE.siteName}`,
+      description: vertical.description ?? BASE.description,
+      image: BASE.image,
+    };
+  }
+  if (route.kind === 'home') return { title: BASE.title, description: BASE.description, image: BASE.image };
+  return null; // /gallery, /honesty — only the app knows what these say
+}
+
+/* ------------------------------------------------------------------ render */
+
+const server = await serveStatic(DIST);
+const browser = await chromium.launch();
+const context = await browser.newContext({
+  viewport: { width: 1280, height: 900 },
+  // Bake the light theme. Capturing whatever the build machine's OS preferred
+  // would ship one viewer's palette to everyone until React re-renders.
+  colorScheme: 'light',
+  reducedMotion: 'reduce',
+});
+
+/** @type {{file: string, html: string}[]} */
+const pending = [];
+/** @type {string[]} */
+const failures = [];
+
+try {
+  for (const route of routes) {
+    const page = await context.newPage();
+    /** @type {string[]} */
+    const crashes = [];
+    page.on('pageerror', (error) => crashes.push(error.message));
+
+    const url = `${server.origin}${route.path}`;
+    await page.goto(url, { waitUntil: 'load', timeout: 30_000 });
+    // The router's lazy routes resolve a chunk before they render anything, so
+    // "load" is too early for every page but the eagerly-imported home.
+    await page.waitForSelector('#root > *', { timeout: 30_000 }).catch(() => {});
+    await page.waitForLoadState('networkidle', { timeout: 30_000 }).catch(() => {});
+    // `document.fonts.ready` resolves to a FontFaceSet, which does not survive
+    // serialisation back across the bridge; return a boolean instead.
+    await page.evaluate(() => document.fonts.ready.then(() => true)).catch(() => {});
+
+    const seen = await page.evaluate(() => ({
+      title: document.title,
+      description: document.querySelector('meta[name="description"]')?.getAttribute('content') ?? '',
+      canonical: document.querySelector('link[rel="canonical"]')?.getAttribute('href') ?? '',
+      image: document.querySelector('meta[property="og:image"]')?.getAttribute('content') ?? '',
+      bodyText: (document.querySelector('#root')?.textContent ?? '').trim().length,
+      lang: document.documentElement.lang,
+    }));
+
+    if (crashes.length) failures.push(`${route.path}: the page threw while rendering — ${crashes[0]}`);
+    if (seen.bodyText < 200) {
+      failures.push(
+        `${route.path}: #root rendered ${seen.bodyText} characters of text. A prerendered page with no content ` +
+          'is worse than none, because it caches an empty page at a real URL.',
+      );
+    }
+
+    const want = expected(route);
+    const appSetItsOwnHead = seen.title !== '' && (route.kind === 'home' || seen.title !== BASE.title);
+    const title = appSetItsOwnHead ? seen.title : want?.title;
+    const description = (appSetItsOwnHead && seen.description) || want?.description || BASE.description;
+    const image = (appSetItsOwnHead && seen.image) || want?.image || BASE.image;
+
+    if (!title) {
+      failures.push(
+        `${route.path}: still carries the homepage title "${BASE.title}" after rendering, and there is no ` +
+          'demo or vertical record to build one from. Call `useSeo(...)` in the page component — a route ' +
+          'that does not set its own head is the exact bug this script exists to fix.',
+      );
+      await page.close();
+      continue;
+    }
+    if (!appSetItsOwnHead) {
+      console.warn(
+        dim(`warn  ${route.path}: the app left the homepage head in place; baking the value from the ${route.kind} record instead.`),
+      );
+    }
+
+    await page.evaluate(bakeHead, {
+      title,
+      description,
+      canonical: absoluteUrl(route.path),
+      image: absoluteUrl(image),
+      siteName: BASE.siteName,
+      robots: null,
+    });
+
+    pending.push({ file: outputFile(route.path), html: await page.content() });
+    await page.close();
+  }
+
+  /* ------------------------------------------------------------- 404.html */
+
+  const page = await context.newPage();
+  // A path no route can ever match, so the router's `*` branch renders.
+  await page.goto(`${server.origin}/__prerender-not-found__`, { waitUntil: 'load', timeout: 30_000 });
+  await page.waitForSelector('#root > *', { timeout: 30_000 }).catch(() => {});
+  await page.evaluate(bakeHead, {
+    title: `Page not found — ${BASE.siteName}`,
+    description: 'That page does not exist. Every demo on this site is a directory in the repository, so a missing one is usually a renamed slug.',
+    canonical: null,
+    image: absoluteUrl(BASE.image),
+    siteName: BASE.siteName,
+    // The 404 body is served at every wrong URL there will ever be. Indexing
+    // it means indexing an unbounded set of URLs that all say "not found".
+    robots: 'noindex, follow',
+  });
+  pending.push({ file: path.join(DIST, '404.html'), html: await page.content() });
+  await page.close();
+} finally {
+  await context.close();
+  await browser.close();
+  await server.close();
+}
+
+if (failures.length) die(`prerender refused to write:\n  - ${failures.join('\n  - ')}`);
+
+/* ------------------------------------------------------------------- write */
+
+for (const { file, html } of pending) {
+  fs.mkdirSync(path.dirname(file), { recursive: true });
+  fs.writeFileSync(file, `${html}\n`, 'utf8');
+}
+
+console.log(green(`prerender: wrote ${pending.length} files under ${rel(DIST)} (origin ${SITE_ORIGIN}).`));
+for (const { file } of pending) console.log(dim(`  ${rel(file)}`));
+
+/* ----------------------------------------------------------------- helpers */
+
+function outputFile(route) {
+  if (route === '/') return path.join(DIST, 'index.html');
+  return path.join(DIST, route.replace(/^\/+/, ''), 'index.html');
+}
+
+/**
+ * Runs INSIDE the page. Rewrites the head, then undoes the theme stamping.
+ *
+ * Written as a single self-contained function because it is serialised to the
+ * browser — it cannot close over anything from this module.
+ */
+function bakeHead(values) {
+  const head = document.head;
+
+  const upsert = (selector, create, content) => {
+    let tag = head.querySelector(selector);
+    if (!tag) {
+      tag = create();
+      head.appendChild(tag);
+    }
+    tag.setAttribute(tag.tagName === 'LINK' ? 'href' : 'content', content);
+    return tag;
+  };
+  const meta = (name, content) =>
+    upsert(`meta[name="${name}"]`, () => {
+      const el = document.createElement('meta');
+      el.setAttribute('name', name);
+      return el;
+    }, content);
+  const property = (prop, content) =>
+    upsert(`meta[property="${prop}"]`, () => {
+      const el = document.createElement('meta');
+      el.setAttribute('property', prop);
+      return el;
+    }, content);
+
+  document.title = values.title;
+  meta('description', values.description);
+
+  property('og:type', 'website');
+  property('og:site_name', values.siteName);
+  property('og:title', values.title);
+  property('og:description', values.description);
+  property('og:image', values.image);
+
+  // summary_large_image is the only card type that shows a 1200x630 image.
+  // Without it X renders a thumbnail and the card reads as a link, not a page.
+  meta('twitter:card', 'summary_large_image');
+  meta('twitter:title', values.title);
+  meta('twitter:description', values.description);
+  meta('twitter:image', values.image);
+
+  if (values.canonical) {
+    upsert('link[rel="canonical"]', () => {
+      const el = document.createElement('link');
+      el.setAttribute('rel', 'canonical');
+      return el;
+    }, values.canonical);
+    property('og:url', values.canonical);
+  } else {
+    head.querySelector('link[rel="canonical"]')?.remove();
+    head.querySelector('meta[property="og:url"]')?.remove();
+  }
+
+  if (values.robots) meta('robots', values.robots);
+  else head.querySelector('meta[name="robots"]')?.remove();
+
+  // next-themes wrote these from the rendering machine's OS preference. Left
+  // in, every visitor downloads that machine's theme and sees it flash to
+  // their own once React boots.
+  document.documentElement.setAttribute('data-theme', 'light');
+  document.documentElement.style.removeProperty('color-scheme');
+  if (document.documentElement.getAttribute('style') === '') document.documentElement.removeAttribute('style');
+}
@@ -111,20 +111,36 @@ function useLineup() {
    const primary = live[0] ?? demos[0];
    const ctaHref = primary ? `/demos/${primary.slug}` : '/demos';

-    return { demos, live, spec, verticals, ctaHref };
+    return { live, spec, verticals, ctaHref };
  }, []);
 }

 /* ------------------------------------------------------------------ panels */

-function DemoRow({ demo, muted = false }: { demo: DemoMeta; muted?: boolean }) {
+/**
+ * Spreads `...rest` onto the Link, and that is not tidiness.
+ *
+ * Both `NavigationMenuLink asChild` and `SheetClose asChild` work by cloning
+ * this element and merging their own `onClick` into it. A component that takes
+ * `demo` and ignores everything else silently drops those handlers: the row
+ * still navigates, and the mega panel or the mobile sheet stays open on top of
+ * the page you just arrived at.
+ */
+function DemoRow({
+  demo,
+  muted = false,
+  className,
+  ...rest
+}: { demo: DemoMeta; muted?: boolean } & Omit<React.ComponentProps<typeof Link>, 'to'>) {
  return (
    <Link
      to={`/demos/${demo.slug}`}
      className={cn(
-        'group/row flex gap-3 rounded-lg p-2.5 transition-colors duration-1 ease-enter hover:bg-surface-2',
+        'flex gap-3 rounded-lg p-2.5 transition-colors duration-1 ease-enter hover:bg-surface-2',
        muted && 'opacity-70 hover:opacity-100',
+        className,
      )}
+      {...rest}
    >
      <DemoIcon name={demo.icon} className="mt-0.5 size-4 text-accent-fg" />
      <span className="flex min-w-0 flex-col gap-0.5">
@@ -272,10 +288,12 @@ function TasksetSource({ className }: { className?: string }) {

 function HowItWorksPanel({ ctaHref }: { ctaHref: string }) {
  return (
-    // 820px, not more: the panel hangs off the LEFT edge of the nav, which
-    // starts about 100px in, so anything wider than this pushes past the right
-    // edge of a 1024px laptop and gives the whole page a horizontal scrollbar.
-    <div className="w-[min(90vw,820px)] p-4">
+    <div
+      // 820px, not more: the panel hangs off the LEFT edge of the nav, which
+      // starts about 100px in, so anything wider pushes past the right edge of
+      // a 1024px laptop and gives the whole page a horizontal scrollbar.
+      className="w-[min(90vw,820px)] p-4"
+    >
      <div className="grid grid-cols-[minmax(0,0.85fr)_minmax(0,1fr)] gap-4">
        <dl className="flex flex-col gap-3">
          {CONCEPTS.map((concept) => (