Capture harness, fixture verification, CI, and the public README

The site does no live inference. Rollouts are captured once against spark-1 and replayed at their recorded wall-clock — a public demo with no auth cannot hold an API key, and a recorded run can be scrubbed, permalinked, blind-compared and verified in ways a live one cannot. What stops it being a video is that the browser re-derives every number from the recorded moves. verify_fixtures.py is the Python half of that: it replays every committed fixture through the engine and reproduces its own rewards. All 16 land at delta 0.0. A fixture that cannot be regenerated is a claim with no receipt. First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats guesses it has already played, invents words (trape, slith, postt, boomy), and contradicts its own feedback — consistency 0.09 to 0.17. That is the published failure taxonomy showing up in our own data on the first run, and it is why `consistency` is a reward component rather than a footnote. A capture failure is recorded as a turn with a null reply, never dropped. A capture that silently discarded failed turns would be reporting a better model than the one that ran. CI gates both halves and four things that fail silently in production: the word lists must rebuild byte-identically, the prerendered routes must carry their own baked og tags (crawlers do not run JS, so without them every shared link previews as the homepage), no blob: URL may reach the bundle (the site's CSP has no worker-src, so it falls back to default-src 'self' and a blob worker is blocked with no error), and the conformance digest must match across languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
2026-08-28 15:47:31 -07:00
parent 69607fbfe9
commit 408ce4a525
43 changed files with 5279 additions and 136 deletions
@@ -0,0 +1,126 @@
+import { Suspense, lazy, useMemo } from 'react';
+import { cn } from '@/lib/utils';
+import { formatNumber, formatSigned, usePrefersReducedMotion } from './format';
+import type { MetricPoint } from './MetricChart';
+import { EditedChip } from './StatStrip';
+
+export type { MetricPoint } from './MetricChart';
+
+// Lazy, and deliberately not a static import: recharts is ~100 kB gzipped and
+// exactly one surface on the site uses it. `vite.config.ts` also names it as
+// its own manual chunk, so this stays out of the entry bundle in both dev and
+// production builds.
+const MetricChart = lazy(() => import('./MetricChart'));
+
+const CHART_HEIGHT = 168;
+
+export interface MetricMoverProps {
+  /** The one number a reader would repeat in a meeting. */
+  label: string;
+  value: number;
+  digits?: number;
+  unit?: string;
+  /** The dashed rule: what the number was before, or what counts as par. */
+  baseline?: { value: number; label: string };
+  series?: MetricPoint[];
+  /** One sentence on what moved it. Not a caveat — the mechanism. */
+  caption?: string;
+  edited?: boolean;
+  className?: string;
+}
+
+/**
+ * The headline number, with the line that shows it moving.
+ *
+ * The reference rule is not decoration: a number on its own is a claim, and a
+ * number against the line it used to sit on is evidence. If a demo has no
+ * baseline to draw, it should not be using this component.
+ */
+export function MetricMover({
+  label,
+  value,
+  digits = 3,
+  unit,
+  baseline,
+  series,
+  caption,
+  edited = false,
+  className,
+}: MetricMoverProps) {
+  const reducedMotion = usePrefersReducedMotion();
+  const delta = baseline ? value - baseline.value : null;
+  const points = useMemo(() => series ?? [], [series]);
+
+  return (
+    <section aria-label={label} className={cn('card p-4', className)}>
+      <div className="flex flex-wrap items-baseline gap-x-3 gap-y-1">
+        <h3 className="text-sm font-medium text-muted">{label}</h3>
+        {edited ? <EditedChip /> : null}
+      </div>
+      <p className="mt-1 flex items-baseline gap-2">
+        <span className="nums text-4xl font-semibold leading-none tracking-tight">
+          {formatNumber(value, digits)}
+        </span>
+        {unit ? <span className="text-sm text-muted">{unit}</span> : null}
+        {delta !== null ? (
+          <span
+            className={cn(
+              'nums text-sm font-medium',
+              delta > 0 ? 'text-positive' : delta < 0 ? 'text-danger' : 'text-muted',
+            )}
+          >
+            {formatSigned(delta, digits)} vs {baseline?.label}
+          </span>
+        ) : null}
+      </p>
+
+      {points.length > 1 ? (
+        <div className="mt-3">
+          <Suspense
+            fallback={
+              // Reserve the exact chart height. A chart that pops in and pushes
+              // the caption down is the layout shift this whole page is trying
+              // not to have.
+              <div
+                style={{ height: CHART_HEIGHT }}
+                className="w-full rounded-lg bg-surface-2"
+                aria-hidden="true"
+              />
+            }
+          >
+            <MetricChart
+              data={points}
+              {...(baseline ? { baseline } : {})}
+              height={CHART_HEIGHT}
+              animate={!reducedMotion}
+              digits={digits}
+            />
+          </Suspense>
+          {/* The chart is a picture of the table; the table is the accessible
+              version of the picture. Both are the same numbers. */}
+          <details className="mt-1">
+            <summary className="tap inline-flex cursor-pointer items-center text-xs text-muted hover:text-fg">
+              Show these points as a table
+            </summary>
+            <table className="nums mt-1.5 w-full border-collapse font-mono text-xs">
+              <tbody className="divide-y divide-border">
+                {points.map((point) => (
+                  <tr key={String(point.x)}>
+                    <th scope="row" className="py-1 text-left font-normal text-muted">
+                      {String(point.x)}
+                    </th>
+                    <td className="py-1 text-right">{formatNumber(point.y, digits)}</td>
+                  </tr>
+                ))}
+              </tbody>
+            </table>
+          </details>
+        </div>
+      ) : null}
+
+      {caption ? (
+        <p className="mt-3 text-sm leading-relaxed text-muted">{caption}</p>
+      ) : null}
+    </section>
+  );
+}