Capture harness, fixture verification, CI, and the public README

The site does no live inference. Rollouts are captured once against spark-1 and replayed at their recorded wall-clock — a public demo with no auth cannot hold an API key, and a recorded run can be scrubbed, permalinked, blind-compared and verified in ways a live one cannot. What stops it being a video is that the browser re-derives every number from the recorded moves. verify_fixtures.py is the Python half of that: it replays every committed fixture through the engine and reproduces its own rewards. All 16 land at delta 0.0. A fixture that cannot be regenerated is a claim with no receipt. First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats guesses it has already played, invents words (trape, slith, postt, boomy), and contradicts its own feedback — consistency 0.09 to 0.17. That is the published failure taxonomy showing up in our own data on the first run, and it is why `consistency` is a reward component rather than a footnote. A capture failure is recorded as a turn with a null reply, never dropped. A capture that silently discarded failed turns would be reporting a better model than the one that ran. CI gates both halves and four things that fail silently in production: the word lists must rebuild byte-identically, the prerendered routes must carry their own baked og tags (crawlers do not run JS, so without them every shared link previews as the homepage), no blob: URL may reach the bundle (the site's CSP has no worker-src, so it falls back to default-src 'self' and a blob worker is blocked with no error), and the conformance digest must match across languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
2026-08-28 15:47:31 -07:00
parent 69607fbfe9
commit 408ce4a525
43 changed files with 5279 additions and 136 deletions
@@ -0,0 +1,116 @@
+import {
+  CartesianGrid,
+  Line,
+  LineChart,
+  ReferenceLine,
+  ResponsiveContainer,
+  Tooltip,
+  XAxis,
+  YAxis,
+} from 'recharts';
+import type { TooltipProps } from 'recharts';
+import { formatNumber } from './format';
+
+export interface MetricPoint {
+  /** Whatever the x axis is counting: checkpoint, step, arm name. */
+  x: string | number;
+  y: number;
+}
+
+export interface MetricChartProps {
+  data: MetricPoint[];
+  /** The rule the headline number is being compared against. */
+  baseline?: { value: number; label: string };
+  height: number;
+  /** Off under `prefers-reduced-motion`; recharts animates on mount by default. */
+  animate: boolean;
+  digits?: number;
+}
+
+/**
+ * The chart, in its own module so `React.lazy` can hold recharts out of the
+ * entry chunk. Nothing else in the shell may import this file directly — an
+ * ordinary import here defeats the whole arrangement and the entry chunk grows
+ * by ~100 kB without anyone noticing.
+ *
+ * Every colour is read from the token layer at paint time rather than passed in
+ * as a literal, so the chart follows the theme toggle without a re-render.
+ */
+export default function MetricChart({
+  data,
+  baseline,
+  height,
+  animate,
+  digits = 3,
+}: MetricChartProps) {
+  return (
+    <div style={{ height }} className="w-full">
+      <ResponsiveContainer width="100%" height="100%">
+        <LineChart data={data} margin={{ top: 8, right: 12, bottom: 4, left: 4 }}>
+          <CartesianGrid stroke="hsl(var(--border))" strokeDasharray="2 4" vertical={false} />
+          <XAxis
+            dataKey="x"
+            tick={{ fill: 'hsl(var(--muted))', fontSize: 11 }}
+            tickLine={false}
+            axisLine={{ stroke: 'hsl(var(--border))' }}
+            minTickGap={12}
+          />
+          <YAxis
+            tick={{ fill: 'hsl(var(--muted))', fontSize: 11 }}
+            tickLine={false}
+            axisLine={false}
+            width={40}
+            tickFormatter={(value: number) => formatNumber(value, digits > 2 ? 2 : digits)}
+          />
+          {baseline ? (
+            <ReferenceLine
+              y={baseline.value}
+              stroke="hsl(var(--muted))"
+              strokeDasharray="5 4"
+              label={{
+                value: baseline.label,
+                position: 'insideTopLeft',
+                fill: 'hsl(var(--muted))',
+                fontSize: 11,
+              }}
+            />
+          ) : null}
+          <Tooltip
+            cursor={{ stroke: 'hsl(var(--border))' }}
+            content={(props: TooltipProps<number, string>) => (
+              <ChartTooltip {...props} digits={digits} />
+            )}
+          />
+          <Line
+            type="monotone"
+            dataKey="y"
+            stroke="hsl(var(--accent))"
+            strokeWidth={2}
+            dot={{ r: 2.5, fill: 'hsl(var(--accent))', strokeWidth: 0 }}
+            activeDot={{ r: 4 }}
+            isAnimationActive={animate}
+            animationDuration={400}
+          />
+        </LineChart>
+      </ResponsiveContainer>
+    </div>
+  );
+}
+
+function ChartTooltip({
+  active,
+  payload,
+  label,
+  digits,
+}: TooltipProps<number, string> & { digits: number }) {
+  if (!active || !payload || payload.length === 0) return null;
+  const value = payload[0]?.value;
+  return (
+    <div className="card px-2.5 py-1.5 text-xs shadow-md">
+      <p className="nums text-muted">{String(label)}</p>
+      <p className="nums font-mono font-semibold">
+        {typeof value === 'number' ? formatNumber(value, digits) : '—'}
+      </p>
+    </div>
+  );
+}