Capture harness, fixture verification, CI, and the public README

The site does no live inference. Rollouts are captured once against spark-1 and
replayed at their recorded wall-clock — a public demo with no auth cannot hold
an API key, and a recorded run can be scrubbed, permalinked, blind-compared and
verified in ways a live one cannot. What stops it being a video is that the
browser re-derives every number from the recorded moves.

verify_fixtures.py is the Python half of that: it replays every committed
fixture through the engine and reproduces its own rewards. All 16 land at
delta 0.0. A fixture that cannot be regenerated is a claim with no receipt.

First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats
guesses it has already played, invents words (trape, slith, postt, boomy),
and contradicts its own feedback — consistency 0.09 to 0.17. That is the
published failure taxonomy showing up in our own data on the first run, and it
is why `consistency` is a reward component rather than a footnote.

A capture failure is recorded as a turn with a null reply, never dropped. A
capture that silently discarded failed turns would be reporting a better model
than the one that ran.

CI gates both halves and four things that fail silently in production: the word
lists must rebuild byte-identically, the prerendered routes must carry their own
baked og tags (crawlers do not run JS, so without them every shared link
previews as the homepage), no blob: URL may reach the bundle (the site's CSP has
no worker-src, so it falls back to default-src 'self' and a blob worker is
blocked with no error), and the conformance digest must match across languages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
2026-08-28 15:47:31 -07:00
parent 69607fbfe9
commit 408ce4a525
43 changed files with 5279 additions and 136 deletions
+13 -3
View File
@@ -247,7 +247,14 @@ function TasksetSource({ className }: { className?: string }) {
<p className="text-xs font-semibold uppercase tracking-wider text-muted">
The actual taskset
</p>
<pre className="overflow-x-auto text-[11px] leading-relaxed text-fg">
{/*
Wrapped, not scrolled. The source is quoted verbatim — reformatting it
to fit would make it stop being a quote — and the widest line is a third
wider than this column at any font size worth reading. In a panel whose
entire job is "this is real code", a visible wrap beats a third of the
line hidden behind a scrollbar nobody in a boardroom will drag.
*/}
<pre className="overflow-x-auto whitespace-pre-wrap break-words text-[11px] leading-relaxed text-fg">
<code className="font-mono">{TASKSET_SOURCE}</code>
</pre>
<a
@@ -265,8 +272,11 @@ function TasksetSource({ className }: { className?: string }) {
function HowItWorksPanel({ ctaHref }: { ctaHref: string }) {
return (
<div className="w-[min(92vw,760px)] p-4">
<div className="grid grid-cols-2 gap-4">
// 820px, not more: the panel hangs off the LEFT edge of the nav, which
// starts about 100px in, so anything wider than this pushes past the right
// edge of a 1024px laptop and gives the whole page a horizontal scrollbar.
<div className="w-[min(90vw,820px)] p-4">
<div className="grid grid-cols-[minmax(0,0.85fr)_minmax(0,1fr)] gap-4">
<dl className="flex flex-col gap-3">
{CONCEPTS.map((concept) => (
<div key={concept.term} className="flex flex-col gap-0.5">