Capture harness, fixture verification, CI, and the public README

The site does no live inference. Rollouts are captured once against spark-1 and
replayed at their recorded wall-clock — a public demo with no auth cannot hold
an API key, and a recorded run can be scrubbed, permalinked, blind-compared and
verified in ways a live one cannot. What stops it being a video is that the
browser re-derives every number from the recorded moves.

verify_fixtures.py is the Python half of that: it replays every committed
fixture through the engine and reproduces its own rewards. All 16 land at
delta 0.0. A fixture that cannot be regenerated is a claim with no receipt.

First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats
guesses it has already played, invents words (trape, slith, postt, boomy),
and contradicts its own feedback — consistency 0.09 to 0.17. That is the
published failure taxonomy showing up in our own data on the first run, and it
is why `consistency` is a reward component rather than a footnote.

A capture failure is recorded as a turn with a null reply, never dropped. A
capture that silently discarded failed turns would be reporting a better model
than the one that ran.

CI gates both halves and four things that fail silently in production: the word
lists must rebuild byte-identically, the prerendered routes must carry their own
baked og tags (crawlers do not run JS, so without them every shared link
previews as the homepage), no blob: URL may reach the bundle (the site's CSP has
no worker-src, so it falls back to default-src 'self' and a blob worker is
blocked with no error), and the conformance digest must match across languages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
2026-08-28 15:47:31 -07:00
parent 69607fbfe9
commit 408ce4a525
43 changed files with 5279 additions and 136 deletions
+100
View File
@@ -0,0 +1,100 @@
import type { ComponentType } from 'react';
import { Link } from 'react-router-dom';
import { ArrowRight } from 'lucide-react';
import type { DemoMeta } from '@/lib/demo-kit/types';
import { cn } from '@/lib/utils';
import { DemoIcon } from './icons';
export interface DemoCardProps<T> {
meta: DemoMeta;
/** Defaults to the canonical demo route. */
href?: string;
/**
* The demo's OWN board, drawn compact, as the thumbnail. A screenshot would
* go stale the first time the board changes and nobody would notice; this
* cannot, because it is the same component the demo page renders.
*/
Surface?: ComponentType<{ state: T; compact?: boolean }>;
/** A representative state for the thumbnail — usually a solved board. */
thumbnailState?: T;
className?: string;
}
export function DemoCard<T>({
meta,
href,
Surface,
thumbnailState,
className,
}: DemoCardProps<T>) {
const to = href ?? `/demo/${meta.slug}`;
const isSpec = meta.status === 'spec';
const showSurface = Surface !== undefined && thumbnailState !== undefined;
return (
<article
className={cn(
'card group relative flex flex-col overflow-hidden transition-colors duration-2 ease-enter hover:border-brand/50',
className,
)}
>
<div className="flex items-start gap-3 p-4 pb-3">
<span className="grid h-10 w-10 shrink-0 place-items-center rounded-lg bg-accent-subtle text-accent-fg">
<DemoIcon name={meta.icon} className="h-5 w-5" />
</span>
<div className="min-w-0 flex-1">
<h3 className="text-base font-semibold leading-tight">
{/* Stretched link: the whole card is the hit target, but there is
still exactly ONE link in the accessibility tree for it. */}
<Link
to={to}
className="after:absolute after:inset-0 after:content-[''] focus-visible:outline-none"
>
{meta.title}
</Link>
</h3>
<p className="mt-0.5 text-sm leading-snug text-muted">{meta.tagline}</p>
</div>
{isSpec ? (
<span className="shrink-0 rounded-md border border-border px-1.5 py-0.5 text-[10px] font-semibold uppercase tracking-wide text-muted">
Spec
</span>
) : null}
</div>
{showSurface ? (
<div className="mx-4 overflow-hidden rounded-lg bg-surface-2 p-3">
{/* Decorative here: the title and tagline already name the demo, and
a screen reader has no use for a board with no run behind it. */}
<div aria-hidden="true">
<Surface state={thumbnailState as T} compact />
</div>
</div>
) : null}
<dl className="mt-3 flex flex-wrap gap-x-4 gap-y-1 px-4 text-xs">
<div className="flex gap-1">
<dt className="text-muted">For</dt>
<dd className="font-medium">{meta.persona}</dd>
</div>
<div className="flex gap-1">
<dt className="text-muted">Vertical</dt>
<dd className="font-medium capitalize">{meta.vertical.replace(/-/g, ' ')}</dd>
</div>
</dl>
<p className="mt-2 px-4 pb-4 text-xs leading-relaxed text-muted">
<span className="font-medium text-fg">Reward: </span>
{meta.rewardLine}
</p>
<p className="mt-auto flex items-center gap-1 border-t border-border px-4 py-2.5 text-sm font-medium text-accent-fg">
{isSpec ? 'Read the specification' : 'Open the demo'}
<ArrowRight
className="h-4 w-4 transition-transform duration-2 ease-enter group-hover:translate-x-0.5"
aria-hidden="true"
/>
</p>
</article>
);
}