Capture harness, fixture verification, CI, and the public README
The site does no live inference. Rollouts are captured once against spark-1 and replayed at their recorded wall-clock — a public demo with no auth cannot hold an API key, and a recorded run can be scrubbed, permalinked, blind-compared and verified in ways a live one cannot. What stops it being a video is that the browser re-derives every number from the recorded moves. verify_fixtures.py is the Python half of that: it replays every committed fixture through the engine and reproduces its own rewards. All 16 land at delta 0.0. A fixture that cannot be regenerated is a claim with no receipt. First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats guesses it has already played, invents words (trape, slith, postt, boomy), and contradicts its own feedback — consistency 0.09 to 0.17. That is the published failure taxonomy showing up in our own data on the first run, and it is why `consistency` is a reward component rather than a footnote. A capture failure is recorded as a turn with a null reply, never dropped. A capture that silently discarded failed turns would be reporting a better model than the one that ran. CI gates both halves and four things that fail silently in production: the word lists must rebuild byte-identically, the prerendered routes must carry their own baked og tags (crawlers do not run JS, so without them every shared link previews as the homepage), no blob: URL may reach the bundle (the site's CSP has no worker-src, so it falls back to default-src 'self' and a blob worker is blocked with no error), and the conformance digest must match across languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
@@ -247,7 +247,14 @@ function TasksetSource({ className }: { className?: string }) {
|
||||
<p className="text-xs font-semibold uppercase tracking-wider text-muted">
|
||||
The actual taskset
|
||||
</p>
|
||||
<pre className="overflow-x-auto text-[11px] leading-relaxed text-fg">
|
||||
{/*
|
||||
Wrapped, not scrolled. The source is quoted verbatim — reformatting it
|
||||
to fit would make it stop being a quote — and the widest line is a third
|
||||
wider than this column at any font size worth reading. In a panel whose
|
||||
entire job is "this is real code", a visible wrap beats a third of the
|
||||
line hidden behind a scrollbar nobody in a boardroom will drag.
|
||||
*/}
|
||||
<pre className="overflow-x-auto whitespace-pre-wrap break-words text-[11px] leading-relaxed text-fg">
|
||||
<code className="font-mono">{TASKSET_SOURCE}</code>
|
||||
</pre>
|
||||
<a
|
||||
@@ -265,8 +272,11 @@ function TasksetSource({ className }: { className?: string }) {
|
||||
|
||||
function HowItWorksPanel({ ctaHref }: { ctaHref: string }) {
|
||||
return (
|
||||
<div className="w-[min(92vw,760px)] p-4">
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
// 820px, not more: the panel hangs off the LEFT edge of the nav, which
|
||||
// starts about 100px in, so anything wider than this pushes past the right
|
||||
// edge of a 1024px laptop and gives the whole page a horizontal scrollbar.
|
||||
<div className="w-[min(90vw,820px)] p-4">
|
||||
<div className="grid grid-cols-[minmax(0,0.85fr)_minmax(0,1fr)] gap-4">
|
||||
<dl className="flex flex-col gap-3">
|
||||
{CONCEPTS.map((concept) => (
|
||||
<div key={concept.term} className="flex flex-col gap-0.5">
|
||||
|
||||
Reference in New Issue
Block a user