karti-ai
408ce4a525
The site does no live inference. Rollouts are captured once against spark-1 and replayed at their recorded wall-clock — a public demo with no auth cannot hold an API key, and a recorded run can be scrubbed, permalinked, blind-compared and verified in ways a live one cannot. What stops it being a video is that the browser re-derives every number from the recorded moves. verify_fixtures.py is the Python half of that: it replays every committed fixture through the engine and reproduces its own rewards. All 16 land at delta 0.0. A fixture that cannot be regenerated is a claim with no receipt. First real measurement, thinking off, 8 seeds: solved 0/8. The model repeats guesses it has already played, invents words (trape, slith, postt, boomy), and contradicts its own feedback — consistency 0.09 to 0.17. That is the published failure taxonomy showing up in our own data on the first run, and it is why `consistency` is a reward component rather than a footnote. A capture failure is recorded as a turn with a null reply, never dropped. A capture that silently discarded failed turns would be reporting a better model than the one that ran. CI gates both halves and four things that fail silently in production: the word lists must rebuild byte-identically, the prerendered routes must carry their own baked og tags (crawlers do not run JS, so without them every shared link previews as the homepage), no blob: URL may reach the bundle (the site's CSP has no worker-src, so it falls back to default-src 'self' and a blob worker is blocked with no error), and the conformance digest must match across languages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
87 lines
3.2 KiB
YAML
87 lines
3.2 KiB
YAML
name: ci
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
|
|
concurrency:
|
|
group: ci-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
web:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- uses: pnpm/action-setup@v4
|
|
with: { version: 11.21.0 }
|
|
- uses: actions/setup-node@v5
|
|
with: { node-version: 22, cache: pnpm }
|
|
|
|
# Asserts the toolchain matches what package.json pins, rather than
|
|
# discovering a mismatch three steps later as an unrelated build error.
|
|
- name: preflight
|
|
run: |
|
|
test "$(pnpm -v)" = "11.21.0" || { echo "pnpm $(pnpm -v) != 11.21.0"; exit 1; }
|
|
|
|
- run: CI=true pnpm install --frozen-lockfile
|
|
- run: pnpm typecheck
|
|
- run: pnpm check
|
|
- run: pnpm test
|
|
- run: pnpm build
|
|
- run: node scripts/bundle-budget.mjs
|
|
|
|
# The prerender pass writes a real HTML file per route. Crawlers do not
|
|
# run JavaScript, so without these every shared link previews as the
|
|
# homepage — assert the baked tags actually landed.
|
|
- name: prerendered head is real
|
|
run: |
|
|
test -f dist/demos/wordle/index.html || { echo "no prerendered demo route"; exit 1; }
|
|
grep -q 'og:title' dist/demos/wordle/index.html || { echo "og tags missing"; exit 1; }
|
|
grep -qv 'PIG Demo — RL environments you can play</title>' dist/demos/wordle/index.html \
|
|
|| { echo "demo route kept the homepage title"; exit 1; }
|
|
test -f dist/404.html || { echo "no 404.html"; exit 1; }
|
|
|
|
# A blob-backed worker is blocked in production and nowhere else: the
|
|
# site's CSP has no worker-src, so it falls back to default-src 'self'.
|
|
# The failure is silent — the solver simply never boots.
|
|
- name: no inline workers
|
|
run: |
|
|
! grep -rqE "createObjectURL|blob:" dist/assets/*.js \
|
|
|| { echo "a blob: URL reached the bundle; CSP will block it in prod"; exit 1; }
|
|
|
|
python:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- uses: astral-sh/setup-uv@v6
|
|
with: { enable-cache: true }
|
|
|
|
- run: uv sync --all-packages
|
|
|
|
# Rebuild the word lists from their committed sources and assert the
|
|
# output is byte-identical. If it is not, every downstream number —
|
|
# the conformance digest included — is describing a different game.
|
|
- name: word lists rebuild identically
|
|
run: |
|
|
uv run python envs/wordle_five/words/build_words.py
|
|
git diff --exit-code envs/wordle_five/words/*.json
|
|
|
|
- run: uv run pytest envs/wordle_five/tests -q
|
|
- run: uv run python envs/probe.py
|
|
|
|
# The cross-language gate. Both halves score all 21.2M (guess, answer)
|
|
# pairs; the digests must match each other and the committed value.
|
|
- uses: pnpm/action-setup@v4
|
|
with: { version: 11.21.0 }
|
|
- uses: actions/setup-node@v5
|
|
with: { node-version: 22, cache: pnpm }
|
|
- run: CI=true pnpm install --frozen-lockfile
|
|
- run: pnpm conformance
|
|
|
|
# Every committed fixture must replay through the Python engine and
|
|
# reproduce its own recorded rewards. A fixture that cannot be
|
|
# regenerated is a claim with no receipt behind it.
|
|
- run: uv run python envs/verify_fixtures.py
|