wordle-five: the engine, the reward, the solver and the probe that checks them

The Python is the source of truth; src/demos/wordle/engine.ts will be a port of
it, and CI gates the two against a SHA-256 over all 21.2M (guess, answer)
pattern pairs rather than a hand-picked vector file — a vector file only ever
catches the cases somebody thought of.

The reward is three weighted components, and the third one is the reason this
demo is worth building. `solved` and `economy` pull toward winning. `consistency`
pulls against them, because a player maximising information deliberately guesses
words that cannot win — a word that splits the remaining candidates evenly
teaches more than a word that might happen to be right. That is good play, and
it costs consistency.

The probe ladder proves the tension is real rather than asserted:

  inaction        0.0000   crude       0.0111   plausible  0.1224
  candidate_only  0.8925   exhaustive  0.9031   oracle     0.9458

The two good policies are 0.05 apart and neither dominates — the entropy oracle
takes 1.00 economy and 0.73 consistency, the candidate-only player takes 0.75
and 1.00. Which one wins is a decision about what you want, which is the whole
argument the site exists to make. probe.py fails CI if either starts dominating.

Two traps found by building it. `consistency` is scored over turns SPENT, not
guesses accepted: counting only legal guesses hands a free 1.0 to a policy that
plays one word and then jams the parser five times — one guess, no
contradictions, perfect score. And `economy`'s denominator is the depth the
SHIPPED solver reaches, not a depth-optimal search: entropy-greedy is not
depth-optimal, so grading it against an exact optimum would make the oracle
rung fail its own assertion on some seeds.

The word lists are built from Wordnik (MIT) intersected with SCOWL, never from
the original game's 2,315 answers. 4,603 answers makes this materially harder
than the original, so the published SALET/3.4212 results are cited as belonging
to that list and our own reference player's TARES/3.72 is measured here.

verifiers is an optional extra. The engine, reward, solver and probe all run —
and gate — without an RL stack resolvable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB

src/content/evidence.ts

@@ -0,0 +1,118 @@
+/**
+ * Every external claim this site makes, with the link that backs it, plus the
+ * handful of numbers about our own word list.
+ *
+ * Home and Honesty both quote these. They are here, once, so the two pages
+ * cannot drift into citing the same fact two different ways — which is the
+ * usual way an honest site becomes a dishonest one.
+ *
+ * The word-list counts are literals rather than imports on purpose: the answer
+ * list is 4,603 strings sitting outside the Vite root, and pulling it in to
+ * render one number would put the whole dictionary in the entry chunk. They are
+ * reproducible with the command in `wordListRebuild` and are checked by CI.
+ */
+
+export interface Citation {
+  /** What the source proves, in one line. */
+  claim: string;
+  label: string;
+  href: string;
+}
+
+/**
+ * The credential. Wordle is not our choice of demo — it is Prime Intellect's
+ * own hello-world, in three separate places in their stack.
+ */
+export const helloWorldCitations: readonly Citation[] = [
+  {
+    claim: 'One of the five basic end-to-end examples in prime-rl, their RL trainer.',
+    label: 'prime-rl / examples / basic / wordle',
+    href: 'https://github.com/PrimeIntellect-ai/prime-rl/tree/main/examples/basic/wordle',
+  },
+  {
+    claim: 'A shipped environment in verifiers, the library the whole ecosystem builds on.',
+    label: 'verifiers / environments / wordle',
+    href: 'https://github.com/PrimeIntellect-ai/verifiers/tree/main/environments/wordle',
+  },
+  {
+    claim: 'The environment used in the official lab-cookbook prompt-optimisation tutorial.',
+    label: 'lab-cookbook / guides / prompt optimization',
+    href: 'https://github.com/PrimeIntellect-ai/lab-cookbook/tree/main/guides/04-prompt-optimization',
+  },
+];
+
+/**
+ * The one measured, citable training result on this site. It is theirs, not
+ * ours, and it is a WIN RATE.
+ *
+ * Their README also publishes average-reward figures for the same runs. We do
+ * not quote those anywhere and neither should you: the reward function has
+ * changed across versions of the environment, the two numbers were produced
+ * under different versions, and nobody re-measured them. A win rate survives a
+ * reward change. An average reward does not.
+ */
+export const trainingResult = {
+  model: 'Qwen3-1.7B',
+  before: '0%',
+  after: '~60%',
+  metric: 'win rate',
+  method: 'SFT warm-up, then multi-turn RL with group-relative advantages (GRPO)',
+  /** From the same README: 20 held-out words, 3 rollouts each. */
+  evalDescription: '20 held-out words the model never trained on, played 3 times each',
+  source: {
+    label: 'prime-rl / examples / basic / wordle',
+    href: 'https://github.com/PrimeIntellect-ai/prime-rl/tree/main/examples/basic/wordle',
+  },
+  checkpoints: [
+    {
+      label: 'PrimeIntellect/Qwen3-1.7B-Wordle-SFT',
+      href: 'https://huggingface.co/PrimeIntellect/Qwen3-1.7B-Wordle-SFT',
+    },
+    {
+      label: 'PrimeIntellect/Qwen3-1.7B-Wordle-RL',
+      href: 'https://huggingface.co/PrimeIntellect/Qwen3-1.7B-Wordle-RL',
+    },
+  ],
+} as const;
+
+/**
+ * Our word list, by the numbers. Answers are the intersection of Wordnik's
+ * headwords and SCOWL's common-American tier, minus a short hand-written
+ * blocklist; guesses are all of Wordnik's five-letter words.
+ */
+export const wordList = {
+  answers: 4603,
+  guesses: 11846,
+  /** Share of answers ending in a plain plural S. */
+  endsInS: '32.9%',
+  /** Share of answers ending in -ED. */
+  endsInEd: '6.1%',
+  /** Both together. */
+  endsInSorEd: '39.0%',
+  /** The original game's hand-curated answer list, for comparison. */
+  originalAnswers: 2315,
+  rebuild: 'uv run python envs/wordle_five/words/build_words.py',
+  sources: [
+    { label: 'Wordnik word list (2021-07-29)', href: 'https://github.com/wordnik/wordlist' },
+    { label: 'SCOWL / wamerican (2020.12.07)', href: 'http://wordlist.aspell.net/' },
+  ],
+} as const;
+
+/**
+ * The famous 3.42. It belongs to the original game's 2,315-word answer list and
+ * to Alex Selby's exact solver, and it is quoted on this site only to say that
+ * it is not ours.
+ */
+export const optimalPlay = {
+  average: '3.42 guesses',
+  label: 'Selby’s exact optimal solution for the original 2,315-word list',
+  href: 'https://sonorouschocolate.com/notes/index.php/The_best_strategies_for_Wordle',
+} as const;
+
+/** Commands anyone can run against a clone of this repository. */
+export const reproduce = {
+  clone: 'git clone https://github.com/karti-ai/PIG-Demo',
+  install: 'cd PIG-Demo && uv sync --all-packages',
+  evaluate: 'uv run vf-eval wordle-five -n 8',
+  conformance: 'pnpm conformance',
+} as const;