Alert Triage: environment #2, built end to end by the pipeline

The first environment shipped through .claude/workflows/new-environment.js:
specification, three adversarial reviews (all 'fixable', none fatal), the
Python environment, the TypeScript port, captured rollouts, and the demo page.
Eleven agents, no errors.

The proof that the platform scales is one line long. Alert Triage has a
completely different shape from Word Five — JSON actions, priced lookups, an
analyst screen instead of a grid — and the only change under
src/components/demo/ is a comment edit, because the isolation lint refused the
word "wordle" there. Zero shell code changed. 415 contract checks now pass
against two demos, up from 206 against one.

The environment is honest by construction. Every alert is synthetic, generated
from the seed, and the banner saying so sits inside the board surface. Two of
the eleven scenario templates are hidden-suspicious: generated by the same code
as their benign twin with the signal overlaid only in lookup data, so the free
screen is identically distributed and a screen-only policy STRUCTURALLY cannot
tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden
seeds. That is the counterweight made real rather than asserted.

Twelve policies, thirteen ladder assertions, a genuine three-way trade:

  fast      0.846   wins hours (0.85), misses every hidden case
  targeted  0.894   wins the shipped total
  thorough  0.820   wins evidence (1.00), spends 2.9 hours

None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at
delta 0, and conformance gated on world + scorer + protocol so the browser shows
the same alert for ?seed= that Python generated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB

envs/alert_triage/tests/test_triage_screen_leak.py

@@ -0,0 +1,87 @@
+"""The hidden tier is hidden only if the generator keeps it so. This is the gate.
+
+Two checks. The first is structural and exact: for every seed the free screen
+is byte-identical with the overlay on or off, for every overlay the screen can
+carry. The second is distributional: across seeds, the screen features a
+policy could key on do not separate hidden from benign beyond noise. The
+generator draws the tier BEFORE the template and the screen, so the two
+populations are the same draw; this test is what would catch a regression that
+made them differ.
+"""
+
+from __future__ import annotations
+
+import math
+
+import pytest
+
+from alert_triage.generator import canonical_json, compatible_overlays, generate, screen_of
+
+ALL = range(4096)
+
+
+@pytest.fixture(scope="module")
+def pairs():
+    out = []
+    for s in ALL:
+        base = generate(s, overlay=None)
+        if base["template"].startswith("S"):
+            continue
+        out.append((s, base))
+    return out
+
+
+def test_overlay_never_touches_the_screen(pairs) -> None:
+    for seed, base in pairs:
+        before = canonical_json(screen_of(base))
+        for kind in compatible_overlays(base):
+            overlaid = generate(seed, overlay=kind)
+            assert overlaid["label"] == "suspicious" and overlaid["overlay"] == kind
+            assert canonical_json(screen_of(overlaid)) == before, (seed, kind)
+
+
+def test_the_seed_overlay_is_one_of_the_compatible_kinds(pairs) -> None:
+    for seed, base in pairs:
+        w = generate(seed)
+        if w["overlay"]:
+            assert w["overlay"] in compatible_overlays(base)
+
+
+def _features(w) -> list[float]:
+    c = w["customer"]
+    screen = screen_of(w)
+    trig = screen["triggering_transactions"]
+    amounts = [t["amount"] for t in trig]
+    return [
+        float(c["prior_alerts"]),
+        float(c["prior_sars"]),
+        float(c["pep"]),
+        float(sum(1 for cp in screen["screen_counterparties"] if cp["direction"] == "out")),
+        float(sum(1 for a in amounts if a % 100000 == 0)) / max(1, len(amounts)),
+        float(len(trig)),
+        float(len({t["branch"] for t in trig if t["branch"]})),
+        float(sum(r["wires_out"] > 0 for r in w["summary"])),
+        float(screen["alert"]["rule"] == "R-STR-01"),
+        float(c["expected_monthly_cash"] > 0),
+    ]
+
+
+def test_screen_features_do_not_separate_hidden_from_benign() -> None:
+    """Per-feature z-test between the two populations; nothing beyond 4 sigma."""
+    benign, hidden = [], []
+    for s in ALL:
+        w = generate(s)
+        if w["tier"] == "benign":
+            benign.append(_features(w))
+        elif w["tier"] == "hidden":
+            hidden.append(_features(w))
+    assert len(hidden) > 400
+    for j in range(len(benign[0])):
+        b = [f[j] for f in benign]
+        h = [f[j] for f in hidden]
+        mb, mh = sum(b) / len(b), sum(h) / len(h)
+        vb = sum((x - mb) ** 2 for x in b) / max(1, len(b) - 1)
+        vh = sum((x - mh) ** 2 for x in h) / max(1, len(h) - 1)
+        se = math.sqrt(vb / len(b) + vh / len(h)) or 1e-9
+        z = abs(mb - mh) / se
+        assert z < 4.0, f"feature {j}: benign {mb:.3f} vs hidden {mh:.3f}, z={z:.1f}"