PIG-Demo/envs/alert_triage

alert-triage — the environment

One transaction-monitoring alert on one synthetic customer. The agent reads the analyst's screen, may pay analyst-hours for lookups, and ends the episode by closing the alert with a cited rationale or escalating it with a typology and the transactions that constitute it. The grader compares the disposition and the citations to what the generator planted and charges the hours. No model, human or prose is read anywhere in it.

SPEC.md is the specification as written and reviewed before this code existed. This file records what was implemented, including every change the reviewers demanded; where the two differ, this file is what ships.

uv run python envs/probe.py --taskset alert-triage   # the ladder
uv run pytest envs/alert_triage/tests                 # ~30 s; hashes 4096 worlds
uv run python -m alert_triage.conformance             # the three digests

Layout

file	what
alert_triage/rng.py	FNV-1a seeding, xorshift32, civil dates. No platform RNG, no platform date.
alert_triage/generator.py	Templates, overlays, the canonical world, reference_minutes, digest (a).
alert_triage/engine.py	Screen, lookups, dispositions, shown-id discipline, the minute meter.
alert_triage/protocol.py	The pinned parse rules, the system prompt, text renderers.
alert_triage/reward.py	The three components. The # region: pig-demo/reward block is what the page quotes.
alert_triage/policies.py	fast, targeted, thorough and the seven cheats. Read only the engine's view.
alert_triage/ladder.py	The probe rungs and their thirteen assertions.
alert_triage/conformance.py	Digests (b) and (c); writes CONFORMANCE.txt.
alert_triage/taskset.py	play_episode, replay, and the optional verifiers Env.
conformance/replies.json	The reply corpus digest (c) runs.
names/	The synthetic name and word lists, with PROVENANCE.md.

The world, and what the reviewers changed

Templates. Six benign builders (B1 seasonal cash, B2 processor settlement, B3 property sale, B4 cash restaurant, B5 family remittance, B6 inheritance) and three visible-suspicious ones (S1 structuring, S3 money mule, S6 smurfing over time). The hidden tier is no longer two fixed twins. A hidden alert is any benign builder run to completion — screen frozen — with one of three overlays applied to lookup data only:

overlay	where it lives	compatible with	planted set
funnel	the counterparty lookup: the top outbound counterparty on screen is shared with 5–14 other alerted accounts	every benign template	CP-x + its transactions; alternate: the transactions
serial_closer	the prior-alerts lookup: every prior alert on the same rule, closed within a day on one identical boilerplate, by different analysts	screens showing ≥ 3 prior alerts and no prior report	the A- ids + the triggering transactions; alternate: the A- ids
doc_mismatch	the documents lookup against the counterparty lookup: the document names a party; the funds came from someone else	B2, B3, B6	the document + CP-x + its triggering credits; alternate: the document + CP-x

The tier is drawn before the template and the screen, and the prior-alert count on the KYC block is drawn before any template code runs, so the screen distribution is the same for hidden and benign by construction, not by effort. tests/test_triage_screen_leak.py asserts the screen is byte-identical with every compatible overlay on or off for every seed, and that no screen feature separates the populations beyond four sigma.

This answers three reviewer findings at once: every rule id now fires on both labels (R-WIRE-04 and R-DORM-05 carry funnel and serial-closer twins); each hidden signal lands on several templates so there is no template→lookup mapping to memorise; and no screen-identifiable segment carries a suspicious prior above the overall 15 / (55 + 15) ≈ 21 %.

Removed: B4's standalone PREVIOUSLY_REVIEWED alternate. A prior closure never resolves an alert on its own. The reason stays in the enum as legal and never planted, so its use renders as a diagnostic.

Documents carry per-world ids (doc.D-<4 hex>), and the engine rejects any citation of an id it has not rendered. fast therefore pulls documents on the templates a document explains, and the reference on those seeds is 0.70 h, not 0.55 h.

The held-out slice is fnv1a(str(seed)) % 8 == 7, about 12.5 % of seeds. It is generated in a fictional currency, KRN, with a fictional 20,000 KRN reporting threshold (amounts scale ×2), rule ids renamed TM-A1..A5, the KYC block before the alert and the summary newest-first. It is never probed and never used to tune anything; the capture stage runs it once, after the arms are frozen, for the side-by-side numbers on the page. No claim about any real jurisdiction's threshold is made anywhere.

Canonical world

Sorted keys, no whitespace, null for absent. Every amount is an integer in minor units; every date is a YYYY-MM-DD string produced by rng.iso_date from a day ordinal. reference_minutes and reference_policy are part of the world and therefore of digest (a): the browser recomputes them by running the ported policies rather than trusting the recorded number, and a mismatch is a verification failure, not an "unverifiable".

Draw order is the contract. Reading generator.py top to bottom in the order the builder calls things is the port; the port must not reorder a single rng call.

The action set and the pinned parse rules

One JSON object per reply. A ```json fence anywhere wins; otherwise the first string-aware balanced {…} span; an unparseable or non-object candidate is a rejection with no further scanning. Enums exact-case. Cites deduplicated in order, matched exact-case against the shown set. The twelve-month window is the twelve calendar months ending in the fire month inclusive. A repeated lookup is rejected. Only accepted lookups add their price; every turn, accepted or rejected, costs TURN_MINUTES.

aborted is exactly MAX_TURNS = 8 turns spent with no accepted disposition and scores 0.000. A run that stops earlier with no disposition is truncated and every component is null — rendered "not scored", never zero. Correct disposition → solved, wrong → failed.

Hours

Integers in minutes, shown as hours. The generator's config, not measurements — nobody timed an analyst:

constant	minutes	hours
SCREEN_MINUTES	30	0.50
TURN_MINUTES	3	0.05
LOOKUP_MINUTES	24	0.40
DOC_MINUTES	6	0.10

LOOKUP_MINUTES was raised from the spec's 0.25 h to 0.40 h so that the hours spent investigating clean screens genuinely exceed the hours a screen-only analyst loses to misses — the trade-off the reviewers said was manufactured in the spec is real at these constants and the ladder measures it (rung 8).

The reward

key	weight	role	definition
caught	0.55	counterweight	1.0 on a correct call; 0.0 on a miss or no disposition; 0.5 on a false escalation ("filed on a clean customer").
hours	0.25	objective	min(1, reference / spent) if correct, else 0. null if no shipped policy got the seed right (CI asserts this never happens). Capped: faster than the reference earns 1.0, not more.
evidence	0.20	objective	F1 of the cites against the best planted alternate; 0 if the call was wrong.

The 0.5 credit is the reviewers' fix for segment-blind defensive filing: with it, a blind escalation only beats a well-cited close when P(benign) is below 0.52 (reward.escalate_break_even(), printed by the probe), and no benign-looking screen segment is that suspicious.

The ladder, measured

uv run python envs/probe.py --taskset alert-triage, 894 main-slice seeds (seeds 0–1023 minus the held-out bucket). These are the numbers the page may quote; the spec's ~0.86 vs ~0.85 text may not.

policy                 caught       hours    evidence     TOTAL  hours@0.45
inaction               0.0000      0.0000      0.0000    0.0000      0.0000
malformed              0.0000      0.0000      0.0000    0.0000      0.0000
close_blind            0.5705      0.5705      0.0000    0.4564      0.4868
close_cites_all        0.5705      0.5705      0.0815    0.4727      0.4988
cite_unseen            0.5705      0.5473      0.0000    0.4506      0.4764
escalate_blind         0.7148      0.4295      0.3006    0.5606      0.5257
fast_defensive         0.7148      0.4295      0.3006    0.5606      0.5257
rule_table             0.7550      0.5593      0.3680    0.6287      0.6102
fast                   0.8535      0.8535      0.8159    0.8460      0.8480
targeted               1.0000      0.6039      0.9625    0.8935      0.8163
thorough               1.0000      0.2784      1.0000    0.8196      0.6753
thorough_wasteful      1.0000      0.2412      1.0000    0.8103      0.6585

Mean hours: fast 0.60, targeted 1.63, thorough 2.92. Caught on hidden seeds: fast 0.0, targeted 1.0, thorough 1.0.

The three shipped analysts, and the claim the page can make:

• fast decides from the screen (plus documents where a document should explain the alert). Wins hours outright. Misses every hidden seed, and only those.
• targeted is fast on a screen that shows a typology, and otherwise pulls exactly the three lookups that could carry a hidden signal. Catches everything. Wins at shipped weights, 0.894 to 0.846. It defines reference_minutes on hidden seeds.
• thorough always runs the full procedure, including two months of itemised history, and cites every constituting transaction it saw: evidence 1.000 exactly, and the lowest hours.

Move hours to 0.45 (the other two weights keeping their ratio) and fast wins, 0.848 to 0.816. Nothing else on the page depends on the direction; rung 8 asserts both margins are at least 0.02.

Note the weakest honest reading: thorough — which gets every call and every citation right — scores below fast at shipped weights, because spending five times the hours on the 85 % of alerts the screen settles costs more than the misses. That is the environment's argument, not a defect.

Rungs a reviewer named

rung	the cheat	assertion
fast_defensive	file blind on every benign-looking screen	total < every good policy
cite_unseen	close citing a document it never opened	rejected on every seed; total < fast
rule_table	close blind on wire/dormancy rules, file blind elsewhere	total < every good policy
close_cites_all	cite every id on the screen	evidence < ½ its caught share; < fast's evidence
thorough_wasteful	pull a month that carries nothing	total < thorough, same caught and evidence

The reference analysts see only the view

policies.py functions take Engine.view() and nothing else — the screen, the lookups so far, the turn and minute counters. Tests assert the signature, that the view carries none of label, typology, planted, overlay, tier, template, reference_*, and that every policy emits identical replies against a world with those fields deleted.

Conformance

Three digests in CONFORMANCE.txt, each a CI gate for the TypeScript port:

1. world — canonical world JSON for seeds 0–4095, reference included. The port must carry the generator, not only the scorer: the Play tab and ?seed= need the same alert on both sides.
2. scorer — score_exact() over a fixed grid of episodes. Exact fractions, so no float formatting is hashed.
3. protocol — parse + engine over conformance/replies.json on fixed worlds: accepted actions, rejection reasons, and the minute meter after every turn. This is the digest wordle does not need.

What this is easier than, for the page

1. Every alert has a knowable answer; real SAR decisions are judgements, and a filed report is not a proven crime. Nine builders and three overlays stand in for a typology space in the hundreds.
2. The 55/30/15 mix is deliberately not the real base rate. The great majority of production alerts close as false positives — no figure is quoted here because none was measured by us. "Hours per thousand alerts" on the page is Σ hours_spent over recorded seeds inside this environment, with the constants above beside it.
3. The hour constants are assumptions.
4. The lookups are idealised. Each returns the decisive fact in one call: a counterparty's other-alerted-account count, a prior alert's analyst and rationale, a document's named party. In production those signals are spread across systems and often absent. This environment tests whether the agent asks; it does not test whether the answer exists.
5. The narrative prose is unscored.
6. The adversary is static: the launderer shaped the data to pass the screen and the held-out slice moves the threshold once. Nothing responds to the agent's policy. The demo must claim exactly that and no more, and the wordle limit that names this demo as its answer should be softened to "nobody designed the data to pass the screen".
7. caught is 1.0 on every benign alert by construction; the counterweight is argued from the ladder over seeds, never from one run.

For the next stages

• Port: rng.py, generator.py, engine.py, protocol.py, reward.py, policies.py (fast, targeted, thorough — verify recomputes reference_minutes). Gate on all three digests.
• Capture: taskset.play_episode(seed, respond) is the loop; taskset.replay(seed, replies) re-scores a fixture from its reply strings. Scripted arms: fast, targeted, thorough, and at least one cheat. Capture the held-out bucket exactly once, after the arms are frozen.
• Demo: the board must carry "Synthetic data — no real customer, account or transaction"; the hour constants as config; the narrative marked "not scored"; the label reveal only after a disposition. The vertical card's reward line must be amended to say "hours as a ratio against a reference analyst policy", not "hours per thousand alerts".
• SPEC.md is stage 1's document and was not edited; §3, §4, §6, §7 and §8 are superseded by this file where they differ.