Alert Triage: environment #2, built end to end by the pipeline
The first environment shipped through .claude/workflows/new-environment.js: specification, three adversarial reviews (all 'fixable', none fatal), the Python environment, the TypeScript port, captured rollouts, and the demo page. Eleven agents, no errors. The proof that the platform scales is one line long. Alert Triage has a completely different shape from Word Five — JSON actions, priced lookups, an analyst screen instead of a grid — and the only change under src/components/demo/ is a comment edit, because the isolation lint refused the word "wordle" there. Zero shell code changed. 415 contract checks now pass against two demos, up from 206 against one. The environment is honest by construction. Every alert is synthetic, generated from the seed, and the banner saying so sits inside the board surface. Two of the eleven scenario templates are hidden-suspicious: generated by the same code as their benign twin with the signal overlaid only in lookup data, so the free screen is identically distributed and a screen-only policy STRUCTURALLY cannot tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden seeds. That is the counterweight made real rather than asserted. Twelve policies, thirteen ladder assertions, a genuine three-way trade: fast 0.846 wins hours (0.85), misses every hidden case targeted 0.894 wins the shipped total thorough 0.820 wins evidence (1.00), spends 2.9 hours None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at delta 0, and conformance gated on world + scorer + protocol so the browser shows the same alert for ?seed= that Python generated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
@@ -0,0 +1,91 @@
|
||||
{
|
||||
"runId": "base-off-s3",
|
||||
"seed": 3,
|
||||
"model": "brain-qwen38-dspark",
|
||||
"capturedAt": "2026-08-28",
|
||||
"rewards": {
|
||||
"caught": 1.0,
|
||||
"hours": 0.55,
|
||||
"evidence": 0.3333333333333333
|
||||
},
|
||||
"metrics": {
|
||||
"hours_spent": 1.0,
|
||||
"reference_hours": 0.55,
|
||||
"lookups": 1.0,
|
||||
"turns_used": 2.0,
|
||||
"rejected_replies": 0.0,
|
||||
"false_escalation": 0.0,
|
||||
"missed": 0.0,
|
||||
"typology_match": null
|
||||
},
|
||||
"truncated": false,
|
||||
"outcome": "solved",
|
||||
"info": {
|
||||
"label": "benign",
|
||||
"template": "B4",
|
||||
"overlay": null,
|
||||
"typology": null,
|
||||
"planted": [
|
||||
[
|
||||
"kyc.expected_monthly_cash"
|
||||
],
|
||||
[
|
||||
"kyc.expected_monthly_cash",
|
||||
"kyc.business_type"
|
||||
]
|
||||
],
|
||||
"reference_minutes": 33,
|
||||
"reference_policy": "fast",
|
||||
"held_out": false,
|
||||
"tier": "benign",
|
||||
"screen": "ALERT A-9453 rule R-STR-01 \u2014 Three or more cash deposits just under the reporting threshold within ten days\n fired 2026-04-23\n triggering transactions:\n T-200 2026-04-14 cash_in 9,770.00 USD branch BR-089\n T-201 2026-04-15 cash_in 9,430.00 USD branch BR-014\n T-206 2026-04-22 cash_in 8,640.00 USD branch BR-089\nKYC FILE C-4718 Lakeside Kitchen LLC\n kyc.account_opened = 2016-10-03\n kyc.business_type = pizzeria\n kyc.country = United States\n kyc.customer_type = business\n kyc.expected_activity = Cash-intensive food service; daily takings deposited most trading days.\n kyc.expected_monthly_cash = 97,000.00 USD\n kyc.kyc_refreshed = 2025-07-13\n kyc.pep = False\n kyc.prior_alerts = 0\n kyc.prior_sars = 0\n kyc.remittance_declared = none declared\n kyc.stated_income = 588,000.00 USD\nTWELVE-MONTH SUMMARY (USD)\n month cash in cash out wires in wires out ach in ach out txns\n 2025-05 52,750 0 0 0 0 16,816 17\n 2025-06 45,240 0 0 0 0 16,317 17\n 2025-07 49,740 0 0 0 0 16,493 17\n 2025-08 40,470 0 0 0 0 16,944 17\n 2025-09 49,720 0 0 0 0 18,522 19\n 2025-10 44,630 0 0 0 0 17,199 16\n 2025-11 49,060 0 0 0 0 18,992 16\n 2025-12 52,570 0 0 0 0 17,992 19\n 2026-01 50,320 0 0 0 0 15,038 17\n 2026-02 51,710 0 0 0 0 17,826 16\n 2026-03 52,240 0 0 0 0 16,934 17\n 2026-04 78,100 0 0 0 0 18,673 18\nCOUNTERPARTIES on screen (top by twelve-month volume):\n CP-1 out 48 txns 132,956.60 USD\n CP-2 out 12 txns 74,793.60 USD\nReporting threshold in this jurisdiction: 10,000.00 USD\nSYNTHETIC DATA \u2014 no real customer, account or transaction.\n\nHours spent so far: 0.50. You have 8 replies left."
|
||||
},
|
||||
"turns": [
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"prior_alerts\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 1531,
|
||||
"completionTokens": 12,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 2064,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "prior_alerts"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "PRIOR ALERTS: none on file.\n\nHours spent so far: 0.95. You have 7 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"close\",\"reason\":\"CONSISTENT_WITH_PROFILE\",\"cites\":[\"T-200\",\"T-201\",\"T-206\",\"kyc.expected_activity\",\"kyc.expected_monthly_cash\"],\"note\":\"Lakeside Kitchen LLC is a cash-intensive pizzeria with a documented expectation of ~$97k/month in cash deposits. The three triggering deposits ($9,770, $9,430, $8,640) are each just under the $10,000 reporting threshold but are consistent with the customer's normal pattern of daily takings deposited across multiple trading days. April's total cash-in of $78,100 is within the expected range and the account has no prior alerts, no SARs, and no PEP flags. The rule fired because three sub-threshold deposits occurred within ten days, but this is a natural consequence of a high-volume cash business depositing daily takings in amounts that individually stay below the CTR threshold. No structuring indicators (e.g., sudden spike, new counterparty, unusual timing) are present.\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 1584,
|
||||
"completionTokens": 233,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 9906,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "close",
|
||||
"reason": "CONSISTENT_WITH_PROFILE",
|
||||
"cites": [
|
||||
"T-200",
|
||||
"T-201",
|
||||
"T-206",
|
||||
"kyc.expected_activity",
|
||||
"kyc.expected_monthly_cash"
|
||||
],
|
||||
"note": "Lakeside Kitchen LLC is a cash-intensive pizzeria with a documented expectation of ~$97k/month in cash deposits. The three triggering deposits ($9,770, $9,430, $8,640) are each just under the $10,000 reporting threshold but are consistent with the customer's normal pattern of daily takings deposited across multiple trading days. April's total cash-in of $78,100 is within the expected range and the account has no prior alerts, no SARs, and no PEP flags. The rule fired because three sub-threshold deposits occurred within ten days, but this is a natural consequence of a high-volume cash business depositing daily takings in amounts that individually stay below the CTR threshold. No structuring indicators (e.g., sudden spike, new counterparty, unusual timing) are present."
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "Disposition recorded. The episode is over."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user