Alert Triage: environment #2, built end to end by the pipeline
The first environment shipped through .claude/workflows/new-environment.js: specification, three adversarial reviews (all 'fixable', none fatal), the Python environment, the TypeScript port, captured rollouts, and the demo page. Eleven agents, no errors. The proof that the platform scales is one line long. Alert Triage has a completely different shape from Word Five — JSON actions, priced lookups, an analyst screen instead of a grid — and the only change under src/components/demo/ is a comment edit, because the isolation lint refused the word "wordle" there. Zero shell code changed. 415 contract checks now pass against two demos, up from 206 against one. The environment is honest by construction. Every alert is synthetic, generated from the seed, and the banner saying so sits inside the board surface. Two of the eleven scenario templates are hidden-suspicious: generated by the same code as their benign twin with the signal overlaid only in lookup data, so the free screen is identically distributed and a screen-only policy STRUCTURALLY cannot tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden seeds. That is the counterweight made real rather than asserted. Twelve policies, thirteen ladder assertions, a genuine three-way trade: fast 0.846 wins hours (0.85), misses every hidden case targeted 0.894 wins the shipped total thorough 0.820 wins evidence (1.00), spends 2.9 hours None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at delta 0, and conformance gated on world + scorer + protocol so the browser shows the same alert for ?seed= that Python generated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
This commit is contained in:
karti-ai
@@ -0,0 +1,177 @@
|
||||
{
|
||||
"runId": "base-off-s4",
|
||||
"seed": 4,
|
||||
"model": "brain-qwen38-dspark",
|
||||
"capturedAt": "2026-08-28",
|
||||
"rewards": {
|
||||
"caught": 1.0,
|
||||
"hours": 0.22,
|
||||
"evidence": 0.7058823529411765
|
||||
},
|
||||
"metrics": {
|
||||
"hours_spent": 2.5,
|
||||
"reference_hours": 0.55,
|
||||
"lookups": 5.0,
|
||||
"turns_used": 6.0,
|
||||
"rejected_replies": 0.0,
|
||||
"false_escalation": 0.0,
|
||||
"missed": 0.0,
|
||||
"typology_match": 1.0
|
||||
},
|
||||
"truncated": false,
|
||||
"outcome": "solved",
|
||||
"info": {
|
||||
"label": "suspicious",
|
||||
"template": "S1",
|
||||
"overlay": null,
|
||||
"typology": "STRUCTURING",
|
||||
"planted": [
|
||||
[
|
||||
"T-46",
|
||||
"T-47",
|
||||
"T-49",
|
||||
"T-50",
|
||||
"T-52",
|
||||
"T-53",
|
||||
"T-54",
|
||||
"T-56"
|
||||
]
|
||||
],
|
||||
"reference_minutes": 33,
|
||||
"reference_policy": "fast",
|
||||
"held_out": false,
|
||||
"tier": "visible",
|
||||
"screen": "ALERT A-8199 rule R-STR-01 \u2014 Three or more cash deposits just under the reporting threshold within ten days\n fired 2026-06-14\n triggering transactions:\n T-49 2026-06-05 cash_in 9,320.00 USD branch BR-027\n T-50 2026-06-06 cash_in 9,820.00 USD branch BR-031\n T-52 2026-06-07 cash_in 9,880.00 USD branch BR-068\n T-53 2026-06-09 cash_in 9,730.00 USD branch BR-027\n T-54 2026-06-10 cash_in 9,680.00 USD branch BR-031\n T-56 2026-06-11 cash_in 9,850.00 USD branch BR-068\nKYC FILE C-5480 Bilal Quintero\n kyc.account_opened = 2017-03-29\n kyc.country = United States\n kyc.customer_type = individual\n kyc.expected_activity = Salaried individual; payroll deposits and routine spending.\n kyc.expected_monthly_cash = 100.00 USD\n kyc.kyc_refreshed = 2026-05-04\n kyc.occupation = delivery driver\n kyc.pep = False\n kyc.prior_alerts = 3\n kyc.prior_sars = 0\n kyc.remittance_declared = none declared\n kyc.stated_income = 45,000.00 USD\nTWELVE-MONTH SUMMARY (USD)\n month cash in cash out wires in wires out ach in ach out txns\n 2025-07 0 220 0 0 3,410 0 3\n 2025-08 0 480 0 0 3,382 0 3\n 2025-09 0 570 0 0 3,444 0 4\n 2025-10 0 570 0 0 3,454 0 5\n 2025-11 0 660 0 0 3,460 0 4\n 2025-12 0 550 0 0 3,403 0 3\n 2026-01 0 680 0 0 3,351 0 5\n 2026-02 0 490 0 0 3,445 0 4\n 2026-03 0 540 0 0 3,433 0 4\n 2026-04 0 430 0 0 3,387 0 4\n 2026-05 0 660 0 0 3,411 0 5\n 2026-06 77,210 420 0 66,400 3,348 0 13\nCOUNTERPARTIES on screen (top by twelve-month volume):\n CP-2 out 1 txns 66,400.00 USD\n CP-1 in 12 txns 40,928.00 USD\nReporting threshold in this jurisdiction: 10,000.00 USD\nSYNTHETIC DATA \u2014 no real customer, account or transaction.\n\nHours spent so far: 0.50. You have 8 replies left."
|
||||
},
|
||||
"turns": [
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"history\",\"month\":\"2026-06\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 1588,
|
||||
"completionTokens": 20,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 1398,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "history",
|
||||
"month": "2026-06"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "HISTORY 2026-06: 13 transactions\n T-45 2026-06-02 ach_in 3,348.00 USD CP-1\n T-46 2026-06-02 cash_in 9,400.00 USD branch BR-031\n T-47 2026-06-03 cash_in 9,530.00 USD branch BR-068\n T-48 2026-06-04 cash_out 110.00 USD branch BR-027\n T-49 2026-06-05 cash_in 9,320.00 USD branch BR-027\n T-50 2026-06-06 cash_in 9,820.00 USD branch BR-031\n T-51 2026-06-07 cash_out 150.00 USD branch BR-027\n T-52 2026-06-07 cash_in 9,880.00 USD branch BR-068\n T-53 2026-06-09 cash_in 9,730.00 USD branch BR-027\n T-54 2026-06-10 cash_in 9,680.00 USD branch BR-031\n T-55 2026-06-11 cash_out 160.00 USD branch BR-068\n T-56 2026-06-11 cash_in 9,850.00 USD branch BR-068\n T-57 2026-06-13 wire_out 66,400.00 USD CP-2\n\nHours spent so far: 0.95. You have 7 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"counterparty\",\"id\":\"CP-2\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 2151,
|
||||
"completionTokens": 17,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 1282,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "counterparty",
|
||||
"id": "CP-2"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "COUNTERPARTY CP-2 Redfern Holdings Co.\n type business; jurisdiction United States; first seen 2026-06-13\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-57\n\nHours spent so far: 1.40. You have 6 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"counterparty\",\"id\":\"CP-1\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 2258,
|
||||
"completionTokens": 17,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 677,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "counterparty",
|
||||
"id": "CP-1"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "COUNTERPARTY CP-1 Stonebridge Diner Group\n type business; jurisdiction United States; first seen 2025-07-03\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-1, T-5, T-7, T-11, T-16, T-21, T-23, T-28, T-32, T-36, T-40, T-45\n\nHours spent so far: 1.85. You have 5 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"prior_alerts\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 2417,
|
||||
"completionTokens": 12,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 563,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "prior_alerts"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "PRIOR ALERTS:\n A-1684 R-STR-01 fired 2025-01-17 closed after 5 day(s) by Andre Abara\n Closed: deposits are documented daily takings; amounts vary with trading days; within kyc.expected_monthly_cash.\n A-3624 R-STR-01 fired 2025-03-06 closed after 8 day(s) by Bilal Laurent\n Closed: deposits are documented daily takings; amounts vary with trading days; within kyc.expected_monthly_cash.\n A-2017 R-CASH-03 fired 2025-05-09 closed after 7 day(s) by Sofia Grimaldi\n Closed: seasonal peak consistent with kyc.expected_activity; volume within stated expectation.\n\nHours spent so far: 2.30. You have 4 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"lookup\",\"what\":\"documents\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 2655,
|
||||
"completionTokens": 10,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 640,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "lookup",
|
||||
"what": "documents"
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "DOCUMENTS ON FILE:\n doc.D-c608 [identity] Government-issued identity document, verified at onboarding\n\nHours spent so far: 2.45. You have 3 replies left."
|
||||
}
|
||||
},
|
||||
{
|
||||
"reply": "{\"action\":\"escalate\",\"typology\":\"STRUCTURING\",\"cites\":[\"T-49\",\"T-50\",\"T-52\",\"T-53\",\"T-54\",\"T-56\",\"kyc.expected_monthly_cash\",\"CP-2\",\"T-57\"],\"narrative\":\"Six cash deposits of 9,320\u20139,880 USD each over 06-05 to 06-11, all just under the 10,000 reporting threshold, at three rotating branches (BR-027, BR-031, BR-068). This is a stark departure from the customer's profile: a salaried delivery driver with kyc.expected_monthly_cash of 100 USD and a 12-month history of zero cash-in until June 2026, when 77,210 USD in cash appeared. On 06-13 the customer wired the full 66,400 USD to CP-2 Redfern Holdings Co., a business first seen on that same date \u2014 a rapid movement of the aggregated cash. The pattern of just-under-threshold deposits across multiple branches followed by a single outflow to a newly-seen counterparty is classic structuring/layering, not documented daily takings. Prior R-STR-01 closures (A-1684, A-3624) referenced amounts within expected monthly cash, which does not apply here given the 770x jump and the immediate wire-out. Escalating for investigation.\"}",
|
||||
"reasoning": null,
|
||||
"call": {
|
||||
"promptTokens": 2725,
|
||||
"completionTokens": 338,
|
||||
"reasoningTokens": null,
|
||||
"durationMs": 15080,
|
||||
"finishReason": "stop"
|
||||
},
|
||||
"info": {
|
||||
"action": {
|
||||
"action": "escalate",
|
||||
"typology": "STRUCTURING",
|
||||
"cites": [
|
||||
"T-49",
|
||||
"T-50",
|
||||
"T-52",
|
||||
"T-53",
|
||||
"T-54",
|
||||
"T-56",
|
||||
"kyc.expected_monthly_cash",
|
||||
"CP-2",
|
||||
"T-57"
|
||||
],
|
||||
"narrative": "Six cash deposits of 9,320\u20139,880 USD each over 06-05 to 06-11, all just under the 10,000 reporting threshold, at three rotating branches (BR-027, BR-031, BR-068). This is a stark departure from the customer's profile: a salaried delivery driver with kyc.expected_monthly_cash of 100 USD and a 12-month history of zero cash-in until June 2026, when 77,210 USD in cash appeared. On 06-13 the customer wired the full 66,400 USD to CP-2 Redfern Holdings Co., a business first seen on that same date \u2014 a rapid movement of the aggregated cash. The pattern of just-under-threshold deposits across multiple branches followed by a single outflow to a newly-seen counterparty is classic structuring/layering, not documented daily takings. Prior R-STR-01 closures (A-1684, A-3624) referenced amounts within expected monthly cash, which does not apply here given the 770x jump and the immediate wire-out. Escalating for investigation."
|
||||
},
|
||||
"rejection": null,
|
||||
"observation": "Disposition recorded. The episode is over."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user