karti-ai
2dfa96939e
The first environment shipped through .claude/workflows/new-environment.js: specification, three adversarial reviews (all 'fixable', none fatal), the Python environment, the TypeScript port, captured rollouts, and the demo page. Eleven agents, no errors. The proof that the platform scales is one line long. Alert Triage has a completely different shape from Word Five — JSON actions, priced lookups, an analyst screen instead of a grid — and the only change under src/components/demo/ is a comment edit, because the isolation lint refused the word "wordle" there. Zero shell code changed. 415 contract checks now pass against two demos, up from 206 against one. The environment is honest by construction. Every alert is synthetic, generated from the seed, and the banner saying so sits inside the board surface. Two of the eleven scenario templates are hidden-suspicious: generated by the same code as their benign twin with the signal overlaid only in lookup data, so the free screen is identically distributed and a screen-only policy STRUCTURALLY cannot tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden seeds. That is the counterweight made real rather than asserted. Twelve policies, thirteen ladder assertions, a genuine three-way trade: fast 0.846 wins hours (0.85), misses every hidden case targeted 0.894 wins the shipped total thorough 0.820 wins evidence (1.00), spends 2.9 hours None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at delta 0, and conformance gated on world + scorer + protocol so the browser shows the same alert for ?seed= that Python generated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
187 lines
9.2 KiB
JSON
187 lines
9.2 KiB
JSON
{
|
|
"runId": "thorough-s5",
|
|
"seed": 5,
|
|
"model": "thorough-analyst",
|
|
"capturedAt": "2026-08-28",
|
|
"rewards": {
|
|
"caught": 1.0,
|
|
"hours": 0.23728813559322035,
|
|
"evidence": 1.0
|
|
},
|
|
"metrics": {
|
|
"hours_spent": 2.95,
|
|
"reference_hours": 0.7,
|
|
"lookups": 6.0,
|
|
"turns_used": 7.0,
|
|
"rejected_replies": 0.0,
|
|
"false_escalation": 0.0,
|
|
"missed": 0.0,
|
|
"typology_match": null
|
|
},
|
|
"truncated": false,
|
|
"outcome": "solved",
|
|
"info": {
|
|
"label": "benign",
|
|
"template": "B3",
|
|
"overlay": null,
|
|
"typology": null,
|
|
"planted": [
|
|
[
|
|
"doc.D-8578"
|
|
],
|
|
[
|
|
"doc.D-8578",
|
|
"kyc.occupation"
|
|
]
|
|
],
|
|
"reference_minutes": 42,
|
|
"reference_policy": "fast",
|
|
"held_out": false,
|
|
"tier": "benign",
|
|
"screen": "ALERT A-6081 rule R-VEL-02 \u2014 Funds received and moved out again within 48 hours\n fired 2026-03-22\n triggering transactions:\n T-52 2026-03-18 wire_in 252,000.00 USD CP-2\n T-54 2026-03-19 wire_out 219,200.00 USD CP-3\nKYC FILE C-7077 Andre Nakamura\n kyc.account_opened = 2022-01-15\n kyc.country = United States\n kyc.customer_type = individual\n kyc.expected_activity = Salaried individual; payroll deposits and routine spending.\n kyc.expected_monthly_cash = 0.00 USD\n kyc.kyc_refreshed = 2025-04-22\n kyc.occupation = teacher\n kyc.pep = False\n kyc.prior_alerts = 3\n kyc.prior_sars = 0\n kyc.remittance_declared = none declared\n kyc.stated_income = 106,000.00 USD\nTWELVE-MONTH SUMMARY (USD)\n month cash in cash out wires in wires out ach in ach out txns\n 2025-04 0 590 0 0 8,866 0 4\n 2025-05 0 820 0 0 8,921 0 4\n 2025-06 0 920 0 0 8,952 0 5\n 2025-07 0 680 0 0 8,870 0 3\n 2025-08 0 1,140 0 0 8,906 0 5\n 2025-09 0 390 0 0 8,870 0 4\n 2025-10 0 850 0 0 8,922 0 4\n 2025-11 0 1,020 0 0 8,916 0 5\n 2025-12 0 950 0 0 8,901 0 5\n 2026-01 0 880 0 0 8,854 0 4\n 2026-02 0 1,170 0 0 8,916 0 5\n 2026-03 0 720 252,000 219,200 8,956 0 6\nCOUNTERPARTIES on screen (top by twelve-month volume):\n CP-2 in 1 txns 252,000.00 USD\n CP-3 out 1 txns 219,200.00 USD\nReporting threshold in this jurisdiction: 10,000.00 USD\nSYNTHETIC DATA \u2014 no real customer, account or transaction.\n\nHours spent so far: 0.50. You have 8 replies left."
|
|
},
|
|
"turns": [
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"documents\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "documents"
|
|
},
|
|
"rejection": null,
|
|
"observation": "DOCUMENTS ON FILE:\n doc.D-5045 [identity] Government-issued identity document, verified at onboarding\n doc.D-8578 [closing_statement] Closing statement \u2014 sale of 266 Birch Avenue; net proceeds to seller disbursed by Iwata Title & Escrow.\n\nHours spent so far: 0.65. You have 7 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"prior_alerts\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "prior_alerts"
|
|
},
|
|
"rejection": null,
|
|
"observation": "PRIOR ALERTS:\n A-4824 R-VEL-02 fired 2025-03-19 closed after 4 day(s) by Jamal Whitlock\n Closed: pass-through matches the settlement terms in the agreement on file; counterparties verified.\n A-1410 R-WIRE-04 fired 2025-04-24 closed after 2 day(s) by Lars Farrow\n Closed: destination and amount match the remittance declared at onboarding; within stated income.\n A-2317 R-CASH-03 fired 2025-09-02 closed after 6 day(s) by Chidi Zapata\n Closed: seasonal peak consistent with kyc.expected_activity; volume within stated expectation.\n\nHours spent so far: 1.10. You have 6 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"counterparty\", \"id\": \"CP-2\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-2"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-2 Iwata Title & Escrow\n type title_company; jurisdiction United States; first seen 2026-03-18\n other alerted accounts sharing this counterparty in twelve months: 1\n transactions: T-52\n\nHours spent so far: 1.55. You have 5 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"counterparty\", \"id\": \"CP-3\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-3"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-3 Harbor Securities\n type brokerage; jurisdiction United States; first seen 2026-03-19\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-54\n\nHours spent so far: 2.00. You have 4 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"history\", \"month\": \"2026-03\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "history",
|
|
"month": "2026-03"
|
|
},
|
|
"rejection": null,
|
|
"observation": "HISTORY 2026-03: 6 transactions\n T-49 2026-03-02 cash_out 160.00 USD branch BR-052\n T-50 2026-03-04 cash_out 320.00 USD branch BR-052\n T-51 2026-03-05 ach_in 8,956.00 USD CP-1\n T-52 2026-03-18 wire_in 252,000.00 USD CP-2\n T-53 2026-03-19 cash_out 240.00 USD branch BR-089\n T-54 2026-03-19 wire_out 219,200.00 USD CP-3\n\nHours spent so far: 2.45. You have 3 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"history\", \"month\": \"2026-02\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "history",
|
|
"month": "2026-02"
|
|
},
|
|
"rejection": null,
|
|
"observation": "HISTORY 2026-02: 5 transactions\n T-44 2026-02-01 ach_in 8,916.00 USD CP-1\n T-45 2026-02-16 cash_out 150.00 USD branch BR-089\n T-46 2026-02-18 cash_out 390.00 USD branch BR-052\n T-47 2026-02-21 cash_out 380.00 USD branch BR-052\n T-48 2026-02-26 cash_out 250.00 USD branch BR-089\n\nHours spent so far: 2.90. You have 2 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"close\", \"reason\": \"DOCUMENTED_SOURCE_OF_FUNDS\", \"cites\": [\"doc.D-8578\"], \"note\": \"Source of funds documented: closing_statement.\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "close",
|
|
"reason": "DOCUMENTED_SOURCE_OF_FUNDS",
|
|
"cites": [
|
|
"doc.D-8578"
|
|
],
|
|
"note": "Source of funds documented: closing_statement."
|
|
},
|
|
"rejection": null,
|
|
"observation": "Disposition recorded. The episode is over."
|
|
}
|
|
}
|
|
]
|
|
}
|