karti-ai
2dfa96939e
The first environment shipped through .claude/workflows/new-environment.js: specification, three adversarial reviews (all 'fixable', none fatal), the Python environment, the TypeScript port, captured rollouts, and the demo page. Eleven agents, no errors. The proof that the platform scales is one line long. Alert Triage has a completely different shape from Word Five — JSON actions, priced lookups, an analyst screen instead of a grid — and the only change under src/components/demo/ is a comment edit, because the isolation lint refused the word "wordle" there. Zero shell code changed. 415 contract checks now pass against two demos, up from 206 against one. The environment is honest by construction. Every alert is synthetic, generated from the seed, and the banner saying so sits inside the board surface. Two of the eleven scenario templates are hidden-suspicious: generated by the same code as their benign twin with the signal overlaid only in lookup data, so the free screen is identically distributed and a screen-only policy STRUCTURALLY cannot tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden seeds. That is the counterweight made real rather than asserted. Twelve policies, thirteen ladder assertions, a genuine three-way trade: fast 0.846 wins hours (0.85), misses every hidden case targeted 0.894 wins the shipped total thorough 0.820 wins evidence (1.00), spends 2.9 hours None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at delta 0, and conformance gated on world + scorer + protocol so the browser shows the same alert for ?seed= that Python generated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
172 lines
12 KiB
JSON
172 lines
12 KiB
JSON
{
|
|
"runId": "base-off-s9",
|
|
"seed": 9,
|
|
"model": "brain-qwen38-dspark",
|
|
"capturedAt": "2026-08-28",
|
|
"rewards": {
|
|
"caught": 0.0,
|
|
"hours": 0.0,
|
|
"evidence": 0.0
|
|
},
|
|
"metrics": {
|
|
"hours_spent": 2.5,
|
|
"reference_hours": 2.05,
|
|
"lookups": 5.0,
|
|
"turns_used": 6.0,
|
|
"rejected_replies": 0.0,
|
|
"false_escalation": 0.0,
|
|
"missed": 1.0,
|
|
"typology_match": null
|
|
},
|
|
"truncated": false,
|
|
"outcome": "failed",
|
|
"info": {
|
|
"label": "suspicious",
|
|
"template": "B3",
|
|
"overlay": "funnel",
|
|
"typology": "FUNNEL_ACCOUNT",
|
|
"planted": [
|
|
[
|
|
"CP-3",
|
|
"T-52"
|
|
],
|
|
[
|
|
"T-52"
|
|
]
|
|
],
|
|
"reference_minutes": 123,
|
|
"reference_policy": "targeted",
|
|
"held_out": false,
|
|
"tier": "hidden",
|
|
"screen": "ALERT A-9537 rule R-VEL-02 \u2014 Funds received and moved out again within 48 hours\n fired 2026-04-19\n triggering transactions:\n T-51 2026-04-16 wire_in 462,000.00 USD CP-2\n T-52 2026-04-17 wire_out 448,100.00 USD CP-3\nKYC FILE C-9673 Samir Tanaka\n kyc.account_opened = 2019-12-21\n kyc.country = United States\n kyc.customer_type = individual\n kyc.expected_activity = Salaried individual; payroll deposits and routine spending.\n kyc.expected_monthly_cash = 0.00 USD\n kyc.kyc_refreshed = 2025-04-04\n kyc.occupation = electrician\n kyc.pep = False\n kyc.prior_alerts = 4\n kyc.prior_sars = 1\n kyc.remittance_declared = none declared\n kyc.stated_income = 84,000.00 USD\nTWELVE-MONTH SUMMARY (USD)\n month cash in cash out wires in wires out ach in ach out txns\n 2025-05 0 1,090 0 0 6,958 0 5\n 2025-06 0 630 0 0 7,009 0 5\n 2025-07 0 780 0 0 6,957 0 4\n 2025-08 0 470 0 0 6,989 0 3\n 2025-09 0 1,040 0 0 6,959 0 5\n 2025-10 0 740 0 0 6,996 0 4\n 2025-11 0 860 0 0 6,947 0 5\n 2025-12 0 900 0 0 6,940 0 5\n 2026-01 0 650 0 0 7,025 0 3\n 2026-02 0 460 0 0 7,026 0 3\n 2026-03 0 620 0 0 6,962 0 3\n 2026-04 0 960 462,000 448,100 6,944 0 7\nCOUNTERPARTIES on screen (top by twelve-month volume):\n CP-2 in 1 txns 462,000.00 USD\n CP-3 out 1 txns 448,100.00 USD\nReporting threshold in this jurisdiction: 10,000.00 USD\nSYNTHETIC DATA \u2014 no real customer, account or transaction.\n\nHours spent so far: 0.50. You have 8 replies left."
|
|
},
|
|
"turns": [
|
|
{
|
|
"reply": "I need to investigate this alert. The customer is a salaried electrician earning $84,000/year (~$6,992/month) with a very consistent pattern of ACH deposits around $7,000/month. Suddenly in April 2026, a $462,000 wire came in and $448,100 went out within a day. This is dramatically inconsistent with their profile. Let me gather more information.\n\n{\"action\":\"lookup\",\"what\":\"counterparty\",\"id\":\"CP-2\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 1432,
|
|
"completionTokens": 120,
|
|
"reasoningTokens": null,
|
|
"durationMs": 4843,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-2"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-2 Okafor Title & Escrow\n type title_company; jurisdiction United States; first seen 2026-04-16\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-51\n\nHours spent so far: 0.95. You have 7 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\":\"lookup\",\"what\":\"counterparty\",\"id\":\"CP-3\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 1645,
|
|
"completionTokens": 17,
|
|
"reasoningTokens": null,
|
|
"durationMs": 504,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-3"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-3 Summit Securities\n type brokerage; jurisdiction United States; first seen 2026-04-17\n other alerted accounts sharing this counterparty in twelve months: 8\n transactions: T-52\n\nHours spent so far: 1.40. You have 6 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\":\"lookup\",\"what\":\"prior_alerts\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 1749,
|
|
"completionTokens": 12,
|
|
"reasoningTokens": null,
|
|
"durationMs": 514,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "prior_alerts"
|
|
},
|
|
"rejection": null,
|
|
"observation": "PRIOR ALERTS:\n A-2906 R-VEL-02 fired 2025-01-09 closed after 4 day(s) by Hana Xiang\n Closed: pass-through matches the settlement terms in the agreement on file; counterparties verified.\n A-2547 R-DORM-05 fired 2025-07-01 escalated \u2014 report filed after 2 day(s) by Jamal Ferreira\n Escalated: pattern not explained by the file at the time.\n A-2855 R-DORM-05 fired 2025-07-17 closed after 8 day(s) by Carlos Zapata\n Closed: reactivation explained by a documented source of funds; document reviewed and retained.\n A-2904 R-VEL-02 fired 2025-12-05 closed after 5 day(s) by Dana Grimaldi\n Closed: pass-through matches the settlement terms in the agreement on file; counterparties verified.\n\nHours spent so far: 1.85. You have 5 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\":\"lookup\",\"what\":\"documents\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 2037,
|
|
"completionTokens": 10,
|
|
"reasoningTokens": null,
|
|
"durationMs": 589,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "documents"
|
|
},
|
|
"rejection": null,
|
|
"observation": "DOCUMENTS ON FILE:\n doc.D-8b75 [identity] Government-issued identity document, verified at onboarding\n doc.D-838c [closing_statement] Closing statement \u2014 sale of 82 Alder Street; net proceeds to seller disbursed by Okafor Title & Escrow.\n\nHours spent so far: 2.00. You have 4 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\":\"lookup\",\"what\":\"history\",\"month\":\"2026-04\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 2150,
|
|
"completionTokens": 20,
|
|
"reasoningTokens": null,
|
|
"durationMs": 634,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "history",
|
|
"month": "2026-04"
|
|
},
|
|
"rejection": null,
|
|
"observation": "HISTORY 2026-04: 7 transactions\n T-46 2026-04-04 ach_in 6,944.00 USD CP-1\n T-47 2026-04-04 cash_out 220.00 USD branch BR-068\n T-48 2026-04-06 cash_out 280.00 USD branch BR-068\n T-49 2026-04-10 cash_out 340.00 USD branch BR-068\n T-50 2026-04-12 cash_out 120.00 USD branch BR-068\n T-51 2026-04-16 wire_in 462,000.00 USD CP-2\n T-52 2026-04-17 wire_out 448,100.00 USD CP-3\n\nHours spent so far: 2.45. You have 3 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "Now I have a complete picture. Let me assess:\n\n**The case:**\n- Customer is a salaried electrician earning ~$7,000/month, with a 12-month pattern of consistent ACH payroll deposits and small cash withdrawals.\n- A $462,000 wire arrived from Okafor Title & Escrow (CP-2), a US title company, first seen 2026-04-16.\n- $448,100 went out the next day to Summit Securities (CP-3), a US brokerage, first seen 2026-04-17.\n- A closing statement on file (doc.D-838c) documents the sale of 82 Alder Street, with net proceeds disbursed by Okafor Title & Escrow.\n- Prior alerts A-2906 and A-2904 (both R-VEL-02, same rule) were closed as pass-throughs matching settlement terms with verified counterparties.\n- No other alerted accounts share CP-2; CP-3 (brokerage) is shared with 8 other accounts, which is typical for a major brokerage.\n\n**Assessment:** This is a documented real estate sale. The customer sold a property, received the net proceeds via a title company (standard for US real estate closings), and wired the funds to a brokerage account (likely to invest the proceeds). The source of funds is documented on file. The counterparty (title company) is a legitimate US entity with no other alerted accounts sharing it. The outflow to a brokerage is a routine investment activity.\n\nThis is consistent with a documented source of funds.\n\n{\"action\":\"close\",\"reason\":\"DOCUMENTED_SOURCE_OF_FUNDS\",\"cites\":[\"doc.D-838c\",\"T-51\",\"CP-2\",\"CP-3\",\"kyc.occupation\",\"kyc.stated_income\"],\"note\":\"The $462,000 wire-in from Okafor Title & Escrow (CP-2) is the net proceeds from the sale of 82 Alder Street, as documented in the closing statement on file (doc.D-838c). The $448,100 wire-out to Summit Securities (CP-3), a US brokerage, is consistent with investing real estate sale proceeds. The customer's profile as a salaried electrician ($84,000/yr) does not account for this amount, but the documented property sale does. CP-2 is a title company with no other alerted accounts sharing it; CP-3 is a major brokerage shared by 8 accounts, typical for a large financial institution. Prior R-VEL-02 alerts on this customer (A-2906, A-2904) were similarly closed as settlement pass-throughs. No structuring, funnel, mule, or rapid-movement typology is indicated.\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": 2476,
|
|
"completionTokens": 626,
|
|
"reasoningTokens": null,
|
|
"durationMs": 38579,
|
|
"finishReason": "stop"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "close",
|
|
"reason": "DOCUMENTED_SOURCE_OF_FUNDS",
|
|
"cites": [
|
|
"doc.D-838c",
|
|
"T-51",
|
|
"CP-2",
|
|
"CP-3",
|
|
"kyc.occupation",
|
|
"kyc.stated_income"
|
|
],
|
|
"note": "The $462,000 wire-in from Okafor Title & Escrow (CP-2) is the net proceeds from the sale of 82 Alder Street, as documented in the closing statement on file (doc.D-838c). The $448,100 wire-out to Summit Securities (CP-3), a US brokerage, is consistent with investing real estate sale proceeds. The customer's profile as a salaried electrician ($84,000/yr) does not account for this amount, but the documented property sale does. CP-2 is a title company with no other alerted accounts sharing it; CP-3 is a major brokerage shared by 8 accounts, typical for a large financial institution. Prior R-VEL-02 alerts on this customer (A-2906, A-2904) were similarly closed as settlement pass-throughs. No structuring, funnel, mule, or rapid-movement typology is indicated."
|
|
},
|
|
"rejection": null,
|
|
"observation": "Disposition recorded. The episode is over."
|
|
}
|
|
}
|
|
]
|
|
}
|