karti-ai
2dfa96939e
The first environment shipped through .claude/workflows/new-environment.js: specification, three adversarial reviews (all 'fixable', none fatal), the Python environment, the TypeScript port, captured rollouts, and the demo page. Eleven agents, no errors. The proof that the platform scales is one line long. Alert Triage has a completely different shape from Word Five — JSON actions, priced lookups, an analyst screen instead of a grid — and the only change under src/components/demo/ is a comment edit, because the isolation lint refused the word "wordle" there. Zero shell code changed. 415 contract checks now pass against two demos, up from 206 against one. The environment is honest by construction. Every alert is synthetic, generated from the seed, and the banner saying so sits inside the board surface. Two of the eleven scenario templates are hidden-suspicious: generated by the same code as their benign twin with the signal overlaid only in lookup data, so the free screen is identically distributed and a screen-only policy STRUCTURALLY cannot tell them apart. The probe ladder measures it: `fast` catches 0.0 of hidden seeds. That is the counterweight made real rather than asserted. Twelve policies, thirteen ladder assertions, a genuine three-way trade: fast 0.846 wins hours (0.85), misses every hidden case targeted 0.894 wins the shipped total thorough 0.820 wins evidence (1.00), spends 2.9 hours None dominates. 92 Python tests, 35 TypeScript tests, 65 fixtures replaying at delta 0, and conformance gated on world + scorer + protocol so the browser shows the same alert for ?seed= that Python generated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
187 lines
12 KiB
JSON
187 lines
12 KiB
JSON
{
|
|
"runId": "thorough-s1",
|
|
"seed": 1,
|
|
"model": "thorough-analyst",
|
|
"capturedAt": "2026-08-28",
|
|
"rewards": {
|
|
"caught": 1.0,
|
|
"hours": 0.1864406779661017,
|
|
"evidence": 1.0
|
|
},
|
|
"metrics": {
|
|
"hours_spent": 2.95,
|
|
"reference_hours": 0.55,
|
|
"lookups": 6.0,
|
|
"turns_used": 7.0,
|
|
"rejected_replies": 0.0,
|
|
"false_escalation": 0.0,
|
|
"missed": 0.0,
|
|
"typology_match": null
|
|
},
|
|
"truncated": false,
|
|
"outcome": "solved",
|
|
"info": {
|
|
"label": "benign",
|
|
"template": "B4",
|
|
"overlay": null,
|
|
"typology": null,
|
|
"planted": [
|
|
[
|
|
"kyc.expected_monthly_cash"
|
|
],
|
|
[
|
|
"kyc.expected_monthly_cash",
|
|
"kyc.business_type"
|
|
]
|
|
],
|
|
"reference_minutes": 33,
|
|
"reference_policy": "fast",
|
|
"held_out": false,
|
|
"tier": "benign",
|
|
"screen": "ALERT A-8971 rule R-STR-01 \u2014 Three or more cash deposits just under the reporting threshold within ten days\n fired 2026-04-26\n triggering transactions:\n T-213 2026-04-16 cash_in 9,440.00 USD branch BR-052\n T-217 2026-04-21 cash_in 8,950.00 USD branch BR-052\n T-219 2026-04-22 cash_in 8,880.00 USD branch BR-052\nKYC FILE C-9647 Oakhill Kitchen Group\n kyc.account_opened = 2019-07-11\n kyc.business_type = pizzeria\n kyc.country = United States\n kyc.customer_type = business\n kyc.expected_activity = Cash-intensive food service; daily takings deposited most trading days.\n kyc.expected_monthly_cash = 86,000.00 USD\n kyc.kyc_refreshed = 2026-03-13\n kyc.pep = False\n kyc.prior_alerts = 4\n kyc.prior_sars = 0\n kyc.remittance_declared = none declared\n kyc.stated_income = 492,000.00 USD\nTWELVE-MONTH SUMMARY (USD)\n month cash in cash out wires in wires out ach in ach out txns\n 2025-05 42,400 0 0 0 0 13,935 18\n 2025-06 38,160 0 0 0 0 12,582 19\n 2025-07 44,260 0 0 0 0 15,928 18\n 2025-08 41,680 0 0 0 0 13,940 18\n 2025-09 44,950 0 0 0 0 15,498 17\n 2025-10 35,860 0 0 0 0 13,284 19\n 2025-11 42,370 0 0 0 0 12,792 19\n 2025-12 39,280 0 0 0 0 13,776 19\n 2026-01 39,860 0 0 0 0 13,940 19\n 2026-02 39,280 0 0 0 0 12,718 18\n 2026-03 36,510 0 0 0 0 11,693 19\n 2026-04 71,720 0 0 0 0 14,218 18\nCOUNTERPARTIES on screen (top by twelve-month volume):\n CP-1 out 48 txns 103,976.00 USD\n CP-2 out 12 txns 60,331.50 USD\nReporting threshold in this jurisdiction: 10,000.00 USD\nSYNTHETIC DATA \u2014 no real customer, account or transaction.\n\nHours spent so far: 0.50. You have 8 replies left."
|
|
},
|
|
"turns": [
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"documents\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "documents"
|
|
},
|
|
"rejection": null,
|
|
"observation": "DOCUMENTS ON FILE:\n doc.D-e0ed [registration] Business registration \u2014 Oakhill Kitchen Group\n doc.D-000c [licence] Food service licence, renewed annually\n\nHours spent so far: 0.65. You have 7 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"prior_alerts\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "prior_alerts"
|
|
},
|
|
"rejection": null,
|
|
"observation": "PRIOR ALERTS:\n A-3217 R-WIRE-04 fired 2024-07-08 closed after 7 day(s) by Beatriz Grimaldi\n Closed: destination and amount match the remittance declared at onboarding; within stated income.\n A-4028 R-DORM-05 fired 2024-10-12 closed after 9 day(s) by Mateo Jensen\n Closed: reactivation explained by a documented source of funds; document reviewed and retained.\n A-2622 R-VEL-02 fired 2025-01-17 closed after 5 day(s) by Samir Pereira\n Closed: pass-through matches the settlement terms in the agreement on file; counterparties verified.\n A-1389 R-STR-01 fired 2026-03-06 closed after 3 day(s) by Marta Eskildsen\n Closed: deposits are documented daily takings; amounts vary with trading days; within kyc.expected_monthly_cash.\n\nHours spent so far: 1.10. You have 6 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"counterparty\", \"id\": \"CP-1\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-1"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-1 Highland Produce Co.\n type business; jurisdiction United States; first seen 2025-05-06\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-5, T-7, T-11, T-12, T-20, T-22, T-29, T-33, T-42, T-50, T-51, T-53, T-64, T-66, T-69, T-71, T-75, T-81, T-85, T-90, T-94, T-95, T-97, T-101, T-114, T-116, T-120, T-121, T-130, T-133, T-135, T-141, T-150, T-152, T-157, T-163, T-168, T-176, T-178, T-182, T-192, T-195, T-198, T-201, T-206, T-211, T-212, T-216\n\nHours spent so far: 1.55. You have 5 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"counterparty\", \"id\": \"CP-2\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "counterparty",
|
|
"id": "CP-2"
|
|
},
|
|
"rejection": null,
|
|
"observation": "COUNTERPARTY CP-2 Highland Foods Co.\n type business; jurisdiction United States; first seen 2025-05-03\n other alerted accounts sharing this counterparty in twelve months: 0\n transactions: T-3, T-25, T-38, T-61, T-76, T-92, T-113, T-131, T-153, T-171, T-189, T-208\n\nHours spent so far: 2.00. You have 4 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"history\", \"month\": \"2026-04\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "history",
|
|
"month": "2026-04"
|
|
},
|
|
"rejection": null,
|
|
"observation": "HISTORY 2026-04: 18 transactions\n T-204 2026-04-01 cash_in 3,380.00 USD branch BR-052\n T-205 2026-04-04 cash_in 3,420.00 USD branch BR-052\n T-206 2026-04-07 ach_out 2,091.00 USD CP-1\n T-207 2026-04-08 cash_in 4,850.00 USD branch BR-052\n T-208 2026-04-10 ach_out 5,436.60 USD CP-2\n T-209 2026-04-11 cash_in 5,220.00 USD branch BR-052\n T-210 2026-04-13 cash_in 4,850.00 USD branch BR-052\n T-211 2026-04-13 ach_out 2,509.20 USD CP-1\n T-212 2026-04-14 ach_out 1,672.80 USD CP-1\n T-213 2026-04-16 cash_in 9,440.00 USD branch BR-052\n T-214 2026-04-16 cash_in 5,010.00 USD branch BR-052\n T-215 2026-04-19 cash_in 3,970.00 USD branch BR-052\n T-216 2026-04-19 ach_out 2,509.20 USD CP-1\n T-217 2026-04-21 cash_in 8,950.00 USD branch BR-052\n T-218 2026-04-21 cash_in 4,640.00 USD branch BR-052\n T-219 2026-04-22 cash_in 8,880.00 USD branch BR-052\n T-220 2026-04-24 cash_in 4,720.00 USD branch BR-052\n T-221 2026-04-25 cash_in 4,390.00 USD branch BR-014\n\nHours spent so far: 2.45. You have 3 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"lookup\", \"what\": \"history\", \"month\": \"2026-03\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "lookup",
|
|
"what": "history",
|
|
"month": "2026-03"
|
|
},
|
|
"rejection": null,
|
|
"observation": "HISTORY 2026-03: 19 transactions\n T-185 2026-03-01 cash_in 2,020.00 USD branch BR-052\n T-186 2026-03-02 cash_in 2,850.00 USD branch BR-014\n T-187 2026-03-03 cash_in 2,990.00 USD branch BR-014\n T-188 2026-03-04 cash_in 2,500.00 USD branch BR-014\n T-189 2026-03-04 ach_out 4,526.40 USD CP-2\n T-190 2026-03-08 cash_in 2,900.00 USD branch BR-014\n T-191 2026-03-11 cash_in 2,610.00 USD branch BR-052\n T-192 2026-03-11 ach_out 1,886.00 USD CP-1\n T-193 2026-03-12 cash_in 2,150.00 USD branch BR-052\n T-194 2026-03-13 cash_in 2,230.00 USD branch BR-014\n T-195 2026-03-15 ach_out 1,886.00 USD CP-1\n T-196 2026-03-17 cash_in 2,310.00 USD branch BR-014\n T-197 2026-03-20 cash_in 2,690.00 USD branch BR-014\n T-198 2026-03-20 ach_out 1,886.00 USD CP-1\n T-199 2026-03-21 cash_in 3,170.00 USD branch BR-052\n T-200 2026-03-22 cash_in 2,900.00 USD branch BR-014\n T-201 2026-03-23 ach_out 1,508.80 USD CP-1\n T-202 2026-03-29 cash_in 2,740.00 USD branch BR-052\n T-203 2026-03-31 cash_in 2,450.00 USD branch BR-014\n\nHours spent so far: 2.90. You have 2 replies left."
|
|
}
|
|
},
|
|
{
|
|
"reply": "{\"action\": \"close\", \"reason\": \"CONSISTENT_WITH_PROFILE\", \"cites\": [\"kyc.expected_monthly_cash\"], \"note\": \"Cash volume within the expectation on file; band deposits are ordinary takings.\"}",
|
|
"reasoning": null,
|
|
"call": {
|
|
"promptTokens": null,
|
|
"completionTokens": null,
|
|
"reasoningTokens": null,
|
|
"durationMs": null,
|
|
"finishReason": "generated"
|
|
},
|
|
"info": {
|
|
"action": {
|
|
"action": "close",
|
|
"reason": "CONSISTENT_WITH_PROFILE",
|
|
"cites": [
|
|
"kyc.expected_monthly_cash"
|
|
],
|
|
"note": "Cash volume within the expectation on file; band deposits are ordinary takings."
|
|
},
|
|
"rejection": null,
|
|
"observation": "Disposition recorded. The episode is over."
|
|
}
|
|
}
|
|
]
|
|
}
|