karti
666310b264
The Motion half of the demo book was two engagements, which was enough to show that the loop works and not enough to show what the product is for. The page that matters asks whether the motion is repeating, and a stage rail of zeroes cannot answer it. Eight engagements now, one at every open stage, hanging off demand deals the demand book already creates — the pipeline happened to have exactly one open deal at each of the eight, so no deal was invented and the quoted pipeline counts are unchanged. Forty-four artefacts, nineteen scores, one promotion. Three things the book is laid out to prove that a folder of templates cannot. Every stage is occupied, and every one of the twelve starter templates is instantiated at least once, so "stages covered 8/8" is a measurement rather than a claim about the seed. Scores move, and sometimes move down. Nineteen scores across eight trajectories, with nine dimensions regressing somewhere — the Verity fine-tuning record runs 62.5 -> 60.5 -> 84.0 -> 80.8, because a scorecard that only ever rises is a ratchet and teaches a reader to distrust it. Every score is computed with `motionScoreBasisPoints` and banded with `motionBand` rather than written as a literal, so the seed and the product cannot disagree about what the same dimensions are worth. The artefact bodies are the customer's own facts — named people, real volumes, the specific thing going wrong, and a live unresolved risk in each. An artefact whose body is the template with the blanks still in it is precisely what this data exists to disprove. Six of the forty-four have no template at all, which is the honest shape of an engagement and the reason `kind` is carried on the artefact rather than derived: `engagement_artifacts.kind` is NOT NULL and a derived kind would have been null for exactly those six. The bodies live in JSON beside the loader for the same reason the starter library's do — forty-four markdown bodies as backtick strings is a module nobody can review. The promotion copies `body` from the artefact verbatim, as `promoteArtifact` does, rather than writing a hand-authored version 2. A demo that produced a row the real path could not have produced would teach the wrong shape of the table. Two name collisions the authors could not see are fixed: a Quillon contact shared a full name with a demo seller, and an Aurelian one shared a surname with another. `usage_count` is raised once per template rather than once per artefact, so it stays symmetrical with the decrement `clear()` already does — verified by tearing the book down and confirming the library returns to twelve templates with every counter back at zero, since a counter left above zero makes a starter template permanently un-editable.
204 lines
28 KiB
JSON
204 lines
28 KiB
JSON
{
|
||
"key": "northwind-fleet-expansion",
|
||
"dealName": "DEMO — Fleet expansion — 32× B200",
|
||
"accountName": "DEMO — Northwind Robotics",
|
||
"stage": "expansion",
|
||
"status": "won",
|
||
"summary": "DEMO — Won: second workflow at Northwind, the recovery-action policy for the mobile picking fleet, live in seven of nine DCs and backed by a 12-month 32× B200 reserved commitment signed on 2026-06-24. The case study is approved for named use; workflow 3 was scoped in July and is blocked on Ingrid Halloway's dock-manoeuvre autonomy threshold sign-off, outstanding since June — a separate item from the shelf-collapse escalation rule she has also not answered since April.",
|
||
"openedDaysAgo": 268,
|
||
"artefacts": [
|
||
{
|
||
"templateSlug": "trainability-qualification",
|
||
"stage": "qualification",
|
||
"title": "DEMO — Trainability scorecard: recovery-action selection (workflow 2)",
|
||
"status": "final",
|
||
"authoredDaysAgo": 255,
|
||
"body": "# Trainability and deal qualification — workflow 2, recovery-action selection\n\nSecond workflow at Northwind Robotics. The first (pick-failure cause classification) has been serving all nine distribution centres since March 2025 and is now gated by Northwind's own eval. This scorecard covers a different loop and is scored on its own evidence, not on the goodwill of the first deployment.\n\nNote for anyone reading across accounts: this is the mobile picking fleet, not the 1,900 fixed cold-chain manipulators, which are a separate programme with their own remote-operations sponsor and their own action space. Nothing here shares a model, an eval or a metric owner with that programme.\n\n## The task\n\nNorthwind runs 4,180 mobile picking robots across nine DCs. When a robot faults or stalls, the on-board planner picks one of nine recovery manoeuvres or escalates to a human teleoperator. Fleet-wide there are roughly 61,000 recovery events a month, of which about 18,400 escalate. Marcus Ebele (Director, Fleet Reliability) is judged on teleoperator minutes per thousand robot-hours and on the safety-critical miss rate — a stall that should have escalated and did not.\n\nThe decision is definable: nine manoeuvres plus escalate, taken from the existing planner's action space. It is not clean. Two manoeuvres (\"reposition and retry\" and \"shelf-clear and retry\") are used interchangeably by the fleet engineers today, which is the first thing the eval will have to settle. Scored 3 until there is a written decision rule an adjudicator can apply.\n\n## Verifier\n\nMostly computable. The sim replays the robot state and scores whether the chosen manoeuvre actually cleared the fault within the 90-second recovery budget. Escalation correctness is not computable — it needs an adjudicated label for \"should a human have been called\", and that is where the rubric judge sits. Provisionally scored 3, not 4, and the reason is written down here so it can be argued with later: we are assuming the adjudication apparatus from workflow 1 transfers, and that assumption has not been tested.\n\n## Baseline\n\nThe weakest dimension by a distance. The incumbent heuristic planner has never been measured on a frozen set. Marcus quotes \"about seventy per cent\" from a 2024 internal review nobody can locate. Scored 2. Nothing gets contracted until the heuristic is re-run unchanged on a frozen set we build first.\n\n## Data and environment\n\nTelemetry, camera frames and teleoperator session logs are Northwind's own and already covered by the MSA training rights schedule signed for workflow 1. No new legal path required. The fleet sim built for workflow 1 exists and has a maintained CI; it does not yet model the recovery action space, which is scoped at roughly two weeks of extension rather than a rebuild.\n\n## Commercial\n\nRhea Sundaram (VP Autonomy) sponsors, as she did the first time. Dana Iyer (Head of Infrastructure Finance) owns the line but has not agreed that reserved capacity rather than on-demand is the right instrument. The forcing function is soft: peak season begins in October and Marcus wants the escalation rate down before it, but no date is contractual.\n\n## Result\n\n80% weighted — Build. Conditions: freeze a held-out set and re-run the heuristic on it before any training; validate the transferred rubric on the new label space before trusting it with reward.\n",
|
||
"kind": "qualification"
|
||
},
|
||
{
|
||
"templateSlug": "post-training-poc",
|
||
"stage": "poc",
|
||
"title": "DEMO — POC structure: autonomous recovery-action selection",
|
||
"status": "final",
|
||
"authoredDaysAgo": 203,
|
||
"body": "# POC structure — autonomous recovery-action selection\n\nEight weeks, paid, on on-demand capacity billed against the workflow 1 order form. There is no reserved block yet and the POC must not assume one. Kick-off 2026-01-30, exit date agreed in advance: 2026-03-27.\n\n## The one question\n\nOn a held-out set of 900 recovery events sampled uniformly by day from Northwind production traffic between 2026-02-01 and 2026-03-14, to be frozen on 2026-03-20 and held by Northwind, an adapted 32B open-weights policy will score at least 85% agreement with the adjudicated action label, against the incumbent heuristic planner's baseline re-run unchanged on the same 900, as measured by the sim verifier plus the escalation rubric judge, whose agreement with Northwind's adjudicators must be at least 88% on a 200-case dual-labelled audit drawn from the dev split, where the human-to-human ceiling is 92%.\n\nSecondary threshold, and the one Marcus actually cares about: the safety-critical miss rate (should have escalated, did not) must not rise. Baseline to be established on the same frozen 900.\n\n## Datasets and how they relate\n\n- Training pool: 71,000 recovery trajectories from 2025-06 to 2025-12, strictly predating both eval sets.\n- Dev split: 700 events from 2026-01, adjudicated, used for iteration, for rubric revision, and for the 200-case judge audit.\n- Held-out set: the frozen 900 described above. Excluded from training and from verifier development, including from the judge audit — that exclusion is why the audit is drawn from the dev split. Marcus Ebele confirms it in writing at freeze.\n\nThe dev split and the held-out set are disjoint by construction and by date. The nine DCs are represented in the held-out set in proportion to their event volume, which means DC-3 and DC-7 together are 31% of it; per-DC recall is reported separately so the aggregate cannot hide a single-site collapse.\n\n## Verifier design\n\nRoughly 78% of the trajectory is programmatically verifiable: the sim executes the chosen manoeuvre against the recorded robot state and scores fault cleared within 90 seconds, collision-free, no shelf disturbance. The remaining 22% is the escalate-or-not judgement and the free-text intervention note handed to the teleoperator, both judged by a rubric model.\n\nThe rubric is inherited from workflow 1 and must be re-validated before it carries reward. Adjudicator calibration is scheduled for week 2 with a defended cut of 0.75 Krippendorff alpha, the same cut Northwind's adjudicators defended last year. Before that, the label space itself has to be written down: \"reposition and retry\" against \"shelf-clear and retry\" is currently a matter of habit, and an adjudicator cannot be calibrated against a habit.\n\n## Recipe\n\nSupervised fine-tuning on the 71,000 trajectories, then RL in the extended fleet sim against a composite reward: manoeuvre success in sim at 0.55, escalation correctness at 0.30, intervention-note rubric at 0.15, with a hard zero on any trajectory producing a simulated collision. Because the note carries the least weight it gets an explicit regression check on the dev split each run.\n\nBudget: 8,900 GPU-hours over a fourteen-day training window on 32 B200s, about 83% occupancy. The nine-day sim extension runs before that window and uses almost no GPU; it is wall-clock, not spend.\n\n## How this fails visibly\n\nIf the re-run heuristic comes back above 82%, the headroom argument is dead and we say so on 2026-03-27 rather than arguing about the metric. If adjudicator alpha will not clear 0.75 after two calibration rounds, the escalation half of the task is undefined and the POC scope shrinks to manoeuvre selection only.\n",
|
||
"kind": "poc"
|
||
},
|
||
{
|
||
"templateSlug": null,
|
||
"stage": "poc",
|
||
"title": "DEMO — Transfer accounting: what workflow 1 paid for and what we re-paid",
|
||
"status": "final",
|
||
"authoredDaysAgo": 166,
|
||
"body": "# What transferred from workflow 1, and what we re-paid for in full\n\nWritten at week 6, on 2026-03-06, so the claim is auditable later. Internal cost figures are Prime Intellect engagement accounting, not anything Northwind is billed against.\n\n## The claim\n\nWorkflow 1 (pick-failure cause classification) ran 19 weeks and cost $1.42m to deliver. Workflow 2 (recovery-action selection) is tracking to 11 weeks and $870k. That is the number to quote, because it is the only one that nets the savings against the one line that rose. The difference is not that we got better at it. It is that three specific assets were already paid for, and one was not.\n\n## What transferred\n\n**The adjudication apparatus.** Northwind has eleven trained adjudicators — fleet engineers who spend four hours a week labelling. Standing them up the first time cost 34 person-days: recruitment inside the reliability org, two days of rubric instruction, three calibration rounds before alpha cleared the defended cut of 0.75. This time we re-used the same eleven people, the same instruction material, the same cut, and the same argument for why 0.75 is the right cut rather than 0.8. Cost this time: 6 person-days.\n\n**The environment scaffolding.** The fleet sim — state replay, telemetry decoding, camera-frame playback, CI on every commit — was seven weeks of build in workflow 1. Extending it to execute the nine recovery manoeuvres took 9 days. The action space was new; nothing under it was.\n\n**The commercial paper.** The workflow 1 reference architecture and the price book went across unchanged: same serving topology, same tail-latency budget shape, same unit rates. No scoping architecture document and no new pricing work exists for workflow 2 because neither was needed, and that is a real saving of about three weeks of lead time, not an omission.\n\nTogether, roughly 28 person-days of adjudicator standing-up, five weeks of environment build and three weeks of architecture and pricing did not have to happen twice.\n\n## What did not transfer\n\n**The eval.** The label space is different — nine manoeuvres plus escalate, against workflow 1's six root-cause classes. Nothing about the old adjudicated corpus is usable. Worse, the label space had to be defined before it could be populated: \"reposition and retry\" and \"shelf-clear and retry\" now separate on wedge state and shelf integrity under a one-page written rule, which the adjudicators applied rather than argued about. We built a new dev split (700) and a new held-out set (900), and the adjudication of those 1,600 events cost 41 person-days against workflow 1's 36. The eval got more expensive, not less, because the held-out set is larger and each case takes an adjudicator longer: judging the correct next action requires watching the recorded recovery, where judging root cause did not.\n\nIn engagement accounting: eval construction was $128k the first time and $141k the second. It is the one line that went up, and it went up inside a total that came down by $550k.\n\n## The thing that nearly went wrong\n\nThe rubric transferred; the adjudicators' habits did not transfer with it. Round one of calibration came back at 0.61 alpha, below the defended cut. The cause, found on 2026-02-19: the same eleven people had spent a year labelling by root cause, and half of them were still doing that — marking \"shelf-clear\" whenever the shelf was the underlying problem, regardless of whether shelf-clear was the right next action for a robot already wedged. One page of counter-examples and a second round fixed it, at a cost of 6 person-days rather than 34.\n\nThat is the honest version of transfer. The apparatus carried over; the reflex it was trained on had to be retrained, and if we had trusted the inherited rubric with reward without re-validating it, the entire escalation half of the reward would have been measuring the wrong thing for the fourteen-day training window.\n\n## What to write down for workflow 3\n\nThe sim, the adjudicator pool, the cut, the architecture and the price book transfer. The eval does not, and should be budgeted at full price every time. Assume the rubric needs one calibration round against any new label space before it carries reward.\n",
|
||
"kind": "pricing"
|
||
},
|
||
{
|
||
"templateSlug": "procurement-and-budget-path",
|
||
"stage": "procurement",
|
||
"title": "DEMO — Procurement and budget path: 32× B200 reserved, 12 months",
|
||
"status": "final",
|
||
"authoredDaysAgo": 50,
|
||
"body": "# Procurement and budget path — 32 B200s, 12-month reserved\n\nWritten 2026-06-30, after signature, while the path is still recoverable from memory.\n\n## The instrument\n\n32 B200s reserved for 12 months, $1,047,859. Signed 2026-06-24. It replaces on-demand consumption that ran at roughly $118k a month across the POC and the workflow 1 retraining cadence. The POC itself ran on that on-demand line against the workflow 1 order form; nothing was reserved until this signature, which is why the run-rate argument below is measured rather than modelled.\n\n## Budget line and its source\n\nFleet Reliability capital, FY26 line \"autonomy platform compute\", owned by Dana Iyer (Head, Infrastructure Finance). This is a named line with a named owner and a stated FY26 figure of $1.4m, which is what moved this dimension from 3 to 4. At the first scorecard the line existed but Dana had not agreed that reserved was the right instrument; she agreed on 2026-06-02 after we showed the twelve-month on-demand projection against the reserved price.\n\n## The path, as it actually ran\n\n1. Marcus Ebele raised the requisition on 2026-06-04 against the existing MSA. No new master agreement was needed — workflow 1's MSA and its IP and training-rights schedules cover this scope, which removed about six weeks from the path.\n2. Finance review with Dana, 2026-06-09. The question was not price, it was utilisation risk: a reserved block that sits idle is a line item she has to defend.\n3. Security: no review. Rob Castellan confirmed in writing on 2026-05-02 that nothing crosses a new boundary.\n4. Rhea Sundaram approved above her $750k delegation threshold via the CTO on 2026-06-19.\n5. Signature 2026-06-24.\n\n## The utilisation argument, and the honest version of it\n\nDana's forecast, which we did not dispute: months 1 to 4 land at about 62% utilisation, because workflow 2 training is done and workflow 3 is not yet scoped as of today. Nothing from Pick Operations lands on this block either: the hard-pick strategy loop Elke Sandberg's organisation is deploying runs on its own reserved Blackwell block in Helsinki, on a separate line, and Dana asked twice to be sure it was not being counted here. Months 5 to 12 depend entirely on whether the dock-approach workflow starts.\n\nWe did not paper over this. The commitment was justified on two things Dana could check: the observed on-demand run rate over the previous seven months, and the monthly retraining cadence both live workflows now carry, which is a floor rather than a hope. Anything above that floor is upside that has to be earned, and if workflow 3 does not start by January the block is over-bought for two quarters. Dana has that in writing and accepted it.\n\n## Open\n\nThe renewal conversation is in May 2027 and its outcome is a function of workflow 3 starting. The blocker on workflow 3 is not budget and not security — it is Ingrid Halloway's sign-off on the dock-manoeuvre autonomy threshold, requested on 2026-06-11 and unanswered. That is a different item from her open question on the shelf-collapse escalation rule in production, and conflating the two has already cost us one wasted meeting.\n",
|
||
"kind": "pricing"
|
||
},
|
||
{
|
||
"templateSlug": "production-readiness",
|
||
"stage": "deployment",
|
||
"title": "DEMO — Production readiness: recovery-action policy across nine DCs",
|
||
"status": "final",
|
||
"authoredDaysAgo": 25,
|
||
"body": "# Production readiness — recovery-action policy, nine DCs\n\nPOC exit cleared on 2026-03-27. Written before shadow start to gate production load, and updated on 2026-07-25 now that full cutover has six weeks behind it; the ramp and fallback sections record what actually happened.\n\n## Inference path\n\nAdapted 32B open-weights, served inside Northwind's own VPC on their existing A100 inference fleet, not on training capacity. Path: robot fault event to the DC edge broker, to the fleet-orchestration service, to the policy endpoint, back with a manoeuvre code and an intervention note. The escalation rubric judge runs out of band on a 5% sample for drift, not inline; inline judging blew the latency budget in load test and buys nothing at decision time.\n\nLatency budget, owned by Marcus Ebele: p95 under 400 ms, p99 under 900 ms, measured at the orchestration service, not at the model. A stalled robot occupies an aisle, so a slow decision is a physical queue and not merely a bad user experience. Load test at 4.2x mean concurrency, held 40 minutes, 99th-percentile context (the long-tail cases are the ones with 60 seconds of prior telemetry attached): p95 310 ms, p99 780 ms, cold start 4.1 s, mitigated by keeping two warm replicas per region.\n\nServing location matches the `inference-location` decision in the workflow 1 Data and Security Brief; Rob Castellan (CISO) confirmed no new review was required on 2026-05-02 because nothing crosses a new boundary.\n\n## Ramp\n\n- Shadow, 14 days, all nine DCs: policy scored on every eligible event, heuristic still acting. Agreement with the adjudicated label was measured on a 400-event adjudicated sample drawn across the fourteen days — that is what the eleven adjudicators can absorb at four hours a week — and held at 88.6%, within noise of the held-out 89.2%.\n- 10% of events in DC-3 and DC-7 only, 10 days.\n- 50% in those two DCs, 14 days.\n- 100% in seven DCs from 2026-06-08. DC-2 and DC-9 stay on the heuristic as a control cohort through the peak-season window, at Marcus's request and to our benefit.\n\n## Fallback\n\nAny of: policy unavailable, p99 breach sustained over 5 minutes, or a manoeuvre code outside the nine — the orchestration service falls back to the heuristic planner, which stays deployed and tested monthly. Fallback is automatic and does not page. Two fallbacks fired in the first six weeks after cutover, both during a broker restart.\n\n## Live eval and retraining\n\nNorthwind adjudicates 120 sampled events a week and runs the held-out eval monthly; the eval gates their own deploys, ours included. Retraining trigger, agreed with Marcus: adjudicated agreement below 86% on two consecutive weekly samples, or safety-critical miss rate above 1.2% on any single week.\n\n## Open at go-live, and still open now\n\nThe sim does not reproduce shelf-collapse contact physics, so roughly 9% of real recovery events — 81 of the frozen 900 — have no faithful simulated counterpart and the policy is effectively unverified on them. Those events are routed to escalation by rule rather than by policy. This is a known, accepted gap, not a solved one, and it constrains both the environment and the verifier: a verifier that cannot score 9% of production traffic is not a finished verifier. Ingrid Halloway (Safety Officer) has not signed off on removing that rule, and has not answered the last two requests to discuss it, the first of which went out in April.\n",
|
||
"kind": "playbook"
|
||
},
|
||
{
|
||
"templateSlug": "case-study-frame",
|
||
"stage": "expansion",
|
||
"title": "DEMO — Case study: Northwind Robotics recovery-action policy",
|
||
"status": "final",
|
||
"authoredDaysAgo": 4,
|
||
"body": "# Case study — Northwind Robotics, autonomous recovery-action selection\n\nApproved by the customer for named use. Second workflow at this account.\n\n**The customer and the workflow.** Northwind Robotics runs 4,180 mobile picking robots across nine distribution centres, roughly 61,000 recovery events a month. When a robot faults, something must choose one of nine recovery manoeuvres or call a teleoperator. The heuristic escalated about 18,400 events a month at a median session of 4 minutes 20 seconds. The expensive failures ran the other way: a stall that should have escalated and did not blocks an aisle until a technician walks to it.\n\n**Why not prompting.** Four months against a frontier model plateaued at an informal 74%: the decision needs 60 seconds of telemetry and a camera frame, and teleoperator corrections were logged and discarded.\n\n**What we built, and what was hard about it.** Almost everything was second-hand; the eval was the whole cost. The hard part came before labelling: two of the nine manoeuvres were used interchangeably by fleet engineers, so nobody could be calibrated until \"reposition and retry\" and \"shelf-clear and retry\" were separated in writing, on wedge state and shelf integrity. That rule took two weeks. The inherited verifier covered the mechanics and none of the judgement: a rubric pointed at an unseen label space is not a verifier yet. The environment was nine days of extension on a sim that cost seven weeks the first time.\n\n**How we know the verifier holds.** Rubric judge against two Northwind adjudicators on 200 dual-labelled dev-split cases, never the held-out 900: judge-to-human 90%, human-to-human 92%. Two reward hacks: over-selecting \"reposition and retry\" where the sim under-charged its time cost; and escalating under ambiguity to flatter the safety metric. Patched with a time penalty and an escalation cost. The sim cannot reproduce shelf-collapse contact physics, so 81 of the frozen 900 (9%) escalate by rule; 60 of those were read by hand by Northwind's engineers.\n\n**The numbers.**\n\n| Measure | Baseline | Result | Measured on | Measured by |\n| --- | --- | --- | --- | --- |\n| Agreement with adjudicated action label | 71.4%, heuristic re-run unchanged | 89.2% | The frozen 900, eval-v2 | Northwind fleet reliability; runs r-4462 (2026-03-23) and r-4471 (2026-03-24), Yusuf Adeyemi |\n| Escalation rate | 30.1% | 17.6% | The frozen 900, eval-v2 | Northwind; runs r-4462 and r-4471 |\n| Safety-critical miss rate | 2.1% | 0.9% | The frozen 900, eval-v2 | Northwind safety; run r-4488, 2026-03-25, Yusuf Adeyemi |\n| Teleoperator minutes per thousand robot-hours, live | 47.0 | 31.5 | Seven DCs, six weeks post-cutover | Northwind session logs, extract fa-0712 (2026-07-22), Marcus Ebele's team |\n\nMinutes per thousand robot-hours is what Marcus is paid on, hence the live row. It is confounded — a firmware release, seasonally low volumes — so DC-2 and DC-9, held on the heuristic at 45.8, are the control.\n\n**What surprised us.** On 2026-02-19 the inherited rubric failed calibration at 0.61 alpha against a defended cut of 0.75: the eleven adjudicators, trained a year earlier on root-cause labelling, were labelling cause rather than correct next action. One page of counter-examples and a second round fixed it in 6 person-days. The apparatus transferred; the reflex did not.\n\n**What we would do differently.** Re-validate an inherited rubric against a new label space in week one, before environment work. Budget the eval at full price: $141k against workflow 1's $128k, the only line that rose. Delivery cost fell to $870k from $1.42m, because the adjudicator pool, the sim, the architecture and the price book were not built twice.\n\n**What the customer owns now.** The extended sim, the verifier and its rubric, both eval sets, the weights, and a weekly 120-event adjudication loop gating their own deploys.\n\n**What came next commercially.** 32 B200s reserved for 12 months, signed 2026-06-24 at $1,047,859 on Fleet Reliability capital, separate from the reserved Blackwell block Pick Operations bought for the hard-pick strategy loop, which shares this account and nothing else. A third workflow, dock-approach fault prediction, was scoped in July 2026 and is blocked on Ingrid Halloway's dock-manoeuvre autonomy threshold sign-off, requested 2026-06-11 and unanswered, with no sponsor named.\n\n**Approvals.** Rhea Sundaram, VP Autonomy, approved v3 on 2026-08-11; legal cleared named use on 2026-08-13 under the MSA publicity terms.\n",
|
||
"kind": "case_study"
|
||
}
|
||
],
|
||
"scores": [
|
||
{
|
||
"daysAgo": 254,
|
||
"note": "DEMO — First scorecard on workflow 2. Strong on rights, environment and sponsorship because workflow 1 already paid for them; weak on baseline, since the heuristic planner has never been measured on a frozen set, and the forcing function is only peak season. Verifier scored 3 on the untested assumption that the workflow 1 rubric transfers, and task definability 3 because two of the nine manoeuvres are used interchangeably today.",
|
||
"dimensions": [
|
||
{
|
||
"id": "task_definability",
|
||
"weight": 10,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "verifier_quality",
|
||
"weight": 13,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "trace_data_rights",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "baseline_measured",
|
||
"weight": 8,
|
||
"score": 2
|
||
},
|
||
{
|
||
"id": "headroom",
|
||
"weight": 10,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "env_constructibility",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "metric_owner",
|
||
"weight": 9,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "budget_source",
|
||
"weight": 9,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "exec_sponsor",
|
||
"weight": 4,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "security_legal_path",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "forcing_function",
|
||
"weight": 8,
|
||
"score": 2
|
||
},
|
||
{
|
||
"id": "expansion_surface",
|
||
"weight": 5,
|
||
"score": 3
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"daysAgo": 20,
|
||
"note": "DEMO — Closing score after go-live and the reserved commitment. Baseline, budget and headroom resolved: the heuristic was re-run on the frozen 900 at 71.4% against the policy's 89.2%, and Dana Iyer named the FY26 line. Task definability rises to 4 because the written wedge-state rule separating reposition-and-retry from shelf-clear-and-retry is now the adjudicated label space, not a habit. The sim gap cuts twice: environment and verifier both sit at 3, because shelf-collapse contact physics cannot be reproduced, about 9% of production events are unverified, and the escalation half of the task is carried by a rubric judge. Forcing function stays at 2 — peak season never became contractual. Exec sponsor drops to 3 and expansion surface stays at 3: workflow 3 is scoped but blocked on an unanswered safety sign-off with no sponsor named.",
|
||
"dimensions": [
|
||
{
|
||
"id": "task_definability",
|
||
"weight": 10,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "verifier_quality",
|
||
"weight": 13,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "trace_data_rights",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "baseline_measured",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "headroom",
|
||
"weight": 10,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "env_constructibility",
|
||
"weight": 8,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "metric_owner",
|
||
"weight": 9,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "budget_source",
|
||
"weight": 9,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "exec_sponsor",
|
||
"weight": 4,
|
||
"score": 3
|
||
},
|
||
{
|
||
"id": "security_legal_path",
|
||
"weight": 8,
|
||
"score": 4
|
||
},
|
||
{
|
||
"id": "forcing_function",
|
||
"weight": 8,
|
||
"score": 2
|
||
},
|
||
{
|
||
"id": "expansion_surface",
|
||
"weight": 5,
|
||
"score": 3
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"promote": {
|
||
"artefactTitle": "DEMO — Case study: Northwind Robotics recovery-action policy",
|
||
"intoSlug": "case-study-frame"
|
||
}
|
||
}
|