7 Commits

Author SHA1 Message Date
karti-ai 162c8a2378 fix(web): redirect authenticated login route
CI / publish (push) Failing after 4s
CI / verify (push) Successful in 9m29s
2026-08-28 12:22:23 -07:00
karti 8a6601dfce Re-cut the Motion films in landscape, and hold the camera still
CI / verify (push) Successful in 7m14s
CI / publish (push) Has been skipped
THE FORMAT. The first pair were vertical. That was wrong for this product: PIG
is a desktop application, and reframing it 9:16 either shrinks it past reading
or crops away the sidebar and half the table. These are 1920x1080, like the five
already on the page, and the app is filmed at a real 1920 desktop viewport —
the previous renderer set the viewport to 960 and captured at 2x, which produced
a 1920px image of the COLLAPSED layout, because the viewport is what the product
responds to and the scale factor is only how much detail there is to zoom into.

THE CAMERA. It now holds still, and eases in on the thing the narration is
naming, holds, and eases back out — the way Cap zooms toward a click. The first
cut drifted a slow zoom across every shot, which reads as restless rather than
as emphasis and is the thing people mean when they call a screen recording
annoying. Six of fourteen shots move in the customer film; the rest are locked
off. The curve is a smoothstep either side of a hold, written as a function of
the frame number rather than accumulated through zoompan's own `zoom` variable,
which refers to the previous frame and drifts off its endpoints.

Zoom targets are chosen as the SMALLEST element already in frame, not the last
match in document order. That was wrong in a way only a long page shows: the
Halcyon engagement prints the same phrases twice — once in the score note and
again inside the artefact bodies below — so "kappa 0.54" resolved to a paragraph
two and a half thousand pixels below the fold that no amount of scrolling would
reveal, because an ancestor clips it. Preferring what is on screen keeps the
zoom inside the frame the shot chose; preferring the smallest match makes naming
a figure land on the figure rather than on the card around it.

THE SCRIPTS were rewritten from nothing through a judge panel — three
independent angles each, scored on hook, concreteness, ear, tension and ending,
then synthesised and adversarially checked against the running product.

The second film is the one that was asked for: one customer, end to end. Halcyon
Research, scored 52.8 and scope down on the day of first discovery because
nothing had been measured, through legal and scoping and a budget path, to 82.8
and build once the verifier, the baseline and the environment landed — and the
two dimensions that fell inside that same twenty-five point rise. It closes on
the week-two dev split reading under its trajectory, because the honest thing
about that record is that the gate might not be met and it says so before anyone
asks. It runs 68 seconds; a feature tour is 30 and a story is not.
2026-08-19 02:59:29 -07:00
karti 075cd765c5 Stop seeding illustrative Cap rows onto Learn
CI / verify (push) Successful in 7m14s
CI / publish (push) Has been skipped
The `supply` and `demand` concept tracks each carried one `DEMO — ` card
embedding a public recording from the Cap instance at video.karti.ai. Both are
gone, for two reasons that arrived together.

One of the recordings is no longer there. `sjqqvthbfma27bm` now answers 404 on
both `/s/` and `/embed/`, so "DEMO — What a hold takes off the board" promised
six seconds of teaching and played a dead frame. That is precisely the failure
the comment above these rows was written to record the last time it happened,
and it came back — because the footage lives on an instance this repository does
not control, so no amount of care in this file can keep the claim true. Checked
just now rather than assumed: the surviving id, `1rqq9rk4dpp71fd`, still answers
200.

The other reason is judgement. /learn is the page deliberately shown to people
outside this company, and a `DEMO — ` card sitting beside genuine product
footage makes the whole page read as half-placeholder to exactly the audience it
exists to convince. The previous version of this file argued the opposite — that
an empty track hides the shape of the page — and that was the wrong trade. An
empty concept track renders an empty state saying nothing has been published to
it yet, which is true, and true beats furnished.

`seedLearn` and its call site go with them rather than being left as an empty
array behind live machinery. If those tracks get purpose-shot recordings they
belong in `HOSTED_LEARN_MANIFEST`, rendered by `scripts/learn-film.mjs` like the
platform track, and served by PIG itself.

The two rows were also deleted from production directly, matching what
`clear()` does for prefixed learn rows, so the page is correct now rather than
at the next reseed. The underlying Cap recording is untouched.
2026-08-19 02:22:15 -07:00
karti e0473e7258 Cut two vertical Motion films, and write down the pipeline that makes them
CI / verify (push) Successful in 7m23s
CI / publish (push) Has been skipped
THE PIPELINE. The five existing platform videos were cut by an ad-hoc process
that was never committed, so the first time the UI moved nobody could re-shoot
them — which is the same failure `scripts/screenshots.mjs` was written to stop.
`scripts/learn-film.mjs` is that pipeline, and it reuses both of the screenshot
script's hard-won lessons: the appearance preference is stored server-side and
adopted after hydration, so the theme has to be forced by rewriting the profile
response rather than by seeding localStorage; and per-device layout state has to
be pinned or the framing is whatever a human last left behind.

Order is load-bearing. The narration is rendered first and its MEASURED duration
drives every shot length, because a shot list timed by guess leaves the narrator
talking over a frozen frame. Shot durations are then proportional to the words
spoken over them, so the cut lands on the sentence.

Typography is composed in the browser rather than in ffmpeg. The product's face
is Manrope Variable and drawtext would have fallen back to DejaVu, which reads
as a different company. Playwright renders a transparent chrome layer per shot
and ffmpeg only moves pixels.

Detail shots crop to the content column rather than the whole viewport. The
first attempt cropped a box around the element being talked about, which is
narrower than the column, so it sliced the cards either side of centre and the
frame read as broken rather than as close. Width first, height from the aspect.

CRF is 22, not 18. The frame is a static UI with a slow zoom, and 18 spent
3.7 Mbps — a 15 MB download for a video whose whole point is that somebody opens
it on a phone between meetings.

THE STAGE RAIL BUG, which the films found. Every stage label was losing its last
letters — "QUALIFICATIO", "PROCUREMEN" — with no ellipsis to show for it. Three
attempts to fix that by widening the card did nothing, because the card was
never the thing being measured: the LI holding it had `min-w-0` and no
`shrink-0`, so it took its flex share of 88px while the card inside stayed 160,
and every card was overpainted by the next one. The DOM reported no overflow the
whole time, because there wasn't any — the clipping was one level up. `shrink-0`
moves to the LI, where it belongs, and the rail scrolls as it was always meant
to. The labels also wrap rather than truncate now: they are a fixed vocabulary
of eight words we control, and losing a letter is worse than taking a line.

THE FILMS. Two 9:16 clips for the team, narrated in Karti's cloned voice through
Chatterbox at 1.20x and scored with the platform's own ambient bed, ducked and
loudness-normalised so they do not jump against the five already on the page.
Both open dark and switch to light at the midpoint. Shot lists live beside the
narration in `docs/learn-films/`, because a script and its shot list timed
against each other are one object and splitting them is how they drift.
2026-08-19 01:58:35 -07:00
karti 666310b264 Fill the demo book's motion: eight engagements, one at every stage
CI / verify (push) Successful in 7m20s
CI / publish (push) Has been skipped
The Motion half of the demo book was two engagements, which was enough to show
that the loop works and not enough to show what the product is for. The page
that matters asks whether the motion is repeating, and a stage rail of zeroes
cannot answer it.

Eight engagements now, one at every open stage, hanging off demand deals the
demand book already creates — the pipeline happened to have exactly one open
deal at each of the eight, so no deal was invented and the quoted pipeline
counts are unchanged. Forty-four artefacts, nineteen scores, one promotion.

Three things the book is laid out to prove that a folder of templates cannot.

Every stage is occupied, and every one of the twelve starter templates is
instantiated at least once, so "stages covered 8/8" is a measurement rather
than a claim about the seed.

Scores move, and sometimes move down. Nineteen scores across eight
trajectories, with nine dimensions regressing somewhere — the Verity
fine-tuning record runs 62.5 -> 60.5 -> 84.0 -> 80.8, because a scorecard that
only ever rises is a ratchet and teaches a reader to distrust it. Every score
is computed with `motionScoreBasisPoints` and banded with `motionBand` rather
than written as a literal, so the seed and the product cannot disagree about
what the same dimensions are worth.

The artefact bodies are the customer's own facts — named people, real volumes,
the specific thing going wrong, and a live unresolved risk in each. An artefact
whose body is the template with the blanks still in it is precisely what this
data exists to disprove. Six of the forty-four have no template at all, which
is the honest shape of an engagement and the reason `kind` is carried on the
artefact rather than derived: `engagement_artifacts.kind` is NOT NULL and a
derived kind would have been null for exactly those six.

The bodies live in JSON beside the loader for the same reason the starter
library's do — forty-four markdown bodies as backtick strings is a module
nobody can review.

The promotion copies `body` from the artefact verbatim, as `promoteArtifact`
does, rather than writing a hand-authored version 2. A demo that produced a row
the real path could not have produced would teach the wrong shape of the table.

Two name collisions the authors could not see are fixed: a Quillon contact
shared a full name with a demo seller, and an Aurelian one shared a surname
with another. `usage_count` is raised once per template rather than once per
artefact, so it stays symmetrical with the decrement `clear()` already does —
verified by tearing the book down and confirming the library returns to twelve
templates with every counter back at zero, since a counter left above zero
makes a starter template permanently un-editable.
2026-08-19 01:05:48 -07:00
karti b7d1ffd2d8 Speak the scorecard's own band labels, not a second vocabulary
The Trainability and Deal Qualification Scorecard publishes five bands and an
action for each — Decline, Defer, Scope down, Qualified conditional, Build —
and `MOTION_BANDS` published four different ones with different edges. So a
reader could read the scorecard, score a deal against the exact dimensions it
defines, and be told "Strategic" by a band table that document has never heard
of. Two answers to the same question from the same product.

The scorecard wins, on two grounds. Its edges were chosen alongside the
dimension weights they sit on top of, so 78 means something there and 7500 was
a round number here. And every one of its labels is a verb the reader can act
on: "Qualified" describes a deal, "Scope down" says what to do about it, which
is the only reason to band a score rather than show it.

The labels are now duplicated between the JSON a customer reads and the table
the product renders, because a rendered label cannot reach into a seeded row.
That duplication gets a test asserting the whole table verbatim, so
re-authoring one copy alone fails rather than drifts.

`apps/api/test/motion.test.ts` asserted the literal 'Strategic'. It now derives
the band through the shared function, so a band-table change is caught by the
test that owns the decision instead of by a write-path test that does not.
2026-08-19 00:26:21 -07:00
karti 2f32186d22 Fix five defects found by running Motion rather than reading it
CI / verify (push) Successful in 7m21s
CI / publish (push) Has been skipped
The markdown parser was in the eager entry chunk. `manualChunks` in its
object form does not leave an unlisted vendor package to Vite's async
splitting, so react-markdown was hoisted into the entry even though its only
importers are lazy routes — 327.70 kB gzip against a 314 kB baseline, on the
one download every route pays for. Naming it as its own chunk puts it back
behind the Motion pages and takes the entry to 282.21 kB, below where it was
before Motion existed.

The starter library could never be improved. Seeding was insert-only, so a
deployment seeded in August was frozen on August's wording for ever with no
upgrade path short of editing production rows by hand — for a feature whose
entire premise is that the library gets better. A second run now refreshes a
starter row, but only while it is still ours: `is_system`, `usage_count = 0`
and no owner. That is the same condition §7a already enforces on the API, so
a template an engagement was cut from is left alone and reported by name
rather than silently overwritten.

The refresh was not idempotent, and the seed lied about it. `jsonb` does not
preserve key order — Postgres sorts keys by length then bytewise — so
comparing `JSON.stringify(stored)` against `JSON.stringify(authored)` marked
every template as changed on every run, and the seed rewrote nine rows each
time while reporting itself clean. Comparison is now canonical. Found by
running the seed three times and reading the counts.

`motion-overflow-check.mjs` measured less than it claimed. It seeded
`pig.sidebar` and `pig.piggy.dock`, neither of which anything reads (the keys
are `pig.sidebarOpen` and `pig.piggyDockOpen`), so the layout it pinned was
whatever the last run left. Its dark pass set `colorScheme` only, and the
appearance preference is stored server-side and adopted after hydration, so
the dark pass measured the light palette a beat after first paint. It now
rewrites the profile response as `screenshots.mjs` does, asserts the rendered
`data-theme`, and fails a page that renders almost no text — a page that
throws inside its own body otherwise measures zero overflow and passes.

The stage rail rendered "1 templates", in the visible label and in every
aria-label. Singular and plural are now both passed.

Also normalised `artifact` to `artefact` in the seeded prose, which had
drifted American in the playbook. The `artifacts` field key is untouched:
FieldsView reads it, and already labels it in British.
2026-08-19 00:07:08 -07:00
31 changed files with 2927 additions and 525 deletions
+4
View File
@@ -24,3 +24,7 @@ backups/
# deployment mounts from the host — a release artefact, not source.
media/
/media
# Render working directory for scripts/learn-film.mjs. The finished mp4 and its
# poster are copied into the media directory, which is itself untracked.
.film/
+5 -2
View File
@@ -27,7 +27,7 @@ import { strict as assert } from 'node:assert';
import { describe, it } from 'node:test';
import { getTableName, isSQLWrapper, isTable, type SQL } from 'drizzle-orm';
import { PgDialect } from 'drizzle-orm/pg-core';
import { motionScoreBasisPoints } from '@pig/core';
import { motionBand, motionScoreBasisPoints } from '@pig/core';
import type { Database, MotionTemplate } from '@pig/db';
import { teamMemberships, users } from '@pig/db';
import { Hono } from 'hono';
@@ -783,7 +783,10 @@ describe('qualification scores', () => {
const written = log.inserted.find((row) => row.table === 'qualification_scores')?.row;
assert.equal(written?.basisPoints, motionScoreBasisPoints(dimensions));
assert.equal(written?.band, 'Strategic');
// Derived from the score by the shared function, never asserted as a
// literal here: a band table edited in @pig/core would otherwise be caught
// by this test rather than by the one that owns the decision.
assert.equal(written?.band, motionBand(motionScoreBasisPoints(dimensions)).label);
assert.equal(typeof written?.basisPoints, 'number');
assert.ok(Number.isInteger(written?.basisPoints), 'a score is an integer, exactly as money is');
});
+6 -7
View File
@@ -244,13 +244,11 @@ function AppRoutes() {
{/*
`/` is the front door, and the front door is Piggy.
--------------------------------------------------
Signing in does not navigate: the auth gate simply starts rendering
these routes at whatever address the browser is already on, which for
anyone arriving fresh is `/`. So "land on Piggy after sign-in" and
"`/` is Piggy" are the same sentence, and this is the only line that
decides it. A post-sign-in `navigate()` was rejected: it fires on one
path through the gate and not on a hard refresh, so the product would
open somewhere different depending on how you got there.
Signing in does not navigate from the form: the auth gate starts
rendering these routes at the browser's current address. `/` goes to
Piggy directly, while `/login` has its own authenticated-only redirect
below so a person who used the explicit sign-in URL does not land on
the catch-all Not found page.
It is a redirect rather than Piggy mounted at the index, because the
workspace needs ONE address. Two paths rendering it would leave the
@@ -273,6 +271,7 @@ function AppRoutes() {
into them still resolve directly, with no pass through here.
*/}
<Route index element={<Navigate to="/piggy" replace />} />
<Route path="login" element={<Navigate to="/" replace />} />
<Route path="overview" element={<RoutePage><Overview /></RoutePage>} />
<Route path="margin" element={<RoutePage><Margin /></RoutePage>} />
<Route path="growth" element={<RoutePage><Growth /></RoutePage>} />
+36 -10
View File
@@ -18,12 +18,17 @@
import { DEMAND_OPEN_STAGES, DEMAND_STAGE_LABELS, type DemandStage } from '@pig/core';
import { cn } from '@/components/ui';
/** Zero takes the plural, as it does in English: "no templates", "0 templates". */
function noun(count: number, [one, many]: readonly [string, string]): string {
return count === 1 ? one : many;
}
export function StageRail({
counts,
coverage,
stages = DEMAND_OPEN_STAGES,
countLabel = 'engagements',
coverageLabel = 'templates',
countNoun = ['engagement', 'engagements'],
coverageNoun = ['template', 'templates'],
activeStage,
onSelect,
className,
@@ -33,8 +38,9 @@ export function StageRail({
/** The second figure, if the caller has one — library cover, typically. */
coverage?: Partial<Record<DemandStage, number>>;
stages?: readonly DemandStage[];
countLabel?: string;
coverageLabel?: string;
/** Singular and plural, because "1 engagements" is read aloud by a screen reader. */
countNoun?: readonly [one: string, many: string];
coverageNoun?: readonly [one: string, many: string];
activeStage?: DemandStage | null;
/** Omit to render a read-only rail: a non-interactive button is a trap. */
onSelect?: (stage: DemandStage) => void;
@@ -50,7 +56,17 @@ export function StageRail({
const body = (
<>
<span className="truncate text-xs font-medium uppercase tracking-wide text-muted">
{/*
* Wraps rather than truncates. The stage names are a fixed
* vocabulary of eight words we control, and truncation was
* silently rendering "QUALIFICATIO" and "PROCUREMEN" — losing a
* letter is worse than taking a second line. Two attempts to fix
* this by widening the card (8.5rem, then 10, then 11) each
* looked right in a DOM measurement and still clipped on the
* rendered frame, which is the argument for not depending on a
* magic width at all.
*/}
<span className="text-xs font-medium uppercase leading-tight tracking-wide text-muted">
{DEMAND_STAGE_LABELS[stage]}
</span>
<span className={cn('nums text-2xl font-semibold leading-none', count === 0 && 'text-muted')}>
@@ -61,31 +77,41 @@ export function StageRail({
// surface — it is where the motion stops repeating — so it is
// called out rather than shown as another grey zero.
<span className={cn('nums truncate text-xs', covered === 0 ? 'text-warning' : 'text-muted')}>
{covered === 0 ? `No ${coverageLabel}` : `${covered} ${coverageLabel}`}
{covered === 0 ? `No ${coverageNoun[1]}` : `${covered} ${noun(covered, coverageNoun)}`}
</span>
)}
</>
);
const shape = cn(
'tap flex w-[8.5rem] shrink-0 flex-col justify-between gap-2 rounded-xl border p-3 text-left',
// 10rem, not 8.5, so the common labels sit on one line; the two
// longest wrap to two rather than losing a letter.
'tap flex w-40 shrink-0 flex-col justify-between gap-2 rounded-xl border p-3 text-left',
active ? 'border-brand bg-surface-2' : 'border-border bg-surface',
);
return (
<li key={stage} className="flex min-w-0">
/*
* `shrink-0` belongs on the LI, not only on the card inside it.
* Without it the list item shrank to its flex share — 88px against
* the card's 160 — and every card was overpainted by the next one,
* so the labels lost their last letters with no ellipsis to show
* for it. Three attempts to fix that by widening the card did
* nothing, because the card was never the thing being measured.
*/
<li key={stage} className="flex shrink-0">
{onSelect ? (
<button
type="button"
onClick={() => onSelect(stage)}
aria-pressed={active}
aria-label={`${DEMAND_STAGE_LABELS[stage]}: ${count} ${countLabel}`}
aria-label={`${DEMAND_STAGE_LABELS[stage]}: ${count} ${noun(count, countNoun)}`}
className={cn(shape, 'transition-colors hover:bg-surface-2')}
>
{body}
</button>
) : (
<div className={shape} aria-label={`${DEMAND_STAGE_LABELS[stage]}: ${count} ${countLabel}`}>
<div className={shape} aria-label={`${DEMAND_STAGE_LABELS[stage]}: ${count} ${noun(count, countNoun)}`}>
{body}
</div>
)}
+1 -1
View File
@@ -314,7 +314,7 @@ export function Engagement() {
<StageRail
counts={counts}
stages={railStages}
countLabel="artefacts"
countNoun={['artefact', 'artefacts']}
activeStage={stageFilter ?? engagement.stage}
onSelect={(stage) => setStageFilter((current) => (current === stage ? null : stage))}
/>
+1 -1
View File
@@ -180,7 +180,7 @@ export function MotionLibrary() {
<StageRail
counts={railCounts}
countLabel="templates"
countNoun={['template', 'templates']}
activeStage={filters.stage}
onSelect={(stage) =>
setParam(setParams, STAGE_PARAM, stage === filters.stage ? '' : stage)
+8
View File
@@ -44,6 +44,14 @@ export default defineConfig({
react: ['react', 'react-dom', 'react-router-dom'],
supabase: ['@supabase/supabase-js'],
query: ['@tanstack/react-query'],
/*
* Named explicitly, or the object form of `manualChunks` hoists the
* markdown parser into the entry chunk even though the only importers
* are lazy routes — measured at 327.70 kB gzip entry with it hoisted
* against 314 kB without. As its own chunk it is fetched when a
* Motion page is opened and never on any other route.
*/
markdown: ['react-markdown', 'remark-gfm'],
},
},
},
@@ -0,0 +1,79 @@
{
"slug": "motion-the-repeatable-motion",
"title": "Motion: the part of a deal PIG never knew",
"summary": "PIG tracked the money on every deal and none of the practice — Motion is the library of what we know how to do, and it gets a version better every time a deal closes.",
"shots": [
{
"route": "/motion",
"scrollTo": null,
"theme": "dark",
"line": "Hey team, check out the motion feature on our platform, because the ledger was never the hard part.",
"caption": "The hard part",
"zoom": null
},
{
"route": "/motion",
"scrollTo": null,
"theme": "dark",
"line": "PIG knew every deal we closed. It never knew how we closed it.",
"caption": "Numbers, not practice",
"zoom": { "on": "Promotions", "scale": 1.5 }
},
{
"route": "/motion",
"scrollTo": "The motion, stage by stage",
"theme": "dark",
"line": "Eight stages, seven live engagements, thirteen templates waiting. Legal sits second, because paperwork gates the work.",
"caption": "Eight stages, legal second",
"zoom": null
},
{
"route": "/motion",
"scrollTo": "Library cover, by kind",
"theme": "dark",
"line": "Nine kinds of conversation, not one of them empty. Nobody improvises this on a call.",
"caption": "Nine kinds, none empty",
"zoom": null
},
{
"route": "/motion/library",
"scrollTo": null,
"theme": "light",
"line": "Filter by kind, stage or visibility. Thirteen shared, nothing stranded in a private draft.",
"caption": "The library",
"zoom": null
},
{
"route": "/motion/library/PLAYBOOK",
"scrollTo": "The stages",
"theme": "light",
"line": "This one is the spine. Every stage names its artefact, its exit test, its failure mode.",
"caption": "The spine",
"zoom": null
},
{
"route": "/motion/library/PLAYBOOK",
"scrollTo": null,
"theme": "light",
"line": "Used once, so PIG closes it to edits. You cut version two instead.",
"caption": "Closed to edits",
"zoom": { "on": "closed to edits", "scale": 1.6 }
},
{
"route": "/motion",
"scrollTo": "What got easier",
"theme": "light",
"line": "Northwind's fleet expansion closed. Its case study came back as version two, carrying the numbers.",
"caption": "Promoted back",
"zoom": { "on": "recovery-action policy", "scale": 1.5 }
},
{
"route": "/motion",
"scrollTo": null,
"theme": "light",
"line": "The library is not what we wrote. It is what we have done. Thanks for watching!",
"caption": "What we have done",
"zoom": null
}
]
}
@@ -0,0 +1,137 @@
{
"slug": "one-customer-end-to-end",
"title": "Halcyon Research, and a gate it might miss",
"summary": "One engagement in order: a 52.8% scope down, the artefacts that closed the gaps, and a score that rose 25.5 points while two dimensions fell.",
"shots": [
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Where the work sits",
"theme": "dark",
"line": "Halcyon is three weeks into a paid pilot that might not survive the twenty-eighth of August.",
"caption": "A pilot under pressure",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "dark",
"line": "It started worse. Fifty-two point eight percent, band scope down, scored on discovery day.",
"caption": "The first honest read",
"zoom": {
"on": "52.8%",
"scale": 2.2
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "dark",
"line": "The task repeats and notes trace. No baseline, no rubric, no budget line.",
"caption": "Nothing was measured",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Artefacts",
"theme": "dark",
"line": "It closed them one at a time. Rights in legal, architecture in scoping.",
"caption": "Closed one stage at a time",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "dark",
"line": "Rescore, fifty-seven point three. Only what scoping touched moved. The forcing function fell.",
"caption": "Rescored at 57.3%",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Artefacts",
"theme": "light",
"line": "Then a budget path named the FY27 line and an October committee date.",
"caption": "A budget path appears",
"zoom": {
"on": "Halcyon Research: budget path to the 8 October committee",
"scale": 1.5
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "Then measurement. Verifier at kappa point seven one, against a point seven three ceiling.",
"caption": "Measurement lands",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "Baseline nought point six four, environment up in week one. Eighty-two point eight percent, build.",
"caption": "82.8%, Build",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "Twenty-five and a half points in one step, and two dimensions fell inside.",
"caption": "Two things got worse",
"zoom": {
"on": "+25.5%",
"scale": 1.8
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "Task definability, because Halcyon's adjudicators agree at kappa point five four on oncology.",
"caption": "Adjudicators disagree",
"zoom": {
"on": "kappa 0.54",
"scale": 1.9
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "The sponsor fell too. Beaumont has missed two of three calls before the ask.",
"caption": "Sponsor going quiet",
"zoom": {
"on": "Beaumont has missed",
"scale": 1.9
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": "Qualification",
"theme": "light",
"line": "Week two read nought point six six, under a point six nine trajectory.",
"caption": "The gate is live",
"zoom": {
"on": "week-two dev split",
"scale": 1.9
}
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": null,
"theme": "light",
"line": "It might not make it, and the record said so before anyone asked.",
"caption": "The record said so first",
"zoom": null
},
{
"route": "/motion/engagements/HALCYON",
"scrollTo": null,
"theme": "light",
"line": "Not spun, not chased down. Written down when it happened. Thanks for watching!",
"caption": "Written when it happened",
"zoom": null
}
]
}
+22 -7
View File
@@ -200,19 +200,34 @@ export function motionScoreBasisPoints(dimensions: readonly MotionDimensionScore
return Math.floor((2 * numerator + denominator) / (2 * denominator));
}
/*
* The band labels are the SCORECARD'S OWN, not a second vocabulary invented
* here. `seed/motion/trainability-qualification.json` publishes five bands and
* an action for each, and a reader who has just read that document and then
* sees the score rendered as "Strategic" has been told two different things
* about the same number by the same product. Every label below is a verb the
* reader can act on, which is the point of a band at all: "Qualified" describes
* the deal, "Scope down" says what to do about it.
*
* Boundaries are the authored percentages in basis points. If the scorecard's
* bands are ever re-authored, these move with them — they are one decision
* recorded twice, and the JSON is the copy a customer reads.
*/
export const MOTION_BANDS = [
{ min: 0, max: 3499, label: 'Decline or defer', tone: 'danger' },
{ min: 3500, max: 5499, label: 'Not yet', tone: 'warning' },
{ min: 5500, max: 7499, label: 'Qualified', tone: 'info' },
{ min: 7500, max: 10_000, label: 'Strategic', tone: 'positive' },
{ min: 0, max: 2999, label: 'Decline', tone: 'danger' },
{ min: 3000, max: 4599, label: 'Defer', tone: 'danger' },
{ min: 4600, max: 6199, label: 'Scope down', tone: 'warning' },
{ min: 6200, max: 7799, label: 'Qualified, conditional', tone: 'info' },
{ min: 7800, max: 10_000, label: 'Build', tone: 'positive' },
] as const;
export type MotionBand = (typeof MOTION_BANDS)[number];
export type MotionBandTone = MotionBand['tone'];
/**
* The band a score falls in. Boundaries are inclusive at `min`, so 3500 is
* "Not yet" and 3499 is not — an off-by-one here changes what a seller is told
* to do without changing any number they can see.
* The band a score falls in. Boundaries are inclusive at `min`, so 6200 is
* "Qualified, conditional" and 6199 is "Scope down" — an off-by-one here
* changes what a seller is told to do without changing any number they can
* see.
*
* Out-of-range input clamps to an end band rather than returning undefined,
* because every caller renders this and none of them has a null branch.
+30 -9
View File
@@ -34,7 +34,7 @@ describe('motionScoreBasisPoints', () => {
it('returns 0 rather than throwing when every weight is zero', () => {
// The naive version divides by the weight total and produces Infinity,
// which `motionBand` would then clamp to "Strategic" — a deal nobody
// which `motionBand` would then clamp to "Build" — a deal nobody
// scored recommended as the best one in the book.
const score = motionScoreBasisPoints([
{ id: 'fit', weight: 0, score: 4 },
@@ -112,20 +112,41 @@ describe('motionBand', () => {
* file and tells a seller to defer a deal that qualified.
*/
it('is inclusive at the lower edge of every band', () => {
assert.equal(motionBand(3500).label, 'Not yet');
assert.equal(motionBand(5500).label, 'Qualified');
assert.equal(motionBand(7500).label, 'Strategic');
assert.equal(motionBand(3000).label, 'Defer');
assert.equal(motionBand(4600).label, 'Scope down');
assert.equal(motionBand(6200).label, 'Qualified, conditional');
assert.equal(motionBand(7800).label, 'Build');
});
it('keeps the basis point below each edge in the band underneath', () => {
assert.equal(motionBand(3499).label, 'Decline or defer');
assert.equal(motionBand(5499).label, 'Not yet');
assert.equal(motionBand(7499).label, 'Qualified');
assert.equal(motionBand(2999).label, 'Decline');
assert.equal(motionBand(4599).label, 'Defer');
assert.equal(motionBand(6199).label, 'Scope down');
assert.equal(motionBand(7799).label, 'Qualified, conditional');
});
it('covers both ends of the scale', () => {
assert.equal(motionBand(0).label, 'Decline or defer');
assert.equal(motionBand(10_000).label, 'Strategic');
assert.equal(motionBand(0).label, 'Decline');
assert.equal(motionBand(10_000).label, 'Build');
});
/**
* The scorecard a customer reads publishes these five bands and an action for
* each. They are duplicated in `MOTION_BANDS` because the product renders a
* label the JSON cannot reach — so this pins the two copies together, and
* fails when somebody re-authors one of them alone.
*/
it('uses the labels and edges the seeded scorecard publishes', () => {
assert.deepEqual(
MOTION_BANDS.map((band) => [band.min, band.max, band.label]),
[
[0, 2999, 'Decline'],
[3000, 4599, 'Defer'],
[4600, 6199, 'Scope down'],
[6200, 7799, 'Qualified, conditional'],
[7800, 10_000, 'Build'],
],
);
});
it('leaves no gap and no overlap across the whole range', () => {
+8 -5
View File
@@ -55,7 +55,7 @@ import { seedCalendar } from './calendar';
import { seedCompliance } from './compliance';
import { seedDemandPaper } from './contracts';
import { seedDemand } from './demand';
import { seedHostedLearn, seedLearn } from './learn';
import { seedHostedLearn } from './learn';
import { seedMotionEngagements } from './motion';
import { seedSupply } from './supply';
@@ -167,7 +167,6 @@ export async function seedDemo(context: DemoContext): Promise<void> {
const ownerUserId = owner?.id ?? null;
const calendar = await seedCalendar(context, ownerUserId);
const learn = await seedLearn(context, ownerUserId);
// The self-hosted set, which is real rather than invented — see the long
// note on `seedHostedLearn`. Folded into the demo seed so one command gives
@@ -220,14 +219,18 @@ export async function seedDemo(context: DemoContext): Promise<void> {
`${compliance.decisions.total} export-control decisions (allow, needs_review, block), ` +
`${calendar.total} calendar entries (${calendar.added} new)`,
);
console.log(
` ${learn.total} illustrative concept videos (${learn.added} new), members-only`,
);
console.log(
` ${motion.engagements} Motion engagement(s) with ${motion.artifacts} artefact(s) and ` +
`${motion.scores} qualification score(s) — ${motion.promoted} artefact promoted back into ` +
'the library as a version 2',
);
if (motion.missingDeals.length > 0) {
console.warn(
` ${motion.missingDeals.length} Motion engagement(s) had no deal to hang off and were ` +
`SKIPPED: ${motion.missingDeals.join(', ')}. Run \`pnpm db:demo -- --clear\` and reseed ` +
'to rebuild the demand book.',
);
}
console.log(
` ${hosted.present} PIG-hosted learn videos (${hosted.added} new)` +
`${hosted.missing > 0 ? `, ${hosted.missing} manifest entries with no file yet` : ''}`,
+55 -114
View File
@@ -1,10 +1,28 @@
/**
* The Learn library: the illustrative demo rows, and the PIG-hosted real ones.
* The Learn library: the PIG-hosted recordings.
*
* They sit in one module because they write to one table and share the
* idempotency key, but they are not the same kind of data: the first set is
* invented and prefixed, the second is genuine product footage and survives
* `--clear`.
* There used to be two illustrative rows here as well, embedding public
* recordings from the Cap instance at video.karti.ai under `DEMO — ` titles, so
* that the `supply` and `demand` concept tracks were not empty. Both are gone,
* for two reasons that arrived together.
*
* One of the two recordings had since been removed from that instance:
* `sjqqvthbfma27bm` answers 404 on both `/s/` and `/embed/`, so the card
* promised six seconds of teaching and played a dead frame. That is the exact
* failure the previous version of this comment was written to record — an id
* that 404s behind a card that claims content — and it came back because the
* footage lives somewhere this repository does not control.
*
* The other is judgement: /learn is the page deliberately shown to outsiders,
* and a `DEMO — ` card sitting beside genuine product footage makes the whole
* page read as half-placeholder to exactly the audience it exists to convince.
* An empty concept track renders a proper empty state saying nothing has been
* published yet, which is true and says so.
*
* If `supply` and `demand` get purpose-shot recordings, they belong in
* `HOSTED_LEARN_MANIFEST` below and rendered by `scripts/learn-film.mjs` like
* the platform track, not embedded from an instance whose contents can change
* underneath this file.
*/
import { LEARN_MEDIA_PATH_PREFIX, isLearnMediaFilename, learnMediaContentType } from '@pig/core';
import { eq } from 'drizzle-orm';
@@ -13,115 +31,6 @@ import { resolve } from 'node:path';
import { learnResources, users } from '../../schema/index';
import type { DemoContext } from './index';
/**
* Every id below is a REAL public recording on the Cap instance at
* video.karti.ai, and every duration is the MEASURED length of the file that
* embed plays. Both were re-checked against the instance — `/s/<id>` and
* `/embed/<id>` answer 200, and the recordings are 60.4s and 6.2s — because
* this list previously stated neither truthfully.
*
* It used to carry four rows, two of them sharing the id `0n6n9p83efnxbs2`
* under different titles and different stated lengths. That id is not a
* recording at all — it 404s, and the only other place it appears in this
* repository is as the fixture string in `apps/api/test/learn.test.ts`, which
* is almost certainly where it was copied from. So two cards each promised a
* quarter of an hour of teaching and played nothing, on the one page that is
* deliberately shown to outsiders. A card that lies about its own content is
* worse than a track with one card in it, and there are exactly two public
* recordings on that instance — so there are exactly two rows here, one per
* concept track.
*
* The titles remain illustrative and prefixed, which is the standing bargain
* for demo rows: the concepts they name are the ones this business actually
* teaches, and the footage behind them is whatever genuinely exists. The
* bargain only holds while the stated LENGTH is true, since that is the one
* claim a viewer can check before pressing play.
*
* PLATFORM rows are not seeded here. The five real recordings in
* HOSTED_LEARN_MANIFEST cover that track, and illustrative Cap rows beneath
* genuine ones made the page read as half-placeholder to the exact audience it
* is meant to convince.
*
* The concept rows stay: `supply` and `demand` have no purpose-shot recordings
* yet, and an empty track hides the shape of the page. They are
* `members`-visible, which the CHECK constraint enforces anyway — only
* `platform` may be `code`.
*/
export async function seedLearn(
context: DemoContext,
ownerUserId: string | null,
): Promise<{ total: number; added: number }> {
const { db, prefix } = context;
const LEARN_RESOURCES = [
{
track: 'supply' as const,
title: `${prefix}How neocloud capacity is actually priced`,
summary: 'Reserved versus on-demand, commitment length, and where the spread comes from.',
externalId: '1rqq9rk4dpp71fd',
visibility: 'members' as const,
// 60.4s on the wire, rounded down: a duration that overstates by a
// second is the same class of claim as one that overstates by minutes.
durationSeconds: 60,
sortOrder: 10,
},
{
track: 'demand' as const,
title: `${prefix}What a hold takes off the board`,
summary:
'A hold reserves hours nobody else can be quoted, and it is not revenue until it converts.',
externalId: 'sjqqvthbfma27bm',
visibility: 'members' as const,
durationSeconds: 6,
sortOrder: 10,
},
];
/*
* The unique key is (track, provider, external_id), so the SAME recording on
* TWO tracks inserts perfectly happily — which is how one id came to sit
* behind two different titles and two different stated durations for as long
* as it did. The database cannot catch that; this can. Thrown before any
* insert, and loudly, because it is a typo in a literal rather than a
* condition of the environment: there is nothing for an operator to fix at
* run time and nothing worth continuing past.
*/
const ids = LEARN_RESOURCES.map((resource) => resource.externalId);
if (new Set(ids).size !== ids.length) {
throw new Error(
'Two demo learn resources share an external id — one of them would be a lie about its own content.',
);
}
let learnAdded = 0;
for (const resource of LEARN_RESOURCES) {
// Idempotent on the unique key rather than an existence check, which is
// the whole reason that constraint exists: onConflictDoNothing without one
// is a silent no-op and has duplicated seed data here twice before.
const inserted = await db
.insert(learnResources)
.values({
track: resource.track,
title: resource.title,
summary: resource.summary,
url: `https://video.karti.ai/s/${resource.externalId}`,
provider: 'cap',
externalId: resource.externalId,
visibility: resource.visibility,
durationSeconds: resource.durationSeconds,
sortOrder: resource.sortOrder,
addedByUserId: ownerUserId,
})
.onConflictDoNothing({
target: [learnResources.track, learnResources.provider, learnResources.externalId],
})
.returning({ id: learnResources.id });
if (inserted.length) learnAdded += 1;
}
return { total: LEARN_RESOURCES.length, added: learnAdded };
}
// ------------------------------------------------------- PIG-hosted learn
//
// Real videos, served by PIG itself from PIG_MEDIA_DIR — not demo data. They
@@ -239,6 +148,38 @@ export const HOSTED_LEARN_MANIFEST: readonly HostedLearnEntry[] = [
durationSeconds: 28,
sortOrder: 5,
},
/*
* Six and seven are landscape, like the five above, and cut by
* `scripts/learn-film.mjs`. A vertical pair was tried first and rejected:
* the product is a desktop application, and reframing it 9:16 either shrinks
* it past reading or crops away the sidebar and half the table. The camera
* holds still and eases in on the figure being named, rather than drifting
* across every shot, which is what made the first cut tiring to watch.
*
* Seven is longer than the rest on purpose — it walks one customer from the
* first discovery to a kill gate that may not be met, and that is a story
* rather than a feature tour.
*/
{
slug: 'motion-the-repeatable-motion',
title: 'Motion: the part of a deal PIG never knew',
summary:
'PIG knew every deal we closed. It never knew how we closed it. The library of what we know how to do, bound to the stages of a real deal.',
track: 'platform',
visibility: 'code',
durationSeconds: 47,
sortOrder: 6,
},
{
slug: 'one-customer-end-to-end',
title: 'Halcyon Research, and a gate it might miss',
summary:
'One customer end to end: scored 52.8 and scope down on discovery day, 82.8 and build once measurement landed, and two dimensions that fell on the way up.',
track: 'platform',
visibility: 'code',
durationSeconds: 68,
sortOrder: 7,
},
];
/**
+223 -324
View File
@@ -1,30 +1,48 @@
/**
* Two demo engagements, chosen so the loop is visible without a live customer.
* The demo book's Motion half: eight engagements, one at every open stage.
*
* The loop is the only reason Motion exists:
*
* library template --instantiate--> engagement artifact --promote--> v2
* library template --instantiate--> engagement artefact --promote--> v2
*
* A screenshot of the library alone shows a folder of documents, which is not
* the claim. So one engagement is **mid-POC with three qualification scores**,
* because a single score is a number and three are a trajectory — the movement
* is the evidence that qualification happened at all — and the other is
* **closed-won with a promoted artifact**, which puts a version 2 into the
* library carrying `supersedes_id` back to the shipped version 1 and
* `origin_artifact_id` back to the engagement that proved it. Both halves of
* the FK cycle are therefore populated on a first run.
* the claim being made. So the book is laid out to prove three things a folder
* cannot. **Every stage is occupied**, because the page that matters asks
* whether the motion is repeating and a rail of zeroes cannot answer it.
* **Scores move, and sometimes move down** — a single score is a number and
* four are a trajectory; a scorecard that only ever rises is a ratchet, so the
* Verity fine-tuning record deliberately goes 62.5 → 60.5 → 84.0 → 80.8, and
* nine dimensions regress somewhere in the book. And **one engagement closes
* the loop**, putting a version 2 into the library carrying `supersedes_id`
* back to the shipped version 1 and `origin_artifact_id` back to the artefact
* that proved it, so both halves of the FK cycle are populated on a first run.
*
* Both hang off demand deals the demand book already created, rather than deals
* of their own: an engagement has no independent existence, and inventing a
* deal here would put a thirteenth row on a pipeline whose counts are quoted in
* `demo/index.ts`.
* The artefact bodies are the point. An artefact whose body is the template
* with the blanks still in it is exactly what this data exists to disprove, so
* every one of them is written as the customer's own facts — named people, real
* volumes, the specific thing going wrong. That is also why they live in JSON
* beside this file rather than in TypeScript literals: forty-four markdown
* bodies as backtick strings would be a module nobody can review.
*
* The scores are computed with `motionScoreBasisPoints` rather than written as
* literals. Hard-coding 6600 would let the seed and the product disagree about
* what the same dimensions are worth, and the first place anyone would notice
* is a demo.
* Every engagement hangs off a demand deal the demand book already created,
* rather than a deal of its own. An engagement has no independent existence,
* and inventing deals here would put rows on a pipeline whose counts are quoted
* in `demo/index.ts`.
*
* Scores are computed with `motionScoreBasisPoints` and banded with
* `motionBand`, never written as literals. Hard-coding 8280 would let the seed
* and the product disagree about what the same dimensions are worth, and the
* first place anyone would notice is a demo.
*/
import { motionBand, motionScoreBasisPoints, type MotionDimensionScore } from '@pig/core';
import {
motionBand,
motionScoreBasisPoints,
type ArtifactStatus,
type DemandStage,
type EngagementStatus,
type MotionDimensionScore,
type MotionKind,
} from '@pig/core';
import { and, eq, like, sql } from 'drizzle-orm';
import {
demandDeals,
@@ -36,120 +54,83 @@ import {
} from '../../schema/index';
import type { DemoContext } from './index';
/**
* The scorecard's dimensions and their authored weights, copied from
* `seed/motion/trainability-qualification.json`.
*
* Copied rather than read out of the seeded row, deliberately: a stored score
* is a snapshot of the framework *as it was scored*, which is the whole reason
* `qualification_scores.dimensions` holds the weights rather than a reference.
* Reading them live would make the demo history silently re-weight itself the
* day someone edits the template, which is precisely the behaviour the column
* exists to prevent.
*/
const SCORECARD_WEIGHTS: readonly { readonly id: string; readonly weight: number }[] = [
{ id: 'task_definability', weight: 10 },
{ id: 'verifier_quality', weight: 13 },
{ id: 'trace_data_rights', weight: 8 },
{ id: 'baseline_measured', weight: 8 },
{ id: 'headroom', weight: 10 },
{ id: 'env_constructibility', weight: 8 },
{ id: 'metric_owner', weight: 9 },
{ id: 'budget_source', weight: 9 },
{ id: 'exec_sponsor', weight: 4 },
{ id: 'security_legal_path', weight: 8 },
{ id: 'forcing_function', weight: 8 },
{ id: 'expansion_surface', weight: 5 },
];
import aurelianCommittedCapacity from './motion/aurelian-committed-capacity.json' with { type: 'json' };
import halcyonPostTraining from './motion/halcyon-post-training.json' with { type: 'json' };
import northwindBlackwell from './motion/northwind-blackwell.json' with { type: 'json' };
import northwindFleetExpansion from './motion/northwind-fleet-expansion.json' with { type: 'json' };
import quillonEvalHarness from './motion/quillon-eval-harness.json' with { type: 'json' };
import tessellateBurstInference from './motion/tessellate-burst-inference.json' with { type: 'json' };
import verityEuFineTuning from './motion/verity-eu-fine-tuning.json' with { type: 'json' };
import verityEuInference from './motion/verity-eu-inference.json' with { type: 'json' };
/**
* Three passes over the same scorecard, in the order they were run.
*
* The trajectory is the teaching: the deal opens borderline on an unmeasured
* verifier and no named budget, the POC scoping call fixes the verifier, and
* the third pass lands once finance names a source. Two dimensions deliberately
* do NOT move — `trace_data_rights` and `expansion_surface` — because a
* scorecard where every number rises on every pass is a scorecard nobody is
* really filling in.
interface DemoArtefact {
readonly templateSlug: string | null;
/*
* Carried on the artefact rather than looked up from its template, because
* six of the forty-four have no template at all — the work a lead does that
* no library entry covers, which is the honest shape of an engagement. A
* `kind` derived from the template would have been null for exactly those
* six, and `engagement_artifacts.kind` is NOT NULL.
*/
const SCORE_PASSES: readonly {
readonly kind: MotionKind;
readonly stage: DemandStage;
readonly title: string;
readonly status: ArtifactStatus;
readonly authoredDaysAgo: number;
readonly body: string;
}
interface DemoScore {
readonly daysAgo: number;
readonly note: string;
readonly scores: Readonly<Record<string, number>>;
}[] = [
{
daysAgo: 52,
note:
'First pass, straight out of technical discovery. The task is well specified and there ' +
'is real headroom, but the verifier is a rubric nobody has run against human labels and ' +
'no budget line has been named. Not yet — and the two things to fix are explicit.',
scores: {
task_definability: 3,
verifier_quality: 1,
trace_data_rights: 2,
baseline_measured: 2,
headroom: 3,
env_constructibility: 2,
metric_owner: 2,
budget_source: 1,
exec_sponsor: 2,
security_legal_path: 2,
forcing_function: 2,
expansion_surface: 3,
},
},
{
daysAgo: 27,
note:
'Re-scored after the POC scoping call. They ran the judge against 300 human-labelled ' +
'items and the baseline is now measured rather than remembered. Budget is still a ' +
'reallocation nobody has signed.',
scores: {
task_definability: 3,
verifier_quality: 3,
trace_data_rights: 2,
baseline_measured: 3,
headroom: 3,
env_constructibility: 3,
metric_owner: 3,
budget_source: 1,
exec_sponsor: 2,
security_legal_path: 3,
forcing_function: 2,
expansion_surface: 3,
},
},
{
daysAgo: 6,
note:
'Third pass, in POC week two. Finance named the source and the VP Engineering owns the ' +
'metric in their own planning doc. Data rights are unchanged: the customer still cannot ' +
'export traces beyond the pilot without a DPA amendment, and that is the live risk.',
scores: {
task_definability: 4,
verifier_quality: 3,
trace_data_rights: 2,
baseline_measured: 3,
headroom: 4,
env_constructibility: 3,
metric_owner: 4,
budget_source: 3,
exec_sponsor: 3,
security_legal_path: 3,
forcing_function: 3,
expansion_surface: 3,
},
},
];
function dimensionsFor(scores: Readonly<Record<string, number>>): MotionDimensionScore[] {
return SCORECARD_WEIGHTS.map((dimension) => ({
id: dimension.id,
weight: dimension.weight,
score: scores[dimension.id] ?? 0,
}));
readonly dimensions: readonly MotionDimensionScore[];
}
interface DemoEngagement {
readonly key: string;
readonly dealName: string;
readonly accountName: string;
readonly stage: DemandStage;
readonly status: EngagementStatus;
readonly summary: string;
readonly openedDaysAgo: number;
readonly artefacts: readonly DemoArtefact[];
readonly scores: readonly DemoScore[];
readonly promote: { readonly artefactTitle: string; readonly intoSlug: string } | null;
}
/**
* Ordered oldest-opened first, so the engagement list reads as a book somebody
* built up rather than eight things that appeared at once.
*/
const DEMO_ENGAGEMENTS = [
northwindFleetExpansion,
northwindBlackwell,
halcyonPostTraining,
verityEuFineTuning,
verityEuInference,
aurelianCommittedCapacity,
tessellateBurstInference,
quillonEvalHarness,
] as unknown as readonly DemoEngagement[];
/**
* Which demo seller owns which account.
*
* Assigned by account rather than by engagement, because the two accounts that
* appear twice (Verity Health AI, Northwind Robotics) are one relationship each
* — a second engagement handed to a different owner would contradict the
* continuity the artefacts themselves describe.
*/
const OWNER_BY_ACCOUNT: Readonly<Record<string, string>> = {
'Quillon AI': 'Wren Abbot',
'Aurelian Systems': 'Marcus Oyelaran',
'Verity Health AI': 'Ines Fabre',
'Tessellate Labs': 'Rosalind Achebe',
'Halcyon Research': 'Ines Fabre',
'Northwind Robotics': 'Marcus Oyelaran',
};
async function findDeal(context: DemoContext, name: string): Promise<string | undefined> {
const [deal] = await context.db
.select({ id: demandDeals.id })
@@ -168,12 +149,17 @@ async function findSeller(context: DemoContext, name: string): Promise<string |
return seller?.id ?? null;
}
/** The shipped version 1 of a lineage. Promotion needs its id and its version. */
async function findTemplate(
context: DemoContext,
slug: string,
): Promise<{ id: string; version: number } | undefined> {
): Promise<{ id: string; version: number; summary: string } | undefined> {
const [template] = await context.db
.select({ id: motionTemplates.id, version: motionTemplates.version })
.select({
id: motionTemplates.id,
version: motionTemplates.version,
summary: motionTemplates.summary,
})
.from(motionTemplates)
.where(and(eq(motionTemplates.slug, slug), eq(motionTemplates.isSystem, true)))
.limit(1);
@@ -182,19 +168,25 @@ async function findTemplate(
export async function seedMotionEngagements(
context: DemoContext,
): Promise<{ engagements: number; artifacts: number; scores: number; promoted: number }> {
): Promise<{
engagements: number;
artifacts: number;
scores: number;
promoted: number;
missingDeals: readonly string[];
}> {
const { db, prefix, at } = context;
/*
* The whole section is skipped if either engagement already exists.
* The whole section is skipped if any demo engagement already exists.
*
* `engagements_demand_deal_key` would refuse a duplicate engagement, but
* nothing would refuse a second set of artifacts or — worse — a second
* nothing would refuse a second set of artefacts or — worse — a second
* promotion, and a second promotion writes case-study-frame v3 on top of the
* v2 this seed already created. `qualification_scores` is append-only by
* design and so has no constraint to conflict on at all; three more rows on
* every run would turn the trajectory into noise. An existence check on the
* parent is the only thing that covers all three.
* design and so has no constraint to conflict on at all; nineteen more rows
* on every run would turn every trajectory into noise. An existence check on
* the parent is the only thing that covers all three.
*/
const existing = await db
.select({ id: engagements.id })
@@ -202,228 +194,144 @@ export async function seedMotionEngagements(
.where(like(engagements.summary, `${prefix}%`))
.limit(1);
if (existing.length > 0) {
return { engagements: 0, artifacts: 0, scores: 0, promoted: 0 };
return { engagements: 0, artifacts: 0, scores: 0, promoted: 0, missingDeals: [] };
}
const playbook = await findTemplate(context, 'strategic-deployment-playbook');
const scorecard = await findTemplate(context, 'trainability-qualification');
const pocTemplate = await findTemplate(context, 'post-training-poc');
const caseStudy = await findTemplate(context, 'case-study-frame');
const templateCache = new Map<string, { id: string; version: number; summary: string }>();
const templateBySlug = async (slug: string) => {
if (!templateCache.has(slug)) {
const found = await findTemplate(context, slug);
if (found) templateCache.set(slug, found);
}
return templateCache.get(slug);
};
const usageByTemplate = new Map<string, number>();
const missingDeals: string[] = [];
let engagementCount = 0;
let artifactCount = 0;
let scoreCount = 0;
let promotedCount = 0;
// ------------------------------------------------- mid-POC, still scoring
const halcyonDealId = await findDeal(context, `${prefix}Managed post-training run`);
const ines = await findSeller(context, 'Ines Fabre');
for (const demo of DEMO_ENGAGEMENTS) {
/*
* A missing deal is reported, not skipped in silence.
*
* Both engagements hang off deals the demand book creates, and that book
* Every engagement hangs off a deal the demand book creates, and that book
* skips an account it has already seen — so a database where the accounts
* survive but the deals were removed leaves these lookups empty and this
* whole section a no-op that reports zeroes and looks like a bug in the
* loader. It was found exactly that way, on a shared development database.
*/
if (!halcyonDealId) {
console.warn(
` No "${prefix}Managed post-training run" deal — skipping the mid-POC engagement. ` +
'Run `pnpm db:demo -- --clear` and reseed to rebuild the demand book.',
);
const dealId = await findDeal(context, demo.dealName);
if (!dealId) {
missingDeals.push(demo.dealName);
continue;
}
if (halcyonDealId) {
const ownerName = OWNER_BY_ACCOUNT[demo.accountName.replace(prefix, '')];
const ownerId = ownerName ? await findSeller(context, ownerName) : null;
const [engagement] = await db
.insert(engagements)
.values({
demandDealId: halcyonDealId,
demandDealId: dealId,
playbookTemplateId: playbook?.id ?? null,
ownerUserId: ines,
status: 'open',
summary: `${prefix}Post-training motion on the Halcyon summarisation workflow — POC week two.`,
openedAt: at(-58),
ownerUserId: ownerId,
status: demo.status,
summary: demo.summary,
openedAt: at(-demo.openedDaysAgo),
closedAt: demo.status === 'won' ? at(-Math.floor(demo.openedDaysAgo / 8)) : null,
})
.returning({ id: engagements.id });
if (engagement) {
if (!engagement) continue;
engagementCount += 1;
await db.insert(engagementArtifacts).values({
engagementId: engagement.id,
templateId: scorecard?.id ?? null,
kind: 'discovery',
stage: 'qualification',
title: `${prefix}Discovery notes — Halcyon summarisation`,
body:
'## What they are trying to do\n\n' +
'Reduce reviewer time on internal research summaries. Today a senior analyst reads a ' +
'40-page source and writes a one-page brief; the bar is "would the head of research ' +
'send this out unedited".\n\n' +
'## Unit of work\n\n' +
'One source document in, one brief out. They have 4,100 historical pairs and the ' +
'reviewer decision on each.\n\n' +
'## Verifier\n\n' +
'A model judge calibrated against 300 of those human decisions. Agreement is 0.81 ' +
'Cohens kappa, measured on a held-out slice rather than the calibration set.\n\n' +
'## The live risk\n\n' +
'Trace export beyond the pilot needs a DPA amendment their counsel has not seen yet.',
status: 'final',
authoredByUserId: ines,
fields: { source: 'Technical discovery call, plus the follow-up with the data owner.' },
});
await db.insert(engagementArtifacts).values({
engagementId: engagement.id,
templateId: pocTemplate?.id ?? null,
kind: 'poc',
stage: 'poc',
title: `${prefix}POC plan — summarisation reward model`,
body:
'## Success criterion, agreed in writing\n\n' +
'Judge-scored acceptance rate on a held-out set of 400 briefs rises from the measured ' +
'baseline of 61% to at least 78%, at no more than 1.4× the current inference cost ' +
'per brief.\n\n' +
'## What we run\n\n' +
'Two weeks on 32× H100. Week one builds the environment and reproduces the ' +
'baseline; week two is the training run and the evaluation.\n\n' +
'## What we hand back\n\n' +
'The environment, the eval harness, and the number — whichever way it comes out. ' +
'A POC that reports a failure honestly is what makes the next number believable.\n\n' +
'## Open\n\n' +
'Trace rights beyond the pilot. Blocked on the DPA amendment.',
status: 'review',
authoredByUserId: ines,
fields: { baselineAcceptancePct: 61, targetAcceptancePct: 78, gpuCount: 32, weeks: 2 },
});
artifactCount += 2;
for (const pass of SCORE_PASSES) {
const dimensions = dimensionsFor(pass.scores);
const basisPoints = motionScoreBasisPoints(dimensions);
await db.insert(qualificationScores).values({
engagementId: engagement.id,
frameworkTemplateId: scorecard?.id ?? null,
// The column is `Record<string, unknown>[]`; Drizzle's jsonb `$type`
// will not take an interface with readonly properties, and widening
// the column to suit the seed would lose the shape everywhere else.
dimensions: dimensions as unknown as Record<string, unknown>[],
basisPoints,
band: motionBand(basisPoints).label,
note: `${prefix}${pass.note}`,
scoredByUserId: ines,
scoredAt: at(-pass.daysAgo),
});
scoreCount += 1;
}
}
}
// ------------------------------------- closed won, and promoted afterwards
const northwindDealId = await findDeal(context, `${prefix}A100 inference pilot`);
const marcus = await findSeller(context, 'Marcus Oyelaran');
if (!northwindDealId) {
console.warn(
` No "${prefix}A100 inference pilot" deal — skipping the closed-won engagement, and ` +
'with it the promotion that demonstrates the loop.',
);
}
if (northwindDealId) {
const [engagement] = await db
.insert(engagements)
.values({
demandDealId: northwindDealId,
playbookTemplateId: playbook?.id ?? null,
ownerUserId: marcus,
status: 'won',
summary: `${prefix}Northwind inference pilot — won, and written up.`,
openedAt: at(-190),
closedAt: at(-96),
})
.returning({ id: engagements.id });
if (engagement) {
engagementCount += 1;
const [artifact] = await db
const artefactIdByTitle = new Map<string, { id: string; body: string }>();
for (const artefact of demo.artefacts) {
const template = artefact.templateSlug ? await templateBySlug(artefact.templateSlug) : undefined;
const [written] = await db
.insert(engagementArtifacts)
.values({
engagementId: engagement.id,
templateId: caseStudy?.id ?? null,
kind: 'case_study',
stage: 'expansion',
title: `${prefix}Northwind Robotics — A100 inference pilot`,
body:
'## The problem\n\n' +
'Northwind ran perception inference on reserved A100s they had bought for training, ' +
'and were paying training prices for a serving workload with a 40:1 peak-to-trough ' +
'shape.\n\n' +
'## What we did\n\n' +
'Moved the trough onto committed capacity and the peak onto burst, with the split ' +
'set by their own 90-day request trace rather than by a headline number.\n\n' +
'## The number\n\n' +
'Cost per million inferences fell 34%. Measured against their own billing, over a ' +
'full quarter, not against a list price.\n\n' +
'## What made it work, and what to reuse\n\n' +
'They had the request trace. Every engagement since has asked for it in the first ' +
'call, which is the change this write-up put into the frame itself.',
status: 'final',
authoredByUserId: marcus,
fields: {
metric: 'Cost per million inferences',
improvementPct: 34,
window: 'One quarter, customer billing',
approvedForExternalUse: false,
},
templateId: template?.id ?? null,
kind: artefact.kind,
stage: artefact.stage,
title: artefact.title,
body: artefact.body,
status: artefact.status,
authoredByUserId: ownerId,
createdAt: at(-artefact.authoredDaysAgo),
updatedAt: at(-artefact.authoredDaysAgo),
})
.returning({ id: engagementArtifacts.id, title: engagementArtifacts.title });
.returning({ id: engagementArtifacts.id });
if (!written) continue;
artifactCount += 1;
artefactIdByTitle.set(artefact.title, { id: written.id, body: artefact.body });
if (template) {
usageByTemplate.set(template.id, (usageByTemplate.get(template.id) ?? 0) + 1);
}
}
for (const score of demo.scores) {
const basisPoints = motionScoreBasisPoints(score.dimensions);
await db.insert(qualificationScores).values({
engagementId: engagement.id,
frameworkTemplateId: (await templateBySlug('trainability-qualification'))?.id ?? null,
/*
* Copied, not referenced. `qualification_scores.dimensions` stores the
* framework AS IT WAS SCORED — weights included — so that re-authoring
* the scorecard later cannot retroactively change what a past score
* meant. Handing the same array to every row would make that snapshot
* a shared object.
*/
dimensions: score.dimensions.map((dimension) => ({ ...dimension })),
basisPoints,
band: motionBand(basisPoints).label,
note: score.note,
scoredByUserId: ownerId,
scoredAt: at(-score.daysAgo),
});
scoreCount += 1;
}
/*
* The promotion, written by hand rather than by calling the API service:
* the seed has no HTTP surface and no principal. It follows the same
* rules the service does — version = previous + 1, `supersedesId` to the
* shipped v1, `originArtifactId` to the artifact, `visibility: 'shared'`,
* `isSystem: false` — because a demo that produced a row the real path
* could not have produced would teach the wrong shape of the table.
* the seed has no HTTP surface and no principal. It follows the same rules
* `promoteArtifact` does — version = newest + 1, `supersedesId` to the
* shipped v1, `originArtifactId` to the artefact, **`body` copied from the
* artefact verbatim**, `visibility: 'shared'`, `isSystem: false` — because
* a demo that produced a row the real path could not have produced would
* teach the wrong shape of the table.
*
* `isSystem` is false and the title is prefixed, which is what lets
* `clear()` find it. It is a version 2 of a shipped lineage, so removing
* the demo book genuinely returns the library to what ships.
*/
if (artifact && caseStudy) {
if (demo.promote) {
const artefact = artefactIdByTitle.get(demo.promote.artefactTitle);
const supersedes = await templateBySlug(demo.promote.intoSlug);
if (artefact && supersedes) {
const [promoted] = await db
.insert(motionTemplates)
.values({
kind: 'case_study',
stage: 'expansion',
slug: 'case-study-frame',
version: caseStudy.version + 1,
title: `${prefix}Case Study Frame — ask for the request trace first`,
slug: demo.promote.intoSlug,
version: supersedes.version + 1,
title: demo.promote.artefactTitle,
summary:
'Version 2, promoted out of the Northwind pilot write-up. Adds the one question ' +
'that made that engagement measurable: get the customers own usage trace ' +
'before anyone quotes a saving.',
body:
'_Promoted from an engagement. The change from version 1 is the section below._\n\n' +
'## Before you frame anything: get the trace\n\n' +
'A case study is only as good as the baseline it is measured against, and the ' +
'only baseline a customer cannot argue with afterwards is their own telemetry. ' +
'Ask for it in the first call, not at write-up time.\n\n' +
'## Everything else\n\n' +
'As version 1: the problem, what we did, the number, what to reuse.',
fields: {
promotedFrom: artifact.title,
addedSection: 'Before you frame anything: get the trace',
},
`Version ${supersedes.version + 1}, promoted out of the ` +
`${demo.accountName.replace(prefix, '')} expansion. The frame as it was actually ` +
'filled in, with the numbers the customer approved for named use.',
body: artefact.body,
fields: { promotedFrom: demo.promote.artefactTitle, engagement: demo.key },
visibility: 'shared',
ownerUserId: marcus,
supersedesId: caseStudy.id,
originArtifactId: artifact.id,
ownerUserId: ownerId,
supersedesId: supersedes.id,
originArtifactId: artefact.id,
isSystem: false,
})
.onConflictDoNothing({ target: [motionTemplates.slug, motionTemplates.version] })
@@ -433,7 +341,7 @@ export async function seedMotionEngagements(
await db
.update(engagementArtifacts)
.set({ promotedTemplateId: promoted.id })
.where(eq(engagementArtifacts.id, artifact.id));
.where(eq(engagementArtifacts.id, artefact.id));
promotedCount += 1;
}
}
@@ -441,21 +349,11 @@ export async function seedMotionEngagements(
}
/*
* `usage_count` is what the API reads to refuse a `PATCH` on a used
* template, so the demo must move it or a library where three templates have
* visibly been instantiated still reads as never used — and the rule the
* whole feature turns on would be undemonstrable. Incremented in one sweep at
* the end rather than per insert, so the count is derived from the artefacts
* that actually landed and `clear()` can undo it by counting the same rows.
* `usage_count` is raised once per template at the end rather than once per
* artefact inside the loop, because it is the counter that makes a template
* permanently un-editable and `clear()` gives back exactly what was borrowed.
* One statement per template keeps the two halves symmetrical.
*/
const instantiated = await db
.select({ templateId: engagementArtifacts.templateId })
.from(engagementArtifacts)
.where(like(engagementArtifacts.title, `${prefix}%`));
const usageByTemplate = new Map<string, number>();
for (const row of instantiated) {
if (row.templateId) usageByTemplate.set(row.templateId, (usageByTemplate.get(row.templateId) ?? 0) + 1);
}
for (const [templateId, count] of usageByTemplate) {
await db
.update(motionTemplates)
@@ -468,5 +366,6 @@ export async function seedMotionEngagements(
artifacts: artifactCount,
scores: scoreCount,
promoted: promotedCount,
missingDeals,
};
}
@@ -0,0 +1,191 @@
{
"key": "aurelian-committed-capacity",
"dealName": "DEMO — Multi-year committed capacity",
"accountName": "DEMO — Aurelian Systems",
"stage": "legal",
"status": "open",
"summary": "DEMO — Three-year reservation of 128 H200s in Frankfurt for Aurelian's Zurich research lab, technically well qualified and stuck in paper for sixteen weeks. Four blocking items remain (the DPO determination on the claims corpus, the base licence redistribution question — the half of weights ownership that is not answered — retention against checkpoint lineage, and Krall's on-site audit demand), the 14 July Group Investment Committee window was missed, and the regulatory deadline that gave the deal its urgency has moved to Q4 2027.",
"openedDaysAgo": 132,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — First Technical Discovery: Aurelian Risk Intelligence Lab",
"status": "final",
"authoredDaysAgo": 124,
"body": "*Filled in from the call on the 124-day mark, Zurich, 95 minutes. Present: Dr Ines Marchetti (Head of the Aurelian Risk Intelligence Lab), Konrad Brenner (Head of Research Infrastructure), Léa Bonnard (senior research engineer, runs the training stack day to day). Anders Lovgren, Group CTO, joined for the last fifteen minutes.*\n\n## The company and the constraint\n\nAurelian Systems AG is a Zurich-headquartered specialty reinsurer and catastrophe risk analytics group, roughly 4,100 staff, with a 40-person internal lab split between Zurich and Edinburgh. They are not buying a workflow outcome. They are buying capacity, and the question this call had to answer is whether the workloads they intend to put on it are real and sized, or aspirational.\n\nThey are compute-bound today and can prove it. 64 A100s in a colocation cage in Zurich-West, median utilisation 91% over the trailing 90 days, p50 queue wait 6.4 hours and p95 at 41 hours. Bonnard showed the Slurm accounting export rather than a slide, which is the reason this section has numbers in it.\n\n## The three workloads\n\n1. **Domain adaptation of a 27B open-weight base** on 1.9M claims narratives and 240k engineering survey reports. Continued pre-training, then instruction tuning. This is the bulk of the reservation and the reason the ask is H200s rather than more A100s: Bonnard's own measurement on a borrowed node puts the 27B continued-pretraining step at 3.1x an A100 at the same batch, so 128 H200s clears Brenner's 2.4x target with margin for the other two workloads.\n2. **RL post-training of the Exposure Note agent**, which drafts facultative reinsurance submissions from a broker packet. Human turnaround today is 3.2 days median per submission. A prompted frontier baseline reaches 61% field-complete against the 14 mandatory fields; the internal acceptance bar is 94%.\n3. **Ensemble catastrophe model calibration**, non-LLM, bursty, currently the thing that starves workloads 1 and 2 every quarter-end.\n\n## Traces, verification, baseline\n\nTraces exist and are unusually good: 11,400 facultative submissions with the adjudicated label attached, because the underwriting committee already records a structured decision. That is the seed of a verifier — 14 mandatory fields are rules-checkable, and narrative adequacy would need a judge calibrated against committee sign-off. Nobody has built either yet. Marchetti agreed to have Bonnard prototype the field checker.\n\nRights to use the narratives are the open question and were the last thing raised. The claims corpus contains bodily-injury records, which means special-category health data under the Swiss FADP and the GDPR. Brenner's honest answer: \"nobody has ever asked to move it, so nobody has ever refused.\"\n\n## The number and who says it\n\nBrenner owns the utilisation number and says it without hedging: research infrastructure must reach 2.4x current training throughput to clear the model recalibration programme's backlog. He also said, unprompted, that 128 reserved GPUs at a fixed monthly cost is easier for him to defend internally than a variable hyperscaler bill, because his budget line is annual and flat.\n\n## What is missing\n\nNo named budget line yet beyond \"the modernisation programme\". No security contact in the room. No environment work of any kind has been attempted — nobody has yet described how the training data would physically reach a Prime Intellect region, and I did not push it in this call. Lovgren was clear that a multi-year commitment goes to the Group Investment Committee, which he chairs, and that he will not take it there until legal have cleared the data question.",
"kind": "discovery"
},
{
"templateSlug": "trainability-qualification",
"stage": "qualification",
"title": "DEMO — Trainability and Deal Qualification: Aurelian Systems",
"status": "final",
"authoredDaysAgo": 116,
"body": "*Scored from the notes of the discovery call. Technical subtotal 33 of 57, commercial 30.25 of 43, weighted result 63.25 — Qualified, conditional.*\n\n## Why this is scored at all for a capacity deal\n\nAurelian are buying reserved compute, not a post-training outcome, so half of this scorecard is arguably out of scope. It is scored anyway, because a three-year commitment is only defensible if the workloads that justify it can actually be built. If the Exposure Note agent turns out to be untrainable, the reservation is oversized by roughly a third within eighteen months and the renewal is lost before it starts.\n\n## Disqualifiers\n\nTwo of the four hard overrides are in play and neither is triggered yet.\n\n**No rights to train on the data.** Not triggered, because nothing has been refused — but nothing has been granted either. Stefan Reuss, Group DPO, has issued no written determination that the claims corpus may be used to train a model. What turns this live is a determination that the bodily-injury subset may be processed but not used for training. If that lands, workload 1 disappears and this record goes to Decline regardless of the arithmetic above.\n\n**Incompatible weights-ownership or deployment expectation.** Not triggered. Aurelian own the checkpoint, the adapters and the merged weights, and we have said so in writing. What turns this live is the cedant portal: Aurelian intend to expose the adapted model to their cedants, which may be redistribution under the 27B base licence. If Hertzfeld and Bau read it that way and the base provider will not licence around it, the deployment expectation and the base are incompatible and this becomes a re-scope onto a different base.\n\n## Where the points are lost\n\n**Verifier availability and quality (2, losing 26 weighted points).** The largest single loss and the one to fix first. The raw material is there — 11,400 adjudicated facultative submissions, committee decisions already structured — but neither the rules-based field checker nor the narrative judge exists. Until Bonnard's prototype runs against a held-out set this is a plan, not an instrument, and the anchors are explicit that \"they said they have it\" scores a 1.\n\n**Traces, rights, and volume (2).** Volume is not the problem; 1.9M claims narratives is more than the domain adaptation needs. Rights are. Bodily-injury records make this special-category data, the corpus has never left the Zurich estate, and there is no written internal determination that it may.\n\n**Environment constructibility (2).** Scored down from my first instinct. Nothing has been built and nothing has been designed: no data path out of the Zurich cage has been described, the sandbox dependencies of the Exposure Note agent are unenumerated, and Applied Research have not been engaged. The workloads are conventional and that is the only reason this is not a 1.\n\n**Headroom (2).** Domain adaptation of a 27B base on 1.9M in-domain narratives has obvious headroom over prompting. The Exposure Note agent is less clear: 61% to 94% field completeness may be reachable with a better packet parser and retrieval rather than RL, and nobody has run that experiment.\n\n**Security and legal path (1).** No DPA, no MSA, no security contact named, and the data in question is the sensitive kind. This is the gating dimension and it is why the deal goes to legal next rather than to scoping.\n\n## Where the deal is strong\n\n**Named metric owner (4).** Brenner owns training throughput, states the target as 2.4x, and produced the Slurm accounting to support it.\n\n**Executive sponsor (4).** Lovgren is Group CTO, chairs the Investment Committee that must approve the term, and volunteered the approval path without being asked.\n\n**Measured baseline (3).** Utilisation, queue waits, human turnaround and the prompted baseline are all measured. The gap to a 4 is that the 91% figure includes the catastrophe calibration bursts, which are not the workload the reservation is being sized for.\n\n## Verdict\n\nQualified, conditional, at 63.25. The condition is legal and it is not a formality.",
"kind": "qualification"
},
{
"templateSlug": "strategic-deployment-playbook",
"stage": "legal",
"title": "DEMO — Engagement spine: Aurelian committed capacity",
"status": "final",
"authoredDaysAgo": 15,
"body": "*Instantiated for Aurelian Systems and rewritten this week, sixteen weeks into legal, because the version written at qualification no longer described the engagement. Maintained by the forward deployed lead.*\n\n## Shape of this deal\n\nThree-year committed capacity: 128 H200s reserved in Frankfurt at $211,046.33 per month, annualised $2,532,556, which works out at roughly $2.26 per GPU-hour if the reservation is fully used. It is the largest commitment in the book and the one with the least room for a graceful exit, which changes how the stages are run. On a POC-shaped deal you can discover a legal problem at scoping and re-scope. Here the paper is the deal: if trace egress lands badly, the customer is left with reserved capacity they cannot legally feed.\n\n## Qualification — closed, day 116\n\nExit met. Discovery produced measured utilisation, three named workloads and a metric owner who states a number. Scorecard 63.25, Qualified conditional. The condition is recorded against legal, not carried as a general worry.\n\n## Legal — open, current stage, sixteen weeks\n\nThe Data and Security Brief is the artefact carrying this stage and the open-items list is what it has become. Exit criteria, written so someone outside the room could judge them:\n\n1. Executed MSA and DPA, or a signed position paper on the four blocking items with a dated path to signature.\n2. A named transfer instrument for every flow that crosses the Swiss or EEA boundary, including Prime Intellect support access.\n3. A written determination from Aurelian's Group Data Protection Officer that the claims corpus may be used to train a model Aurelian owns.\n\nItem 3 decides whether workload 1 exists. Nothing else in this engagement matters more.\n\n## My own worst mistake on this engagement\n\nI quoted Frankfurt as the region in the first commercial conversation without checking whether the Zurich-to-Frankfurt crossing needed a transfer instrument of its own. It does — Switzerland is a third country for GDPR purposes in one direction and an adequacy case in the other, and Fontaine's team spent three weeks unpicking a position I had presented as settled. That is three of the sixteen weeks this stage has cost, and it is why Reuss now reads everything I send twice. It also means the region is harder to change than it should be, because reopening it reopens that argument.\n\n## Scoping — not started\n\nApplied Research have not been engaged and will not be until the DPA position is known. Marchetti has asked twice for an environment design session and I have said no twice. I am less sure of that than I was: a narrow session on the data path alone would have surfaced the crossing problem above months earlier.\n\n## Procurement — pre-worked, out of sequence\n\nYannick Depré (Category Manager, Indirect Technology) confirmed on day 61 that the spend sits under the Group Technology Modernisation programme, line TM-2027-014, owned by the CFO's office. Commitments beyond twelve months require Group Investment Committee approval. The Committee meets quarterly and Lovgren chairs it. The 14 July 2026 meeting was the target and was missed. The next window is 13 October 2026 and it is not safe either: the paper cannot be drafted without items 1 and 2.\n\n## The honest risk\n\nThe forcing function has gone. The reason a three-year commitment was going to be signed this quarter is that the internal model recalibration had a board-committed submission date of 30 April 2027. Brenner mentioned in passing around day 70 that the Claims Data Fabric migration was slipping and the date might move; it was rumour then and I recorded it as rumour. Lovgren confirmed it on the day-19 call: the submission date is Q4 2027. Nobody at Aurelian has said the deal is less urgent. Everybody's behaviour says it. The counter-move is to stop selling the recalibration timeline and start selling the queue: Brenner's p95 wait of 41 hours is a problem that exists this week regardless of what the regulator's calendar does.",
"kind": "playbook"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Data and Security Brief: Aurelian Systems",
"status": "draft",
"authoredDaysAgo": 18,
"body": "*Opened with Marchetti and Brenner over three sessions from day 96, then rewritten after the first joint call with their outside counsel on day 19 and written up the following day. Status: draft — six of the fourteen decisions are unanswered and four of those block. Their counsel is Marguerite Fontaine, Deputy General Counsel, supported by Hertzfeld and Bau in Zurich; Group DPO is Stefan Reuss.*\n\n## Legal basis and roles\n\nPrime Intellect paper starts; Fontaine accepted this without argument. Aurelian is controller and Prime Intellect processor, for the reserved capacity and for the domain adaptation runs alike — the model is trained for Aurelian, on Aurelian's instruction. Recorded because Reuss's first written question assumed otherwise.\n\n## Permitted use — answered, not blocking\n\nAurelian data trains Aurelian models and nothing else: no cross-customer training, no shared base, no de-identified service-improvement carve-out. Fontaine's note back said this removed \"the objection we were preparing to lead with\".\n\n## Weights ownership on a derived model — half answered, half blocking\n\nAurelian own the continued-pretrained checkpoint, the adapters and the merged weights outright; that half is answered in writing and Fontaine has accepted it. Two parts need counsel:\n\n- The base is a 27B open-weight model whose licence carries a commercial-use threshold and a downstream redistribution condition. Aurelian intend to expose the adapted model to their cedants through a portal, which is arguably redistribution. That is a question for Hertzfeld and Bau against the base licence, not a question for us, and I have said so in writing to avoid giving advice we are not able to give. **This is the blocking half.**\n- Which party contracts with the base provider. Aurelian would prefer we do. That moves the licence risk above onto us and is a leadership decision, not a lead's.\n\n## Trace egress — blocking\n\nThe claims corpus contains bodily-injury records: special-category health data under both the FADP and the GDPR. Aurelian policy reads \"EEA or Switzerland only\". Frankfurt satisfies that, after three weeks of argument I caused by quoting the region before checking whether the Zurich-to-Frankfurt crossing carried its own instrument. Two gaps remain.\n\nFirst, Reuss has not issued a written determination that the corpus may be used for model training at all, as distinct from being processed. Everything in workload 1 depends on this one memo. It has been \"with the DPO\" for six weeks.\n\nSecond, support access. Our on-call rotation includes engineers outside the EEA, and they are ours, not Aurelian's — an onward transfer processor to processor, so the instrument is EU standard contractual clauses Module 3 with the Swiss addendum plus a supplementary measures note. Fontaine considers this workable; Reuss has not seen it yet.\n\n## Output retention and checkpoint lineage — blocking\n\nRollouts from the Exposure Note agent reproduce claim narrative text verbatim, so they inherit the sensitivity of the source. Aurelian's retention schedule says 90 days for derived artefacts, and that collides with checkpoint lineage: deleting an artefact does not remove its influence from the weights, and removing influence means retraining from a checkpoint predating the ingest. Retaining those pre-ingest checkpoints for the life of the model contradicts the 90-day schedule on its face.\n\nProposed and not yet accepted: artefact deletion and suspension of the affected checkpoint from serving inside Aurelian's own 30-day erasure SLA, with retraining on a separate, priced schedule. A lineage record from artefact to run to checkpoint is a build commitment on our side, and I wrote it into this brief before asking engineering. That was a mistake and it is still live.\n\n## Sandbox egress — answered\n\nThe agent sandbox needs one internal pricing service and nothing else: single-host allowlist, no general egress, sandbox and logs destroyed at 30 days. Bonnard confirmed no second dependency.\n\n## Audit — blocking, stalled on a person\n\nBettina Krall, Group Head of Information Security, asked for on-site inspection of the Frankfurt facility. The right answer is the rung below: a named third-party auditor's review under NDA, plus our published report and 30 days notice of subprocessor change. She has been out since day 40 and her interim, Rui Peixoto, will not overrule her. Her sign-off is a condition precedent on the MSA security schedule, so this blocks signature.",
"kind": "discovery"
},
{
"templateSlug": null,
"stage": "legal",
"title": "DEMO — Legal open items and counsel routing",
"status": "review",
"authoredDaysAgo": 12,
"body": "*A running list, maintained by the forward deployed lead, of what stands between this engagement and a signature. Reviewed with Marchetti every Tuesday. Nothing here is legal advice; routing is about who can answer, not who is right.*\n\n## Blocking, in the order they will actually resolve\n\n**1. The DPO memo (Stefan Reuss).** A written determination that the 1.9M claims narratives may be used to train a model Aurelian owns. Six weeks and five days with the DPO now, a number that has grown at every review of this list. Marchetti chased on day 26 and day 15. Reuss wants Hertzfeld and Bau's view on the bodily-injury subset first, which makes this dependent on item 2. Routing: their counsel. All I can do is keep the question narrow — the memo need cover only the subset the domain adaptation reads.\n\n**2. Base licence redistribution (Hertzfeld and Bau).** Whether exposing the adapted 27B model to cedants through the Aurelian portal is redistribution under the base licence, and whether the commercial threshold is crossed. Routing: their outside counsel. Twice asked for our interpretation, twice declined in writing. If the answer is unfavourable the fallback is a different base — a re-scope, not a re-paper.\n\n**3. Retention against lineage.** Our proposal — artefact deletion plus checkpoint suspension inside Aurelian's 30-day erasure SLA, retraining priced separately — is with Fontaine. Routing: joint. Complication of my own making: I wrote the artefact-to-run-to-checkpoint lineage record into the brief as a commitment before engineering agreed to build it. They have since pushed back — no roadmap, and the owner wants a scoping week first. I have not told Fontaine, and the longer I do not the harder the position paper is to unwind.\n\n**4. The audit rung (Bettina Krall).** Her sign-off is a condition precedent on the MSA security schedule, so an unresolved on-site inspection demand blocks signature as surely as the DPO memo does. She has been out since day 40 and Peixoto will not decide. Wait for her return, expected in about three weeks, or ask Lovgren to break the tie — I would rather spend that favour on the Committee paper.\n\n## Not blocking, answered by the lead\n\n- Permitted use and no cross-customer training: answered day 18, accepted.\n- Ownership of adapters and the merged checkpoint: answered day 18, accepted.\n- Sandbox egress, single-host allowlist, 30-day log destruction: answered day 18, accepted.\n- Region: Frankfurt. Stockholm was offered and declined by Brenner on staging grounds, not latency: the initial corpus stage is 41 TB out of the Zurich cage, and Aurelian hold contracted 10 Gbps fibre to Frankfurt against a 1 Gbps commodity path to Stockholm — about nine hours versus nearly four days per restage, three restages expected in year one.\n\n## Verifier — the one thing that moved forward\n\nBonnard's field checker ran on day 20 against a held-out set of 1,150 of the 11,400 submissions, held out by submission date rather than at random. It agrees with the adjudicated labels on 13 of the 14 mandatory fields at 0.96 exact match; the fourteenth, loss-cause narrative adequacy, it cannot check at all and needs a judge calibrated against committee sign-off. An instrument for most of the task and a plan for the rest — which is why it moved to 3 and not 4.\n\n## Re-score, day 19\n\n266 of 400 — 66.5, still Qualified conditional, up from 63.25. Two dimensions moved in opposite directions and matter more than the total: security and legal path 1 to 2 (a named instrument set and an agreed structure, but no MSA, no DPA, no DPO memo) and forcing function 3 to 1 on the Q4 2027 slip. Environment constructibility moved 2 to 3, the data boundary now pinned down. A deal can score higher and be further from signature, and this one is.\n\n## Commercial consequence, stated plainly\n\nThe 14 July Committee window was missed; the next is 13 October 2026 and no paper can be drafted without items 1 and 2. Brenner's queue is the live argument now — p95 wait of 41 hours, 91% utilisation, a team losing roughly a day and a half a week — and it is a weaker case than the regulatory one it replaces.",
"kind": "discovery"
}
],
"scores": [
{
"daysAgo": 116,
"note": "DEMO — First score, taken straight from discovery: 253 of 400, 63.25, Qualified conditional. Technically strong on the things that are measured (utilisation, queue waits, a metric owner who states 2.4x) and weak on the three things nobody has built or written down: a verifier for the Exposure Note agent, any determination that the claims corpus may be used for training, and any environment design at all — no data path out of the Zurich cage has been described and Applied Research have not been engaged, which is why environment constructibility is a 2 and not the 3 the conventional workloads would suggest.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 3
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 2
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 3
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 4
},
{
"id": "security_legal_path",
"weight": 8,
"score": 1
},
{
"id": "forcing_function",
"weight": 8,
"score": 3
},
{
"id": "expansion_surface",
"weight": 5,
"score": 2
}
]
},
{
"daysAgo": 19,
"note": "DEMO — Re-scored after the first joint call with their outside counsel: 266 of 400, 66.5, still Qualified conditional. Verifier 2 to 3 on Bonnard's field checker agreeing with the adjudicated labels on 13 of 14 mandatory fields at 0.96 exact match over a held-out set of 1,150 submissions, with narrative adequacy still unverifiable; environment constructibility 2 to 3 because the brief pinned the boundary down — single-host allowlist, no general egress, 30-day destruction, no second dependency; security and legal path 1 to 2 only, because a named instrument set is not an executed MSA or DPA and the DPO determination is still missing after six weeks. Forcing function 3 to 1: Lovgren confirmed on this call that the recalibration submission date has slipped to Q4 2027, and the 14 July Committee window went with it. Expansion surface held at 2 deliberately — Applied Research are unengaged, the Edinburgh half of the lab has not been approached, and a deal frozen in paper has not grown its surface.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 3
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 3
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 4
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 1
},
{
"id": "expansion_surface",
"weight": 5,
"score": 2
}
]
}
],
"promote": null
}
@@ -0,0 +1,266 @@
{
"key": "halcyon-post-training",
"dealName": "DEMO — Managed post-training run",
"accountName": "DEMO — Halcyon Research",
"stage": "poc",
"status": "open",
"summary": "DEMO — Halcyon Research is in week three of an eight-week paid POC to beat a measured 0.64 prompted baseline on regulatory-note summarisation; the verifier is measured at Cohen's kappa 0.71 against a 0.73 human ceiling, but the week-two dev-split reading came in at 0.66, under the trajectory needed for the 0.72 kill gate on 28 August. Two things are live: that trajectory, and the oncology stratum, where Halcyon's own adjudicators agree at only kappa 0.54 and nobody has chosen whether to re-adjudicate, exclude it or report it separately.",
"openedDaysAgo": 132,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — Halcyon Research: first technical discovery",
"status": "final",
"authoredDaysAgo": 128,
"body": "*Discovery call, 90 minutes, on site in Basel. Present from Halcyon: Dr Elsa Wennerberg (VP Research Operations, owns the metric), Joachim Sterk (Head of Editorial Systems), Liv Sørensen (senior analyst, 11 years). From us: forward deployed lead plus one forward deployed engineer. Applied Research not present, by design.*\n\n## The workflow\n\nHalcyon sells regulatory-intelligence subscriptions to life-sciences clients. The unit of work is a **signal note**: an analyst reads a source bundle (registry updates, agency correspondence, filings, sponsor press material) and writes a structured note of roughly 420 words with a materiality call, a claim list and citations back to the bundle.\n\nVolume: about 14,200 source documents ingested a month, condensed into **3,800 published notes a month** by a pool of 31 analysts. Median analyst throughput is 5.6 notes a day. Notes are drafted in Cormorant, their internal editorial system, and pass a senior review before publication.\n\nThe pain is not cost, it is latency. Client SLA is a note within four working hours of the source appearing. Wennerberg says they breach on roughly 9% of notes and every breach is visible to the client, because the client sees the source on the registry at the same moment Halcyon does.\n\n## The failed attempt\n\nSterk's team ran a prompting programme from October to January: nine prompt variants against a frontier model, three-shot, with retrieval over the bundle. Measured on 120 hand-checked notes, the best variant reached **0.58 on what they call faithful-and-complete rate** (a note counts only if it contains zero unsupported claims and misses no material event). It plateaued: variants seven, eight and nine were within noise of each other. Human analysts sit at 0.91 on the same check. They concluded prompting had run out and stopped. That is the headroom argument, and they made it themselves before we did.\n\n## Traces and rights\n\nCormorant holds 41 months of note history at that rate, roughly **156,000 published notes**, each linked to its source bundle. Two complications. First, about 22% derive from client-confidential documents that Halcyon cannot use for training under existing subscription contracts, leaving roughly 122,000. Second, and better than it sounds: Cormorant retains the **pre-review and post-review versions** for 9,400 notes, which is an edit-diff corpus showing exactly what senior reviewers correct. Sørensen believes most corrections are missed materiality, not style.\n\n## Verification\n\nTwo-part and plausibly computable. Every numeric value, date and registry identifier in a note must appear in the bundle, which is deterministic string and range matching. Materiality and completeness are judgement, and Halcyon already has a six-point internal rubric used in review. Nobody has ever measured whether two senior analysts applying that rubric agree with each other. That measurement is the first thing we need.\n\n## What we did not get\n\nNo baseline on a held-out set — the 120 notes were sampled by Sterk's team from cases they had already looked at. No budget line. Wennerberg said \"productivity budget exists\" and could not name it. Chief Research Officer Alan Beaumont is described as supportive but did not attend.\n\n## Next\n\nScore the trainability scorecard today. Open the data and security brief with Sterk and Isabelle Vance (General Counsel) before we design anything.",
"kind": "discovery"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Halcyon Research: data and security brief",
"status": "final",
"authoredDaysAgo": 111,
"body": "*Filled in with Joachim Sterk and Isabelle Vance (General Counsel), 40 minutes, and reviewed by Vance in writing. Not legal advice and not presented as such.*\n\n## Data classes in scope\n\n1. **Source bundles.** Public and semi-public registry, agency and sponsor material. Halcyon asserts no restriction on use. Roughly 14,200 documents a month.\n2. **Published signal notes.** Halcyon's own copyright, about 156,000 in Cormorant.\n3. **Client-confidential derivatives.** Roughly 22% of notes derive from documents provided under client NDA. **Excluded entirely**, at source, by the Cormorant `client_conf` flag. Vance was unambiguous and we did not push. Usable corpus is therefore about 122,000 notes.\n4. **Edit diffs.** 9,400 pre-review and post-review pairs. The same exclusion applies; 7,300 survive it.\n\n## Rights to train\n\nVance confirmed in writing that Halcyon's client terms grant Halcyon a perpetual licence to its own analytical output and that training a model on non-flagged notes is within scope. She reserved one point: notes authored before March 2024 predate a terms revision and she will not warrant them. We have accepted the post-March-2024 subset rather than wait for a review nobody has budget to run.\n\nThat cut applies to both corpora and it is the largest single reduction in this brief. Notes: 122,000 falls to about **78,000**. Diffs: 7,300 falls to about **4,900** — proportionally less, because Cormorant only started retaining pre-review versions in mid-2023, so the diff corpus already skews recent. Every downstream number in the architecture and the POC plan is 78,000 and 4,900, not the headline figures.\n\n## Boundary and residency\n\nHalcyon is Basel-domiciled with EU clients. Training runs in the EU region only. Vance did not ask for a Swiss residency guarantee, which surprised us; her constraint is transfer to the United States, not location as such. Recorded as a hard constraint: **no customer data at rest or in transit through US-region infrastructure.** Scrubbing runs inside Halcyon's tenancy before egress: the `client_conf` filter, then the March-2024 date filter, then removal of analyst identity from the diff corpus, then a sample audit of 300 records by Sørensen before anything moves.\n\n## Personal data\n\nAnalyst names appear in the diff metadata and are stripped. Source bundles occasionally contain investigator names, which are public in registry entries; Vance treats these as public-domain professional data and no further action is required. No patient-level data exists anywhere in this corpus, which is why this brief was 40 minutes rather than three weeks.\n\n## Retention and deletion\n\nTraining corpus retained for the POC term plus 30 days. Adapted weights retained by Prime Intellect for the POC term; Halcyon receives the adapter under a perpetual internal-use licence at POC close whatever the result. Deletion certificate on request. Evaluation traces and verifier outputs are retained by Halcyon, not by us — see the held-out set arrangement in the POC plan.\n\n## Weights and deployment expectation\n\nChecked explicitly because it is a disqualifier if wrong. Halcyon wants to self-host inference in their own tenancy within 12 months and accepts an adapter over an open-weights base under licence rather than a bespoke owned model. Sterk has GPU capacity for inference and none for training. Compatible.\n\n## Open items\n\nVance wants a named sub-processor list before signature, which we owe her. The 22% exclusion is settled and should not be reopened by anyone hoping to grow the corpus; asking again will cost more goodwill than the data is worth.",
"kind": "discovery"
},
{
"templateSlug": "reference-architectures",
"stage": "scoping",
"title": "DEMO — Halcyon signal notes: environment and eval architecture",
"status": "final",
"authoredDaysAgo": 94,
"body": "*Scoping output. Architecture chosen: **shape 3, RL in a constructed environment**, bootstrapped from the shape-2 trace corpus, with a customer-held eval. Reviewed with Sterk, and with Applied Research on a single 60-minute structured hand-in.*\n\n## Why not eval-first\n\nEval-first is normally the correct opening and we rejected it here for one reason: Halcyon has already done the eval-first work themselves and stopped at 0.58. Repeating it as a paid engagement would insult the people who ran it. What they lack is not a measurement, it is a measurement they trust — their 120-case sample was drawn from cases the team had already inspected, so it is contaminated and cannot serve as a baseline. We fold the clean measurement into the mobilisation week of the POC rather than sell it as a phase.\n\n## The shape\n\n```\nHalcyon tenancy Prime Intellect Lab (EU)\n Cormorant --> client_conf filter --> scrubbed corpus (78k notes,\n (156k notes) + March-2024 cut 4.9k edit diffs)\n + analyst strip |\n Source bundle store ------------------> environment: single-turn\n (read-only mirror, EU) note generation over a\n retrieved bundle\n |\n Held-out set (700 cases) <-- verifier composite --> RL loop, 32B\n held by Halcyon, never leaves open-weights base\n```\n\n**Environment.** Single-turn. The policy receives a source bundle and the note schema and emits a structured note. Bundle retrieval is offline against a read-only mirror Sterk provisions in the EU region, so no call ever touches live Cormorant. No tools, no side effects, no reset problem — this is the cheapest environment in the library and it is available to us only because the task is genuinely single-turn. Anything involving Cormorant write access would have doubled the scoping.\n\n**Warm start, then RL.** The 4,900 edit diffs are supervised material: pre-review in, post-review out. They are shape-2 trace data and they exist to move the policy into the neighbourhood before any RL spend. The RL loop then optimises against the composite verifier in the constructed environment. That hybrid is why this is filed as shape 3 rather than shape 2 — the trace corpus is the bootstrap, not the product.\n\n**Verifier, as an architectural component.** Runs on our side during training and on Halcyon's side during evaluation, from the same pinned container image. Two stages: a deterministic checker over numerals, dates, registry identifiers and schema conformance, then a judge applying Halcyon's own six-point materiality rubric. Composite is a hard gate on the deterministic stage followed by the rubric score. The judge is the risky half and its agreement figure is measured in the POC, not assumed here.\n\n**The held-out set is the important box.** 700 cases sampled by Sørensen from November 2025 to March 2026 traffic, stratified by therapeutic area, adjudicated by a panel of three senior analysts, and held inside Halcyon's tenancy. We never see the labels. We submit a candidate, Halcyon runs the pinned verifier, Halcyon reports the number. This is slower than doing it ourselves and it is the only version of the result their own analysts will believe.\n\n## Where data crosses a boundary\n\nExactly once, outbound: scrubbed corpus to the EU training region, after Sørensen's 300-record audit. Nothing crosses inbound except adapter weights and evaluation submissions. No US-region component anywhere, including logging and object storage, which Sterk will verify against our sub-processor list.\n\n## Decisions that redraw this picture\n\nIf Halcyon later wants notes drafted inside Cormorant with tool access to their citation store, this becomes a multi-turn environment with a reset requirement and a different cost. Out of scope, worth flagging as the obvious expansion.\n\n## Failure mode to watch\n\nThe 4,900 edit diffs are the highest-signal data we have and they are not uniformly sourced. Two reviewers produced a disproportionate share. If the reward learns their house style rather than Halcyon's standard, the model will win the verifier and lose the analysts. Hold one reviewer's pairs out entirely and read the gap.",
"kind": "architecture"
},
{
"templateSlug": "procurement-and-budget-path",
"stage": "procurement",
"title": "DEMO — Halcyon Research: budget path to the 8 October committee",
"status": "final",
"authoredDaysAgo": 26,
"body": "*Written after two calls with Solveig Rask (Director, FP&A) and one with Wennerberg. This closes the budget-source gap that has sat open on the scorecard since discovery.*\n\n## The number\n\nFirst-year value $799,027: managed post-training programme, training compute, environment and verifier engineering, and Applied Research time at the agreed allocation. POC at $120,000, credited in full against year one if Halcyon proceeds.\n\n## The line, named\n\nRask named it on the second call, which is the finding this artefact exists to record. The money is the **FY27 Research Productivity programme**, $1.05M, of which $850,000 was uncommitted as of her 14 July forecast. It is an operating line owned by Wennerberg's function, not a capital request, and it is not the AI budget — Halcyon does not have one, which is why nobody could name it in discovery. Asking \"which budget\" produced nothing for six weeks; asking \"what did you stop doing to free headcount last year\" produced this in eleven minutes.\n\nThe POC's $120,000 comes from a different pocket: the FY26 innovation reserve, discretionary to Beaumont up to $150,000 and therefore signable without a committee. That is why the POC can start on 3 August rather than after the FY27 planning cycle closes in October.\n\n## Approval path\n\n1. Wennerberg submits a business case to the Research Operations budget review. Done, 22 July.\n2. Beaumont approves as sponsor. Nominally routine.\n3. Above $500,000 the Executive Committee approves at its quarterly meeting. Next meeting **8 October**. The one after is in January.\n4. Procurement and Vance run security review and contract in parallel, not sequentially. Vance has agreed to this and it saves roughly three weeks.\n\n## The forcing function, and its honest weight\n\nHalcyon's four largest subscription contracts renew on 31 December, together about 34% of revenue. Wennerberg's argument to the committee is the 9% SLA breach rate, which appears in two of those four renewal conversations as a stated client complaint. That is real but it is not a cliff: no client has threatened to leave, and if the committee slips to January the deal survives at reduced pace. We should not tell ourselves otherwise, and we should not tell Halcyon a story about urgency they know better than we do. Score the forcing function down, not up.\n\n## What must be true by 8 October\n\nThe POC result must be in Wennerberg's hands with enough margin to write the paper. Week eight of the POC lands Friday 25 September, which leaves thirteen days. That is tight and it is the reason the kill gate in the POC plan is placed at week four rather than week six.\n\n## Risks on this path\n\nBeaumont is the sponsor of record and has now missed two of three steering calls, sending an apology once and nothing the second time. Wennerberg says he is consumed by a client escalation. If he is not personally in the room on 8 October, an $799k operating request presented by a VP into a committee that has not heard the argument before is a coin flip. We have asked Wennerberg for a 20-minute briefing with him before mid-September and do not have a date.\n\n## Not required\n\nNo new vendor onboarding: Halcyon already holds an MSA with us from a 2025 evaluations pilot that never scaled. Procurement confirmed the MSA is live and only an order form is needed.",
"kind": "pricing"
},
{
"templateSlug": "post-training-poc",
"stage": "poc",
"title": "DEMO — Halcyon signal-note POC: plan and kill gate",
"status": "draft",
"authoredDaysAgo": 2,
"body": "*POC plan, amendment 3, revised Monday 17 August at the start of week three, pending countersignature. Kickoff was Monday 3 August.*\n\n## The hypothesis\n\n> On a held-out set of 700 signal-note cases sampled from November 2025 to March 2026 production traffic and held by Halcyon, an adapted open-weights 32B model will score at least **0.79 faithful-and-complete rate** against a prompted frontier baseline of **0.64**, as measured by the Halcyon composite verifier, whose agreement with the senior analyst panel is at least **Cohen's kappa 0.68** on a 180-case dual-labelled audit drawn from the held-out set, where the human-to-human ceiling is **kappa 0.73 aggregate** (0.77 ex-oncology).\n\nHuman analysts score 0.91 on the same check. The target is a 15-point gain on a measure Halcyon already uses in review, deliberately below the human rate: a model that matched analysts would not be believed.\n\n## Eval design\n\n700 cases, sampled by Liv Sørensen, stratified by therapeutic area to match the published mix, adjudicated by three senior analysts, held inside Halcyon's tenancy. We never see the labels: we submit a candidate adapter, Halcyon runs the pinned verifier image and reports the number. A separate **240-case dev split**, disjoint from the 700, is shared with us for iteration. The 180-case audit is a dual-labelled subset of the 700 and measures only the verifier.\n\n## Verifier, measured\n\nStage one is deterministic: numerals, dates, registry identifiers, schema conformance. Stage two is a judge on Halcyon's six-point materiality rubric. Measured in week one on the 180-case audit: **composite agreement kappa 0.71 aggregate** (0.73 ex-oncology), ceiling 0.73 aggregate. Above the 0.68 threshold, so the gate clears.\n\n## The stratum problem, unresolved\n\nThe audit surfaced what the aggregate hides. On the oncology stratum — 34 of the 180 audit cases, 126 of the 700 held-out cases — the three adjudicators agree with each other at only **kappa 0.54**, against 0.77 everywhere else. They disagree about what is material in a dose-escalation amendment. The judge tracks them at 0.61 there, reproducing a disagreement rather than a standard. At n=34 the interval is wide, as we have said in writing.\n\nThree options, none chosen: re-adjudicate against a tightened rubric (roughly 9 analyst-days, which Wennerberg has not offered); exclude the stratum and report on 574 cases with that stated in the headline; or report it separately with its own number. This cannot be decided after we see the result. **Decision required by 26 August**, and amendment 3 is unsigned until it is made.\n\n## Baseline\n\nMeasured in the mobilisation week on the full 700 by Halcyon, not by us: **0.64** on Sterk's best prompt variant. Their own contaminated figure was 0.58, so the bar went up before we started.\n\n## The weeks\n\n- **Mobilisation, 2731 July. Done.** Corpus scrubbed and audited by Sørensen; 78,000 notes and 4,900 diffs across. Baseline 0.64.\n- **Week 1, 37 August. Done.** Verifier at kappa 0.71. Environment stood up against the EU bundle mirror, no reset requirement. Exit met.\n- **Week 2, 1014 August. Done, below plan.** Warm start on the diff corpus, first RL runs. Dev split reads **0.66** against the 0.69 our trajectory needed for a credible 0.72 at week four. Held-out reviewer check, first read: 0.71 on that reviewer's pairs, 0.61 on everyone else's. The house-style failure mode flagged at scoping is firing.\n- **Week 3, 1721 August. Current.** Reward rebalanced away from stylistic edits, diffs reweighted by reviewer. Second dev-split reading Friday 21 August.\n- **Week 4, ends Friday 28 August. Kill gate.** Requires **0.72 or better on the dev split**. At week four, not six, because Wennerberg needs the result before the 8 October committee.\n- **Week 6, ends 11 September.** First held-out submission. No target; calibration only.\n- **Week 8, Friday 25 September.** Final held-out submission, single shot, verdict against 0.79.\n\n## Kill gate\n\nIf the week-four dev-split reading is below 0.72 we stop. Halcyon keeps the corpus work, the measured verifier, the clean baseline and the adapter, and we write the negative result up jointly. Sterk has been told this in those words; on the week-two reading the gate is live rather than ceremonial.\n\n## Risks\n\nThe reviewer skew, now measured rather than suspected. The stratum decision above. Beaumont's attention.",
"kind": "poc"
},
{
"templateSlug": null,
"stage": "poc",
"title": "DEMO — Halcyon POC week one readout",
"status": "final",
"authoredDaysAgo": 12,
"body": "*Sent to Wennerberg, Sterk and Sørensen on Friday 7 August, the end of week one. Reproduced here as the record.*\n\nBoth week-one exit criteria were met. The verifier is measured, the baseline is clean, and one decision is now blocking.\n\n**The verifier holds up.** On the 180-case dual-labelled audit the composite agrees with your adjudicated labels at Cohen's kappa 0.71 aggregate, against a human-to-human ceiling of 0.73 measured on the same cases. Our threshold was 0.68.\n\nWhat surprised us is the split of the work. The deterministic stage catches 61% of all failures, and the shape of them is dull in the best way: dose units written as mg rather than mg/m², enrolment counts carried from an earlier protocol version, EudraCT and NCT identifiers transposed by one digit, and effective dates taken from the agency letterhead rather than the decision. None of that needs a judge. The judge carries materiality and completeness, and that is the half we will keep watching.\n\nIn the audit review Liv put it better than we would have: the model's misses \"read like a tired analyst at five o'clock, not like a machine\". That is the right kind of error profile to be training against, and it is why the warm start on the diff corpus is where week two goes.\n\n**The baseline is clean and it is higher than yours.** Halcyon measured 0.64 on the full 700-case held-out set, against the 0.58 your prompting programme reported in January. The difference is sampling: your 120 cases were drawn from work the team had already inspected. This is good news about your prompt engineering and bad news about our bar, and we would rather carry the higher number than argue for the flattering one at week eight.\n\n**The oncology stratum is the wound.** Your three adjudicators agree with each other at kappa 0.54 on the 34 oncology cases in the audit — 126 of them in the full held-out set — against 0.77 everywhere else. At n=34 that 0.54 carries a wide interval and we are not going to pretend otherwise, but the direction is unambiguous and it matches what Liv said before we measured anything. The disagreement is concentrated in dose-escalation amendments and it is a genuine editorial question, not a labelling error. The judge sits at 0.61 there, faithfully reproducing your disagreement rather than correcting it.\n\nWe cannot report a headline number over a stratum your own senior analysts cannot score consistently, and we will not decide what to do about it after we see the result. The three options are set out in the draft amendment 3 circulated on Thursday. Elsa, this is a call for you and Liv. **We need it by 26 August**, and amendment 3 stays unsigned until then.\n\n**Also open.** We are holding out one reviewer's edit-diff pairs entirely, so that we can see whether the model is learning Halcyon's standard or one person's house style. First read comes with the week-two dev split. If that reading is bad it does not stop the POC, but it changes what we train on for weeks five to eight.\n\nWeek two is the supervised warm start on the diff corpus and the first RL runs. Dev-split reading next Friday. The week-four kill gate is unchanged at 0.72 on the dev split.",
"kind": "poc"
}
],
"scores": [
{
"daysAgo": 128,
"note": "DEMO — Scored the same day as the first discovery. The task is repeatable and traced, but nothing is measured: no clean baseline (their 0.58 was drawn from cases the team had already inspected), no evidence the materiality rubric is reproducible between two humans, and nobody can name a budget line.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 1
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 4
}
]
},
{
"daysAgo": 88,
"note": "DEMO — Legal and scoping done, and nothing was measured in between, so only the dimensions that scoping actually touched move. Vance confirmed training rights in writing on the post-March-2024 subset and the 22% client-confidential exclusion is settled, so rights and the security path both rise; pinning the note schema and the six-point rubric in scoping takes task definability to 4. Verifier, baseline and budget source are held at their discovery values because nothing has been measured or named. Metric owner stays capped at 3: Wennerberg owns the metric but not the corpus decision. Forcing function falls to 1 — on the evidence we have the December renewals are a talking point, not a cliff, and nobody at Halcyon has costed a slip.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 1
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 3
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 1
},
{
"id": "expansion_surface",
"weight": 5,
"score": 4
}
]
},
{
"daysAgo": 2,
"note": "DEMO — Three measurements landed and two things got worse. Verifier at kappa 0.71 against a 0.73 human ceiling, clean baseline at 0.64 on the customer-held 700 with the human rate at 0.91, and the environment stood up in week one with no reset requirement. The FY27 Research Productivity line named in July now has a committee date on 8 October, which moves budget source and lifts the forcing function off the floor. Down: task definability, because the oncology stratum shows Halcyon's own adjudicators at kappa 0.54 and the target population is not yet fully defined, and the sponsor, because Beaumont has missed two of three steering calls before an $799k committee request he is meant to carry. Metric owner still capped at 3 until Wennerberg makes the stratum call. Not yet reflected here: the week-two dev split read 0.66 against a 0.69 trajectory, so the week-four kill gate is live.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 4
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 3
},
{
"id": "baseline_measured",
"weight": 8,
"score": 4
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 4
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 2
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 4
}
]
}
],
"promote": null
}
@@ -0,0 +1,201 @@
{
"key": "northwind-blackwell",
"dealName": "DEMO — Blackwell production block",
"accountName": "DEMO — Northwind Robotics",
"stage": "deployment",
"status": "open",
"acv": 1676300,
"summary": "DEMO — The pick-strategy model cleared its POC at 0.771 top-1 and has carried 25 per cent of the robot cohort at three of Northwind's nine sites since 20 July, on a reserved 64-GPU Blackwell block. Production found a long-payload wave failure the replay harness could not have shown, and the metric owner has moved to the Europe build-out with her successor yet to sign the definition or authorise the ramp to 100 per cent.",
"openedDaysAgo": 240,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — Northwind Robotics: first technical discovery, hard-pick strategy selection",
"status": "final",
"authoredDaysAgo": 232,
"body": "# First technical discovery — Northwind Robotics, three hours on site at Ghent\n\nPresent: Elke Sandberg (Director, Pick Operations), Dr Sanne Brouwer (principal engineer, autonomy), Piet Vandenberg (VP Fleet Engineering) for the first hour. Our side: forward deployed lead plus one forward deployed engineer. Applied Research not in the room.\n\nA note on where this sits. Northwind numbers its autonomy workflows off the Fleet Reliability roadmap — pick-failure cause classification has served all nine distribution centres since March 2025, and recovery-action selection has just been qualified with Marcus Ebele and has not started its POC. This loop is not on that roadmap. It belongs to Elke's Pick Operations organisation, it will be funded from a different line, and it earns nothing from the goodwill of the other two. Scored on its own evidence.\n\n## The task, in one sentence\n\nGiven the item record, four to twelve frames of the pick face and the telemetry from the previous attempt, choose one of 23 grasp strategies for a pick the planner has flagged as hard, or hand it back to the planner's heuristic default.\n\n## The workflow as it runs today\n\nNorthwind runs 4,180 mobile picking robots across nine distribution centres, three of them cold-chain. The planner flags a pick as hard on eleven documented reasons — a deformable item, a crowded face, a slipped previous attempt and so on. That is roughly 2.4 million flagged picks a month out of about 54 million picks. Today the planner picks a strategy from a hand-tuned table that Sanne's predecessor last revised in 2023.\n\nFlagged picks are expensive twice over. They take 11.4 seconds against 4.2 for an ordinary pick, at a mean of 1.9 attempts. And 1.6 per cent of them end in a stall — 38,400 of the roughly 61,000 recovery events the fleet raises each month, which is the majority of the recovery load the fleet-reliability workflows exist to clean up downstream.\n\nElke manages to picks per robot-hour and to intervention-minutes per thousand picks. She quotes both without looking them up and both are on Vandenberg's board slide.\n\n## The failed AI attempt\n\nSanne's team already prompts a frontier vision-language model over the same frames behind a retrieval layer of past picks. Measured properly: 0.61 top-1 strategy match against adjudicated labels, against 0.82 for an experienced pick engineer given the same frames. It plateaued four months ago and two prompt rewrites have moved nothing. That is the encouraging signal, not the discouraging one — the easy headroom is gone.\n\nImportant caveat we wrote down in the room: 0.61 is unconstrained. Nobody has measured what that scaffold scores inside the latency the aisle actually allows.\n\n## Traces, and rights to them\n\nEvery flagged pick is recorded: frames, item record, strategy chosen, attempts, outcome and any subsequent stall. Eighteen months retained, roughly 43 million flagged picks. Northwind's own equipment on Northwind-operated floor for six of the nine sites. Sanne believes the customer-site clause in the three sites Northwind operates for a third-party grocer may restrict off-site processing of face imagery. Flagged for the security brief, not resolved here.\n\n## The verifier\n\nThis is the good news. The fleet sim built for the earlier workflows can replay a recorded pick scene and execute a chosen strategy against it, scoring whether the item was acquired within two attempts without damage or shelf disturbance. Extending its action space to the 23 strategies is scoped at two weeks, not a rebuild. Its agreement with Northwind's pick adjudicators has never been measured. That measurement is the first thing we do.\n\n## Open after the call\n\nNobody would name the budget line. Vandenberg said \"we will find it\" twice, which is not a line. There is no date on anything. And the harness replays one scene at a time, so it will tell us nothing about how this behaves when the warehouse management system releases a whole aisle block at once.",
"kind": "discovery"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Data and security brief: pick-face imagery and the three grocery-contract sites",
"status": "final",
"authoredDaysAgo": 210,
"body": "# Data and security brief — pick-strategy loop\n\nWritten after two sessions with Rob Castellan (Chief Information Security Officer) and one with Northwind's commercial counsel. This brief covers the pick-strategy loop only. Pick-failure cause classification and the recovery-action POC are covered by the workflow 1 brief from 2025 and nothing here reopens them.\n\n## What data moves, and where\n\nThree classes. Pick-face imagery, four to twelve frames per flagged pick. Item records, which carry SKU, weight, packaging class and — for the grocery contracts — a customer product identifier. Planner telemetry from the previous attempt, which carries no third-party data at all.\n\nTraining corpus: 43 million flagged picks over eighteen months, held in Northwind's own object store in Helsinki. Serving traffic: the same three classes, in flight, from the site edge to the model service and back.\n\n## Decisions recorded\n\n**egress-boundary.** Nothing leaves Northwind's VPC. The model service runs on reserved Prime Intellect capacity inside the boundary, peered to the fleet network; Prime Intellect operators have no path to pick imagery or item records. Rob's rule is the one he has applied since the first workflow: imagery that can identify a customer's inventory does not cross an organisational boundary, and a hosted endpoint fails it regardless of contract terms.\n\n**inference-location.** Helsinki region, single region, confirmed in writing by Rob on 2026-01-19. Two consequences we accepted rather than argued: no multi-region failover for the model path, and a hard dependency on the fallback behaving well, because when Helsinki is unavailable there is no second site to route to. That is why the down fallback is a gateway-level switch to the planner's heuristic table and not a second model deployment.\n\n**training-rights.** The master agreement's training-rights schedule, signed for the first workflow, covers Northwind's own operational data by class rather than by workflow. Counsel confirmed it extends without amendment. Northwind owns the adapted weights; we hold no rights to them and no rights to the corpus beyond the engagement.\n\n**the three customer-site contracts.** This was the live issue at discovery and it is the reason this brief exists. Rotterdam, Lyon and Zaragoza are operated by Northwind under contract to a grocery group, and clause 11.4 of that contract restricts processing of face imagery to \"operational purposes on site\". Counsel's reading is that model training is not an operational purpose on site. Resolved two ways: a clause amendment was requested in January and is expected in Q3, and until it lands those three sites are excluded from training and from the held-out set entirely. Rob asked for the exclusion to be enforced at the corpus builder rather than by convention, and Väinö Salo implemented it as a site allowlist with a failing CI test.\n\nThat exclusion is also why the initial deployment sites are Ghent, Vantaa and Ede. They are the three highest-volume sites Northwind operates for itself, which means nothing in the first ramp depends on an amendment nobody controls.\n\n## Retention and deletion\n\nModel service holds no request payloads. The live-eval sampler writes its sample to Northwind's store under the same eighteen-month retention as the source traces. Northwind's existing deletion path applies unchanged; there is nothing to delete on our side.\n\n## What is not settled\n\nWhether the grocery amendment, if it lands, permits the three grocery-contract sites into training or only into serving. Counsel expects the narrower reading. Nothing in this deal's plan depends on the wider one.",
"kind": "discovery"
},
{
"templateSlug": "reference-architectures",
"stage": "scoping",
"title": "DEMO — Reference architecture: fleet sim, serving path and the tail budget",
"status": "final",
"authoredDaysAgo": 196,
"body": "# Reference architecture — pick-strategy model, fleet sim, serving path\n\nChosen shape: adapted open weights, served inside Northwind's own boundary on reserved Prime Intellect capacity. Rejected shapes recorded below so nobody reopens them in month four.\n\n## Serving shape\n\nAn adapted open-weights 32B vision-language model, weights owned by Northwind, served on a reserved Blackwell block in the Helsinki region with a warm floor. The A100 fleet that carries pick-failure classification, and that the recovery-action policy will join at its own cutover, is untouched: those two stay where they are, on their own reservation and their own renewal date, and this block does not serve them. That separation is deliberate and Rob Castellan asked for it in writing.\n\nRejected: a hosted frontier endpoint (fails the egress-boundary decision on face imagery and cannot meet the tail budget from outside the region); on-robot inference (32B does not fit the compute on the current manipulator, and Sanne will not gate a fleet hardware upgrade on a model that changes monthly); reusing the A100 fleet (it has no headroom at the flagged-pick rate and the fault detector's latency would suffer).\n\n## Component table\n\n| Component | Owner | Notes |\n| --- | --- | --- |\n| Pick gateway | Northwind | Receives the planner's hard-pick flag, assembles frames, item record and prior-attempt telemetry, calls the model |\n| Model service | Prime Intellect capacity, Northwind weights | Two replicas in Helsinki, warm floor of two, no scale-to-zero |\n| Planner executor | Northwind | Executes the returned strategy, or the heuristic table on abstain, timeout or switch |\n| Fleet sim replay harness | Northwind | The verifier, and the environment; runs out of band on recorded scenes |\n| Live eval sampler | Joint | Draws the daily stratified sample from served traffic into the harness |\n\n## The verifier at serve time\n\nThe harness cannot score a live pick. What it checks is whether the item was acquired within two attempts without damage, and that is known only when the attempt sequence finishes, up to forty seconds later. So the verifier runs out of band, on a delay, and it cannot be the confidence signal for abstention. That constraint is what forces a separate inline proxy, and it is the single most consequential line in this document.\n\n## Latency budget\n\nThe aisle is the constraint, not the operator. A robot holding position at a pick face blocks it, and at Ghent a face blocked longer than four seconds backs up the two robots behind it. Budget agreed with Sanne: p95 900 ms, p99 1,400 ms, measured gateway to planner-executor and including frame assembly. The planner executor abandons the call at 1,500 ms and uses the heuristic table.\n\n## Environment specification\n\nThe fleet sim is the environment, extended with the 23-strategy action space. Case types are Northwind's own flag reasons: deformable or crushable item (28 per cent of flagged picks), crowded or overhanging face (22), previous attempt slipped (17), low-confidence pose estimate (13), mixed-SKU tote (11), and a tail of six reasons covering the remaining 9 per cent.\n\n## What the harness does not do\n\nTwo gaps, both known and both carried forward. It replays scenes one at a time, so it has no concept of a wave and nothing built on it will predict behaviour under correlated arrivals. And it replays each scene at its own recorded context length without ever telling us what the distribution of context length looks like inside a burst — flagged picks in a wave come from the same aisle block and carry the longest payloads the path produces. A burst mode is on Sanne's backlog and is not funded. Both gaps go into the load test as a synthetic arrival generator instead, which is a weaker instrument and should be recorded as such.",
"kind": "architecture"
},
{
"templateSlug": "pricing-and-packaging",
"stage": "proposal",
"title": "DEMO — Reserved Blackwell block: sizing and pricing inputs",
"status": "final",
"authoredDaysAgo": 168,
"body": "# Pricing and packaging inputs — reserved Blackwell block\n\n## What is being bought\n\nTwelve months of reserved Blackwell capacity: 64 B200 GPUs, Helsinki region, warm floor guaranteed, at an illustrative effective $2.99 per GPU-hour — a Helsinki rate against a twelve-month commitment, and not the rate on the smaller fleet-reliability block. Contract value $1,676,300 for the term, or $139,692 a month. This replaces nothing: the A100 inference fleet carrying the fleet-reliability workflows continues on its own reservation and its own renewal date, which is deliberate so that a renegotiation of one does not open the other.\n\n## The split\n\n| Allocation | GPUs | Monthly | What it carries |\n| --- | --- | --- | --- |\n| Serving | 16 | $34,923 | Two replicas in Helsinki with a warm floor of two; sized for the peak of a wave |\n| Training and replay | 48 | $104,769 | Monthly retraining, the fleet sim at corpus scale, live-eval scoring |\n\nThe serving allocation is not sized against mean throughput, and any argument that starts from the mean is wrong. At all nine sites the mean is under one call a second, which no serious block is built for. Northwind's arrivals are bursty by construction — the warehouse management system releases work an aisle block at a time — so serving is sized against concurrency at the peak of a release, against the longest payloads the path produces, and against the fact that a cold start inside a wave is a blocked pick face. Ruth Ackerman (Finance Director, Pick Operations) pushed hard on this and accepted it when Sanne put the face-blocking arithmetic beside it.\n\nReserved rather than on-demand for two reasons Ruth named herself. Pick Operations budgets annually against picks, so a variable inference line is worse than a larger fixed one. And the training allocation is not spiky: retraining is monthly and the harness runs the live eval every day.\n\n## The unit economics that decided it\n\nServing cost per flagged pick, at the serving allocation only:\n\n| Ramp | Flagged picks served per month | Cost each |\n| --- | --- | --- |\n| Three sites, 25 per cent of robots | 200,000 | $0.175 |\n| Three sites, all robots | 800,000 | $0.044 |\n| All nine sites | 2,400,000 | $0.015 |\n\nAgainst that: at all nine sites, a 22 per cent reduction in stalls on flagged picks prevents about 8,400 recovery events a month, which Northwind costs at $4.10 when the recovery policy clears it and $11.60 when it reaches a teleoperator — $47,000 a month at today's escalation mix. Add roughly 2,700 robot-hours returned from the attempt reduction and the prevented blocks, which Elke costs at $21, and the case is about $105,000 a month against $34,923 of serving.\n\nSay the uncomfortable half plainly, because Ruth will find it anyway: that is a good return on the serving allocation and it does not, on its own, pay for the training allocation until the workflow is at all nine sites. Ruth has that in writing.\n\n**Ruth's bar, stated in writing:** serving cost below $0.05 per flagged pick served, which the table above reaches only at 100 per cent of the three initial sites. It is not met at the opening ramp and is not claimed to be.\n\n## Commercial structure\n\nSingle twelve-month reservation, invoiced quarterly in advance, starting on the go-live date rather than on signature — conceded during procurement to cover the POC extension and the shadow period, and worth roughly four months of the term on the current plan. Northwind owns the adapted weights outright. There is no per-inference or per-seat component anywhere in this deal, which was the deciding difference against the API vendor Vandenberg was also talking to.\n\n## What is not priced here\n\nApplied Research time beyond the POC. Support is business hours with a four working hour response target. The remaining six sites are a second reservation and a separate conversation.",
"kind": "pricing"
},
{
"templateSlug": "post-training-poc",
"stage": "poc",
"title": "DEMO — POC structure and readout: strategy selection on 1,400 held-out flagged picks",
"status": "final",
"authoredDaysAgo": 62,
"body": "# POC structure and readout — hard-pick strategy selection\n\n## The one question\n\nOn a held-out set of 1,400 flagged picks sampled from March to May production traffic across the four distribution centres on the current manipulator generation that Northwind operates for itself, frozen on 2026-05-18 and held by Northwind, an adapted open-weights 32B vision-language model will score at least 0.74 top-1 strategy match against a prompted frontier baseline of 0.61, as measured by the fleet sim replay harness, whose agreement with Northwind's pick adjudicators is at least Cohen's kappa 0.70 on a 260-case dual-labelled audit where the human-to-human ceiling is kappa 0.81.\n\nThat sentence is the agreement. Nothing else in this document restates it.\n\n## Result\n\n0.771 on the sealed set at the final readout, on the third of the four scored submissions in the budget. Verifier agreement measured at kappa 0.72 against adjudicated labels against a ceiling of 0.81, which clears the library floor on the absolute branch and not only the ceiling-relative one. Second metric, abstain-correctness: 0.83 recall on picks an adjudicator judged genuinely unservable by any of the 23 strategies.\n\n## How the sets were built\n\nFour splits. Train and dev on flagged picks before 1 March. Held-out on March to May. Out-of-distribution on the two sites running the newer manipulator generation, which are outside the held-out population by construction so the two splits do not overlap. Rotterdam, Lyon and Zaragoza are excluded from every split by the site allowlist in the corpus builder, per the security brief.\n\nStratified by flag reason, site, shift and by whether the pick arrived inside a wave. Near-duplicate detection on scene embeddings found 4.1 per cent overlap between train and held-out on recurring failures at a single Vantaa pick face; those cases were removed from train and the split re-hashed in week two.\n\nThe held-out set never left Northwind. Sanne ran it; we received aggregates.\n\n## The proxy problem, resolved here\n\nThe harness scores an outcome known up to forty seconds after the fact, so it cannot gate a live answer. The POC therefore carried a second deliverable: a cheap inline confidence proxy over the model's own logit margin and the flag reason, calibrated against the harness verdict. On held-out data it reaches 0.61 recall of would-be-wrong strategies at 0.88 precision. That is the abstention signal in production and it is weaker than anyone wanted. It is why deformable-item picks stay on the heuristic table at go-live unless the proxy is confident.\n\n## Out-of-distribution\n\nOn the newer manipulator generation the model scores 0.69, eight points below held-out. Expected and accepted; those two sites are out of scope for this deployment.\n\n## What the POC did not test, and what we failed to get\n\nArrival shape. Every held-out case was replayed in isolation because the harness cannot do otherwise, so the POC says nothing about behaviour when an aisle block lands at once, and the readout said so in writing.\n\nAnd the number we still do not have: the frontier scaffold measured under the 900 ms budget rather than unconstrained. Sanne's team ran out of engineers in April and it was dropped. Our 0.771 is therefore compared against an unconstrained 0.61, which flatters us, and I have said so to Elke twice. It stays open and it holds the baseline dimension down.\n\n## Readout\n\nDelivered to Elke Sandberg, Sanne Brouwer, Ruth Ackerman and Piet Vandenberg. Signed as met on 2026-06-18. Elke asked the only hard question in the room: whether 0.771 removes pick engineers. The honest answer given was no — the first year's return is robot-hours and prevented stalls, not headcount.",
"kind": "poc"
},
{
"templateSlug": "production-readiness",
"stage": "deployment",
"title": "DEMO — Production readiness: pick-strategy model on the Blackwell block",
"status": "draft",
"authoredDaysAgo": 9,
"body": "# Production readiness — pick-strategy model on the Blackwell block\n\nAt 25 per cent of the cohort at Ghent, Vantaa and Ede since day 12. Every gate closed before traffic moved; two ungated items are open. Draft: the p99 line was rewritten once.\n\n## The inference path\n\nHard-pick flag to gateway, which assembles frames, item record and prior-attempt telemetry; model service on the reserved block in Helsinki; strategy back to the executor, or an abstain onto the table. Northwind's weights, pinned by hash, warm floor of four replicas, no scale-to-zero, in the region Rob Castellan confirmed against the egress boundary. The verifier only knows an outcome forty seconds later, so it runs out of band and abstention rides the POC proxy: 0.61 recall, 0.88 precision.\n\nBudget, owned by Sanne Brouwer in writing: p95 900 ms, p99 1,400 ms, gateway to executor. Load test before go-live, 24 concurrent for 20 minutes at the median payload: p99 1,180 ms. Inside budget, and wrong.\n\n## The surprise\n\nDay 4, at 5 per cent: Vantaa released an aisle block, 214 flagged picks in 38 seconds. Never more than seven calls in flight — but each carried what a wave contains, twelve frames and 60 seconds of telemetry, 5.4x the median prompt, so prefill queued behind itself. Measured p99 that minute: 3,240 ms. Eleven picks passed the executor's 1,500 ms abandon; four became stalls and two pick faces backed up. The test was not wrong about concurrency. It was wrong about what a wave contains.\n\nTwo changes: admission control, which sheds the three lowest-value flag reasons to the table above 90,000 queued prefill tokens, and the warm floor from two replicas to four, which is where 16 serving GPUs stopped looking generous. Retested at the design peak, 22 concurrent at the 99th-percentile payload: p99 1,290 ms, held eight days.\n\nSecond-order effect, worse and open: shedding drops exactly the picks most likely to stall, in the minute a site is already bursting, so it all reaches the teleoperator queue at once. Day 9 that queue peaked at 41 teleoperators-worth of work against a rota of 34. That rota is sized against the recovery-action policy's post-cutover figure of 31.5 teleoperator minutes per thousand robot-hours, down from 47.0 — so admission control is spending headroom the fleet-reliability programme delivered, on a workflow that does not report to Marcus Ebele. Flagged to him on day 6; his answer was that the rota is not ours to spend.\n\n## The three fallbacks\n\nWrong: abstain below the proxy bar; deformable-item picks stay on the table regardless, because that is where the proxy is weakest. Slow: executor abandons at 1,500 ms onto the table — day 4 showed that converts latency into stalls, not into nothing. Down: Kwame Adjei's no-deploy switch, config not a release. There is no second region, by decision, so that switch is the whole of it.\n\n## The ramp\n\n- Shadow, 14 days from 2026-07-06, all three sites: harness agreement 0.768 against held-out 0.771.\n- 5 per cent of the cohort from day 0, 2026-07-20; 25 per cent from day 12.\n- 100 per cent of the three sites by 1 November, when peak season starts and Ruth Ackerman's $0.05 bar first clears.\n\n## Live eval and trigger\n\n640 flagged picks a day, about 10 per cent of served traffic, stratified by flag reason and wave arrival, scored by the harness on the POC rubric; 1,600 at full ramp to hold per-reason cells above 30. Human-rated stream, 45 a week. Baseline over the first eight days: mean daily top-1 0.763, band 0.730 to 0.796 — the 1.96 standard-error band at n=640. Rolling reading 0.759; wave arrivals 0.734, bottom decile.\n\nRetrain below 0.730 for three consecutive weekly readings, pulled by Sanne. Product trigger: the Europe build-out adds two manipulator generations, which no monitor sees; Sanne tells us before it ships. Second product trigger, borrowed from workflow 2's runbook: the recovery-action policy retrains monthly, and a change in how it clears a stall changes the outcome our own harness scores forty seconds later. The flag-reason distribution monitor is unbuilt — ungated, open, the first thing I would fix.\n\n## Paging, handover, open\n\nAdjei's rota owns the path, the switch and the fallbacks, verified in the paging tool rather than verbally; the runbook says when to wake us. Rollback and the down drill passed day -2, before any traffic; retrain-and-reproduce cleared day 21, Väinö Salo driving; rebuild from source is open, day 30.\n\nElke Sandberg moved to the Europe build-out in June. Dale Whitaker has signed neither the revised intervention-minutes definition nor the ramp, and has deferred the review twice. Every month at 25 per cent is $26,000 of serving capacity carrying no traffic, which makes that signature a commercial problem.",
"kind": "playbook"
}
],
"scores": [
{
"daysAgo": 230,
"note": "DEMO — Scored off the discovery notes. This loop sits under Pick Operations rather than the Fleet Reliability roadmap, so it earns nothing from the two live workflows and is scored on its own evidence: the fleet sim is a genuine computable verifier and extends in two weeks, which carries the deal. Against that, nobody would name a budget line, there is no date on anything, and the frontier scaffold has only been measured unconstrained rather than under the 900 ms aisle budget.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 3
},
{
"id": "baseline_measured",
"weight": 8,
"score": 2
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 2
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 26,
"note": "DEMO — Up on six with a reason each: task definability 4 on the frozen 23-strategy action space and eleven flag reasons, headroom 4 on 0.61 to 0.771, traces 4 and security 4 on Rob Castellan's written egress and inference-location decisions plus the site allowlist that quarantines the three grocery contracts, budget 4 on Ruth Ackerman's Pick Operations opex line, forcing function 4 on peak season making 1 November a hard date for 100 per cent of the three sites. Held deliberately: verifier quality at 3, because kappa 0.72 is measured but the harness still cannot score a live pick and a 0.61-recall proxy carries abstention; baseline at 3, because the latency-constrained frontier number was never run; exec sponsor at 3, because Piet Vandenberg attends and Ruth Ackerman signs the money, and neither has ever carried this loop internally the way Elke Sandberg did; expansion at 3, because the remaining six sites are a second reservation nobody has costed. Down on two: environment constructibility to 2, since the harness's blindness to arrival shape and payload length caused a production incident rather than a caveat, and metric owner to 2, since Elke Sandberg moved to the Europe build-out and Dale Whitaker has signed nothing.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 4
},
{
"id": "baseline_measured",
"weight": 8,
"score": 3
},
{
"id": "headroom",
"weight": 10,
"score": 4
},
{
"id": "env_constructibility",
"weight": 8,
"score": 2
},
{
"id": "metric_owner",
"weight": 9,
"score": 2
},
{
"id": "budget_source",
"weight": 9,
"score": 4
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 4
},
{
"id": "forcing_function",
"weight": 8,
"score": 4
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
}
],
"promote": null
}
@@ -0,0 +1,203 @@
{
"key": "northwind-fleet-expansion",
"dealName": "DEMO — Fleet expansion — 32× B200",
"accountName": "DEMO — Northwind Robotics",
"stage": "expansion",
"status": "won",
"summary": "DEMO — Won: second workflow at Northwind, the recovery-action policy for the mobile picking fleet, live in seven of nine DCs and backed by a 12-month 32× B200 reserved commitment signed on 2026-06-24. The case study is approved for named use; workflow 3 was scoped in July and is blocked on Ingrid Halloway's dock-manoeuvre autonomy threshold sign-off, outstanding since June — a separate item from the shelf-collapse escalation rule she has also not answered since April.",
"openedDaysAgo": 268,
"artefacts": [
{
"templateSlug": "trainability-qualification",
"stage": "qualification",
"title": "DEMO — Trainability scorecard: recovery-action selection (workflow 2)",
"status": "final",
"authoredDaysAgo": 255,
"body": "# Trainability and deal qualification — workflow 2, recovery-action selection\n\nSecond workflow at Northwind Robotics. The first (pick-failure cause classification) has been serving all nine distribution centres since March 2025 and is now gated by Northwind's own eval. This scorecard covers a different loop and is scored on its own evidence, not on the goodwill of the first deployment.\n\nNote for anyone reading across accounts: this is the mobile picking fleet, not the 1,900 fixed cold-chain manipulators, which are a separate programme with their own remote-operations sponsor and their own action space. Nothing here shares a model, an eval or a metric owner with that programme.\n\n## The task\n\nNorthwind runs 4,180 mobile picking robots across nine DCs. When a robot faults or stalls, the on-board planner picks one of nine recovery manoeuvres or escalates to a human teleoperator. Fleet-wide there are roughly 61,000 recovery events a month, of which about 18,400 escalate. Marcus Ebele (Director, Fleet Reliability) is judged on teleoperator minutes per thousand robot-hours and on the safety-critical miss rate — a stall that should have escalated and did not.\n\nThe decision is definable: nine manoeuvres plus escalate, taken from the existing planner's action space. It is not clean. Two manoeuvres (\"reposition and retry\" and \"shelf-clear and retry\") are used interchangeably by the fleet engineers today, which is the first thing the eval will have to settle. Scored 3 until there is a written decision rule an adjudicator can apply.\n\n## Verifier\n\nMostly computable. The sim replays the robot state and scores whether the chosen manoeuvre actually cleared the fault within the 90-second recovery budget. Escalation correctness is not computable — it needs an adjudicated label for \"should a human have been called\", and that is where the rubric judge sits. Provisionally scored 3, not 4, and the reason is written down here so it can be argued with later: we are assuming the adjudication apparatus from workflow 1 transfers, and that assumption has not been tested.\n\n## Baseline\n\nThe weakest dimension by a distance. The incumbent heuristic planner has never been measured on a frozen set. Marcus quotes \"about seventy per cent\" from a 2024 internal review nobody can locate. Scored 2. Nothing gets contracted until the heuristic is re-run unchanged on a frozen set we build first.\n\n## Data and environment\n\nTelemetry, camera frames and teleoperator session logs are Northwind's own and already covered by the MSA training rights schedule signed for workflow 1. No new legal path required. The fleet sim built for workflow 1 exists and has a maintained CI; it does not yet model the recovery action space, which is scoped at roughly two weeks of extension rather than a rebuild.\n\n## Commercial\n\nRhea Sundaram (VP Autonomy) sponsors, as she did the first time. Dana Iyer (Head of Infrastructure Finance) owns the line but has not agreed that reserved capacity rather than on-demand is the right instrument. The forcing function is soft: peak season begins in October and Marcus wants the escalation rate down before it, but no date is contractual.\n\n## Result\n\n80% weighted — Build. Conditions: freeze a held-out set and re-run the heuristic on it before any training; validate the transferred rubric on the new label space before trusting it with reward.\n",
"kind": "qualification"
},
{
"templateSlug": "post-training-poc",
"stage": "poc",
"title": "DEMO — POC structure: autonomous recovery-action selection",
"status": "final",
"authoredDaysAgo": 203,
"body": "# POC structure — autonomous recovery-action selection\n\nEight weeks, paid, on on-demand capacity billed against the workflow 1 order form. There is no reserved block yet and the POC must not assume one. Kick-off 2026-01-30, exit date agreed in advance: 2026-03-27.\n\n## The one question\n\nOn a held-out set of 900 recovery events sampled uniformly by day from Northwind production traffic between 2026-02-01 and 2026-03-14, to be frozen on 2026-03-20 and held by Northwind, an adapted 32B open-weights policy will score at least 85% agreement with the adjudicated action label, against the incumbent heuristic planner's baseline re-run unchanged on the same 900, as measured by the sim verifier plus the escalation rubric judge, whose agreement with Northwind's adjudicators must be at least 88% on a 200-case dual-labelled audit drawn from the dev split, where the human-to-human ceiling is 92%.\n\nSecondary threshold, and the one Marcus actually cares about: the safety-critical miss rate (should have escalated, did not) must not rise. Baseline to be established on the same frozen 900.\n\n## Datasets and how they relate\n\n- Training pool: 71,000 recovery trajectories from 2025-06 to 2025-12, strictly predating both eval sets.\n- Dev split: 700 events from 2026-01, adjudicated, used for iteration, for rubric revision, and for the 200-case judge audit.\n- Held-out set: the frozen 900 described above. Excluded from training and from verifier development, including from the judge audit — that exclusion is why the audit is drawn from the dev split. Marcus Ebele confirms it in writing at freeze.\n\nThe dev split and the held-out set are disjoint by construction and by date. The nine DCs are represented in the held-out set in proportion to their event volume, which means DC-3 and DC-7 together are 31% of it; per-DC recall is reported separately so the aggregate cannot hide a single-site collapse.\n\n## Verifier design\n\nRoughly 78% of the trajectory is programmatically verifiable: the sim executes the chosen manoeuvre against the recorded robot state and scores fault cleared within 90 seconds, collision-free, no shelf disturbance. The remaining 22% is the escalate-or-not judgement and the free-text intervention note handed to the teleoperator, both judged by a rubric model.\n\nThe rubric is inherited from workflow 1 and must be re-validated before it carries reward. Adjudicator calibration is scheduled for week 2 with a defended cut of 0.75 Krippendorff alpha, the same cut Northwind's adjudicators defended last year. Before that, the label space itself has to be written down: \"reposition and retry\" against \"shelf-clear and retry\" is currently a matter of habit, and an adjudicator cannot be calibrated against a habit.\n\n## Recipe\n\nSupervised fine-tuning on the 71,000 trajectories, then RL in the extended fleet sim against a composite reward: manoeuvre success in sim at 0.55, escalation correctness at 0.30, intervention-note rubric at 0.15, with a hard zero on any trajectory producing a simulated collision. Because the note carries the least weight it gets an explicit regression check on the dev split each run.\n\nBudget: 8,900 GPU-hours over a fourteen-day training window on 32 B200s, about 83% occupancy. The nine-day sim extension runs before that window and uses almost no GPU; it is wall-clock, not spend.\n\n## How this fails visibly\n\nIf the re-run heuristic comes back above 82%, the headroom argument is dead and we say so on 2026-03-27 rather than arguing about the metric. If adjudicator alpha will not clear 0.75 after two calibration rounds, the escalation half of the task is undefined and the POC scope shrinks to manoeuvre selection only.\n",
"kind": "poc"
},
{
"templateSlug": null,
"stage": "poc",
"title": "DEMO — Transfer accounting: what workflow 1 paid for and what we re-paid",
"status": "final",
"authoredDaysAgo": 166,
"body": "# What transferred from workflow 1, and what we re-paid for in full\n\nWritten at week 6, on 2026-03-06, so the claim is auditable later. Internal cost figures are Prime Intellect engagement accounting, not anything Northwind is billed against.\n\n## The claim\n\nWorkflow 1 (pick-failure cause classification) ran 19 weeks and cost $1.42m to deliver. Workflow 2 (recovery-action selection) is tracking to 11 weeks and $870k. That is the number to quote, because it is the only one that nets the savings against the one line that rose. The difference is not that we got better at it. It is that three specific assets were already paid for, and one was not.\n\n## What transferred\n\n**The adjudication apparatus.** Northwind has eleven trained adjudicators — fleet engineers who spend four hours a week labelling. Standing them up the first time cost 34 person-days: recruitment inside the reliability org, two days of rubric instruction, three calibration rounds before alpha cleared the defended cut of 0.75. This time we re-used the same eleven people, the same instruction material, the same cut, and the same argument for why 0.75 is the right cut rather than 0.8. Cost this time: 6 person-days.\n\n**The environment scaffolding.** The fleet sim — state replay, telemetry decoding, camera-frame playback, CI on every commit — was seven weeks of build in workflow 1. Extending it to execute the nine recovery manoeuvres took 9 days. The action space was new; nothing under it was.\n\n**The commercial paper.** The workflow 1 reference architecture and the price book went across unchanged: same serving topology, same tail-latency budget shape, same unit rates. No scoping architecture document and no new pricing work exists for workflow 2 because neither was needed, and that is a real saving of about three weeks of lead time, not an omission.\n\nTogether, roughly 28 person-days of adjudicator standing-up, five weeks of environment build and three weeks of architecture and pricing did not have to happen twice.\n\n## What did not transfer\n\n**The eval.** The label space is different — nine manoeuvres plus escalate, against workflow 1's six root-cause classes. Nothing about the old adjudicated corpus is usable. Worse, the label space had to be defined before it could be populated: \"reposition and retry\" and \"shelf-clear and retry\" now separate on wedge state and shelf integrity under a one-page written rule, which the adjudicators applied rather than argued about. We built a new dev split (700) and a new held-out set (900), and the adjudication of those 1,600 events cost 41 person-days against workflow 1's 36. The eval got more expensive, not less, because the held-out set is larger and each case takes an adjudicator longer: judging the correct next action requires watching the recorded recovery, where judging root cause did not.\n\nIn engagement accounting: eval construction was $128k the first time and $141k the second. It is the one line that went up, and it went up inside a total that came down by $550k.\n\n## The thing that nearly went wrong\n\nThe rubric transferred; the adjudicators' habits did not transfer with it. Round one of calibration came back at 0.61 alpha, below the defended cut. The cause, found on 2026-02-19: the same eleven people had spent a year labelling by root cause, and half of them were still doing that — marking \"shelf-clear\" whenever the shelf was the underlying problem, regardless of whether shelf-clear was the right next action for a robot already wedged. One page of counter-examples and a second round fixed it, at a cost of 6 person-days rather than 34.\n\nThat is the honest version of transfer. The apparatus carried over; the reflex it was trained on had to be retrained, and if we had trusted the inherited rubric with reward without re-validating it, the entire escalation half of the reward would have been measuring the wrong thing for the fourteen-day training window.\n\n## What to write down for workflow 3\n\nThe sim, the adjudicator pool, the cut, the architecture and the price book transfer. The eval does not, and should be budgeted at full price every time. Assume the rubric needs one calibration round against any new label space before it carries reward.\n",
"kind": "pricing"
},
{
"templateSlug": "procurement-and-budget-path",
"stage": "procurement",
"title": "DEMO — Procurement and budget path: 32× B200 reserved, 12 months",
"status": "final",
"authoredDaysAgo": 50,
"body": "# Procurement and budget path — 32 B200s, 12-month reserved\n\nWritten 2026-06-30, after signature, while the path is still recoverable from memory.\n\n## The instrument\n\n32 B200s reserved for 12 months, $1,047,859. Signed 2026-06-24. It replaces on-demand consumption that ran at roughly $118k a month across the POC and the workflow 1 retraining cadence. The POC itself ran on that on-demand line against the workflow 1 order form; nothing was reserved until this signature, which is why the run-rate argument below is measured rather than modelled.\n\n## Budget line and its source\n\nFleet Reliability capital, FY26 line \"autonomy platform compute\", owned by Dana Iyer (Head, Infrastructure Finance). This is a named line with a named owner and a stated FY26 figure of $1.4m, which is what moved this dimension from 3 to 4. At the first scorecard the line existed but Dana had not agreed that reserved was the right instrument; she agreed on 2026-06-02 after we showed the twelve-month on-demand projection against the reserved price.\n\n## The path, as it actually ran\n\n1. Marcus Ebele raised the requisition on 2026-06-04 against the existing MSA. No new master agreement was needed — workflow 1's MSA and its IP and training-rights schedules cover this scope, which removed about six weeks from the path.\n2. Finance review with Dana, 2026-06-09. The question was not price, it was utilisation risk: a reserved block that sits idle is a line item she has to defend.\n3. Security: no review. Rob Castellan confirmed in writing on 2026-05-02 that nothing crosses a new boundary.\n4. Rhea Sundaram approved above her $750k delegation threshold via the CTO on 2026-06-19.\n5. Signature 2026-06-24.\n\n## The utilisation argument, and the honest version of it\n\nDana's forecast, which we did not dispute: months 1 to 4 land at about 62% utilisation, because workflow 2 training is done and workflow 3 is not yet scoped as of today. Nothing from Pick Operations lands on this block either: the hard-pick strategy loop Elke Sandberg's organisation is deploying runs on its own reserved Blackwell block in Helsinki, on a separate line, and Dana asked twice to be sure it was not being counted here. Months 5 to 12 depend entirely on whether the dock-approach workflow starts.\n\nWe did not paper over this. The commitment was justified on two things Dana could check: the observed on-demand run rate over the previous seven months, and the monthly retraining cadence both live workflows now carry, which is a floor rather than a hope. Anything above that floor is upside that has to be earned, and if workflow 3 does not start by January the block is over-bought for two quarters. Dana has that in writing and accepted it.\n\n## Open\n\nThe renewal conversation is in May 2027 and its outcome is a function of workflow 3 starting. The blocker on workflow 3 is not budget and not security — it is Ingrid Halloway's sign-off on the dock-manoeuvre autonomy threshold, requested on 2026-06-11 and unanswered. That is a different item from her open question on the shelf-collapse escalation rule in production, and conflating the two has already cost us one wasted meeting.\n",
"kind": "pricing"
},
{
"templateSlug": "production-readiness",
"stage": "deployment",
"title": "DEMO — Production readiness: recovery-action policy across nine DCs",
"status": "final",
"authoredDaysAgo": 25,
"body": "# Production readiness — recovery-action policy, nine DCs\n\nPOC exit cleared on 2026-03-27. Written before shadow start to gate production load, and updated on 2026-07-25 now that full cutover has six weeks behind it; the ramp and fallback sections record what actually happened.\n\n## Inference path\n\nAdapted 32B open-weights, served inside Northwind's own VPC on their existing A100 inference fleet, not on training capacity. Path: robot fault event to the DC edge broker, to the fleet-orchestration service, to the policy endpoint, back with a manoeuvre code and an intervention note. The escalation rubric judge runs out of band on a 5% sample for drift, not inline; inline judging blew the latency budget in load test and buys nothing at decision time.\n\nLatency budget, owned by Marcus Ebele: p95 under 400 ms, p99 under 900 ms, measured at the orchestration service, not at the model. A stalled robot occupies an aisle, so a slow decision is a physical queue and not merely a bad user experience. Load test at 4.2x mean concurrency, held 40 minutes, 99th-percentile context (the long-tail cases are the ones with 60 seconds of prior telemetry attached): p95 310 ms, p99 780 ms, cold start 4.1 s, mitigated by keeping two warm replicas per region.\n\nServing location matches the `inference-location` decision in the workflow 1 Data and Security Brief; Rob Castellan (CISO) confirmed no new review was required on 2026-05-02 because nothing crosses a new boundary.\n\n## Ramp\n\n- Shadow, 14 days, all nine DCs: policy scored on every eligible event, heuristic still acting. Agreement with the adjudicated label was measured on a 400-event adjudicated sample drawn across the fourteen days — that is what the eleven adjudicators can absorb at four hours a week — and held at 88.6%, within noise of the held-out 89.2%.\n- 10% of events in DC-3 and DC-7 only, 10 days.\n- 50% in those two DCs, 14 days.\n- 100% in seven DCs from 2026-06-08. DC-2 and DC-9 stay on the heuristic as a control cohort through the peak-season window, at Marcus's request and to our benefit.\n\n## Fallback\n\nAny of: policy unavailable, p99 breach sustained over 5 minutes, or a manoeuvre code outside the nine — the orchestration service falls back to the heuristic planner, which stays deployed and tested monthly. Fallback is automatic and does not page. Two fallbacks fired in the first six weeks after cutover, both during a broker restart.\n\n## Live eval and retraining\n\nNorthwind adjudicates 120 sampled events a week and runs the held-out eval monthly; the eval gates their own deploys, ours included. Retraining trigger, agreed with Marcus: adjudicated agreement below 86% on two consecutive weekly samples, or safety-critical miss rate above 1.2% on any single week.\n\n## Open at go-live, and still open now\n\nThe sim does not reproduce shelf-collapse contact physics, so roughly 9% of real recovery events — 81 of the frozen 900 — have no faithful simulated counterpart and the policy is effectively unverified on them. Those events are routed to escalation by rule rather than by policy. This is a known, accepted gap, not a solved one, and it constrains both the environment and the verifier: a verifier that cannot score 9% of production traffic is not a finished verifier. Ingrid Halloway (Safety Officer) has not signed off on removing that rule, and has not answered the last two requests to discuss it, the first of which went out in April.\n",
"kind": "playbook"
},
{
"templateSlug": "case-study-frame",
"stage": "expansion",
"title": "DEMO — Case study: Northwind Robotics recovery-action policy",
"status": "final",
"authoredDaysAgo": 4,
"body": "# Case study — Northwind Robotics, autonomous recovery-action selection\n\nApproved by the customer for named use. Second workflow at this account.\n\n**The customer and the workflow.** Northwind Robotics runs 4,180 mobile picking robots across nine distribution centres, roughly 61,000 recovery events a month. When a robot faults, something must choose one of nine recovery manoeuvres or call a teleoperator. The heuristic escalated about 18,400 events a month at a median session of 4 minutes 20 seconds. The expensive failures ran the other way: a stall that should have escalated and did not blocks an aisle until a technician walks to it.\n\n**Why not prompting.** Four months against a frontier model plateaued at an informal 74%: the decision needs 60 seconds of telemetry and a camera frame, and teleoperator corrections were logged and discarded.\n\n**What we built, and what was hard about it.** Almost everything was second-hand; the eval was the whole cost. The hard part came before labelling: two of the nine manoeuvres were used interchangeably by fleet engineers, so nobody could be calibrated until \"reposition and retry\" and \"shelf-clear and retry\" were separated in writing, on wedge state and shelf integrity. That rule took two weeks. The inherited verifier covered the mechanics and none of the judgement: a rubric pointed at an unseen label space is not a verifier yet. The environment was nine days of extension on a sim that cost seven weeks the first time.\n\n**How we know the verifier holds.** Rubric judge against two Northwind adjudicators on 200 dual-labelled dev-split cases, never the held-out 900: judge-to-human 90%, human-to-human 92%. Two reward hacks: over-selecting \"reposition and retry\" where the sim under-charged its time cost; and escalating under ambiguity to flatter the safety metric. Patched with a time penalty and an escalation cost. The sim cannot reproduce shelf-collapse contact physics, so 81 of the frozen 900 (9%) escalate by rule; 60 of those were read by hand by Northwind's engineers.\n\n**The numbers.**\n\n| Measure | Baseline | Result | Measured on | Measured by |\n| --- | --- | --- | --- | --- |\n| Agreement with adjudicated action label | 71.4%, heuristic re-run unchanged | 89.2% | The frozen 900, eval-v2 | Northwind fleet reliability; runs r-4462 (2026-03-23) and r-4471 (2026-03-24), Yusuf Adeyemi |\n| Escalation rate | 30.1% | 17.6% | The frozen 900, eval-v2 | Northwind; runs r-4462 and r-4471 |\n| Safety-critical miss rate | 2.1% | 0.9% | The frozen 900, eval-v2 | Northwind safety; run r-4488, 2026-03-25, Yusuf Adeyemi |\n| Teleoperator minutes per thousand robot-hours, live | 47.0 | 31.5 | Seven DCs, six weeks post-cutover | Northwind session logs, extract fa-0712 (2026-07-22), Marcus Ebele's team |\n\nMinutes per thousand robot-hours is what Marcus is paid on, hence the live row. It is confounded — a firmware release, seasonally low volumes — so DC-2 and DC-9, held on the heuristic at 45.8, are the control.\n\n**What surprised us.** On 2026-02-19 the inherited rubric failed calibration at 0.61 alpha against a defended cut of 0.75: the eleven adjudicators, trained a year earlier on root-cause labelling, were labelling cause rather than correct next action. One page of counter-examples and a second round fixed it in 6 person-days. The apparatus transferred; the reflex did not.\n\n**What we would do differently.** Re-validate an inherited rubric against a new label space in week one, before environment work. Budget the eval at full price: $141k against workflow 1's $128k, the only line that rose. Delivery cost fell to $870k from $1.42m, because the adjudicator pool, the sim, the architecture and the price book were not built twice.\n\n**What the customer owns now.** The extended sim, the verifier and its rubric, both eval sets, the weights, and a weekly 120-event adjudication loop gating their own deploys.\n\n**What came next commercially.** 32 B200s reserved for 12 months, signed 2026-06-24 at $1,047,859 on Fleet Reliability capital, separate from the reserved Blackwell block Pick Operations bought for the hard-pick strategy loop, which shares this account and nothing else. A third workflow, dock-approach fault prediction, was scoped in July 2026 and is blocked on Ingrid Halloway's dock-manoeuvre autonomy threshold sign-off, requested 2026-06-11 and unanswered, with no sponsor named.\n\n**Approvals.** Rhea Sundaram, VP Autonomy, approved v3 on 2026-08-11; legal cleared named use on 2026-08-13 under the MSA publicity terms.\n",
"kind": "case_study"
}
],
"scores": [
{
"daysAgo": 254,
"note": "DEMO — First scorecard on workflow 2. Strong on rights, environment and sponsorship because workflow 1 already paid for them; weak on baseline, since the heuristic planner has never been measured on a frozen set, and the forcing function is only peak season. Verifier scored 3 on the untested assumption that the workflow 1 rubric transfers, and task definability 3 because two of the nine manoeuvres are used interchangeably today.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 4
},
{
"id": "baseline_measured",
"weight": 8,
"score": 2
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 3
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 4
},
{
"id": "security_legal_path",
"weight": 8,
"score": 4
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 20,
"note": "DEMO — Closing score after go-live and the reserved commitment. Baseline, budget and headroom resolved: the heuristic was re-run on the frozen 900 at 71.4% against the policy's 89.2%, and Dana Iyer named the FY26 line. Task definability rises to 4 because the written wedge-state rule separating reposition-and-retry from shelf-clear-and-retry is now the adjudicated label space, not a habit. The sim gap cuts twice: environment and verifier both sit at 3, because shelf-collapse contact physics cannot be reproduced, about 9% of production events are unverified, and the escalation half of the task is carried by a rubric judge. Forcing function stays at 2 — peak season never became contractual. Exec sponsor drops to 3 and expansion surface stays at 3: workflow 3 is scoped but blocked on an unanswered safety sign-off with no sponsor named.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 4
},
{
"id": "baseline_measured",
"weight": 8,
"score": 4
},
{
"id": "headroom",
"weight": 10,
"score": 4
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 4
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 4
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
}
],
"promote": {
"artefactTitle": "DEMO — Case study: Northwind Robotics recovery-action policy",
"intoSlug": "case-study-frame"
}
}
@@ -0,0 +1,117 @@
{
"key": "quillon-eval-harness",
"dealName": "DEMO — Evaluation harness pilot",
"accountName": "DEMO — Quillon AI",
"stage": "qualification",
"status": "open",
"acv": 135475,
"summary": "DEMO — Quillon AI's contract-review product ships on vibes: 31,000 documents a month, rich acceptance telemetry, and a 42-case spreadsheet that passed 42 of 42 straight through a regression that cost them a top-ten account's confidence. Scored 54.5%, scope down, so we are selling the measurement and nothing else; the blocker is that there is no budget code and the metric owner has cancelled twice.",
"openedDaysAgo": 34,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — First technical discovery: Quill Review clause suggestions",
"status": "final",
"authoredDaysAgo": 22,
"body": "*Discovery call, 90 minutes, on their side: Tom Feld (CTO, co-founder), Asha Venkataraman (VP Engineering), Radek Wilczek (staff engineer, owns the eval spreadsheet). Rosalind Danquah (Head of Product) accepted and did not attend.*\n\n## The task, in one sentence\n\nGiven an inbound supplier contract and the buying organisation's clause playbook, Quill Review produces a clause-level redline plus a severity-ranked risk summary, which a procurement reviewer accepts, edits or rejects clause by clause inside the Quill web app.\n\n## Volume and shape\n\nAbout 31,000 documents a month across 60 enterprise accounts, from a query Radek ran on the call rather than from memory. Nine flagged clauses per document on average, so roughly 280,000 clause suggestions a month. The shape repeats: the same twelve clause families (limitation of liability, indemnity, data processing, assignment, termination for convenience, audit rights and so on) account for 78% of all suggestions. This is a high-volume, repeating, same-shape task, which is the first thing I was listening for.\n\n## What is actually going wrong\n\nThey cannot tell whether a change makes Quill better. On 9 March they replaced the retrieval layer that grounds suggestions in the customer's playbook. Acceptance on data-processing clauses fell from 58% to 39% and nobody noticed for five weeks, until a top-ten account escalated after their own counsel found three inverted DPA suggestions in one week. Tom's phrase, verbatim: \"we shipped it because the demo felt better.\"\n\nThey have evals. Forty-two hand-written cases in a spreadsheet, judged pass or fail by Tom, last edited in February, currently passing 42 of 42. It did not move on 9 March and it has never moved. It is decorative, and Radek said so in the room before I had to.\n\n## Traces, and the rights to use them\n\nStrong. Every suggestion has been logged since March 2024 with the reviewer's disposition (accepted as written, edited, rejected) and, where edited, the reviewer's final text. That is a per-clause outcome signal at roughly 280,000 rows a month, already joined to the document and the playbook version.\n\nTwo problems with it. Four tenants have bulk auto-accept switched on and represent 22% of volume, so their dispositions are meaningless. And reviewers under deadline accept things they would not defend, which Asha believes but cannot quantify. So the telemetry is a proxy, not an adjudicated label.\n\nRights: their MSA permits use of customer documents for service improvement in 44 of 60 accounts; 16 have opted out. Nobody has asked counsel about training, but nobody needs to yet, because nothing here is a training engagement. The 16 opt-outs do bind the measurement work, and I have carried them forward as a hard constraint on the sampling frame.\n\n## Verification\n\nThere is a computable proxy today and a credible path to adjudicated labels: two of their solutions engineers are ex-procurement and can label clause suggestions against the playbook. No human-labelled held-out set exists. Zero labels, today.\n\n## What I wrote down\n\nRepeatable: yes. Traced: yes, richly. Checkable: proxy only, unmeasured against human judgement. That combination is a scoping conversation with a verifier-measurement phase in front of it, and the verifier-measurement phase is the entire deal. No training in scope. Nothing should be trained until somebody can say what better means and prove the measurement agrees with a human.",
"kind": "discovery"
},
{
"templateSlug": "trainability-qualification",
"stage": "qualification",
"title": "DEMO — Trainability scorecard read-out: measure before you train",
"status": "final",
"authoredDaysAgo": 20,
"body": "*Scored from the discovery notes of two days ago, with Asha Venkataraman on the phone for the commercial half. This is the trainability scorecard, run here to establish the opposite of what it usually establishes: that Quillon are not yet trainable. Weighted result 218 of 400, or 54.5%. Band: scope down.*\n\n## Where the points went\n\nTechnical subtotal 132 of 228. Commercial subtotal 86 of 172. That split is the whole story: a real, well-traced problem inside a company that has not decided to pay for it out of any particular pocket.\n\n**Budget line and its source, loss of 27.** This is the bad one and I am naming it as bad rather than softening it. There is no budget code. Tom gestured at a roughly $400k infrastructure envelope he controls and said \"this would come out of that\", which is an intention, not a line. Nothing is displaced, and no approval path exists because nothing of this shape has been bought here before. Scored 1. If this is still a 1 at proposal, we do not have a deal, we have a workshop.\n\n**Verifier availability and quality, loss of 26.** Scored 2. The acceptance telemetry is a genuine computable proxy, which keeps this clear of the no-verifier disqualifier, but it has never been measured against human judgement, 22% of volume comes from auto-accept tenants and is worthless, and there are zero adjudicated labels in existence. The unblocking artefact is the labelled set sized in the spreadsheet note: 240 adjudicated items, 20 per clause family, with rater-pair agreement reported as kappa alongside judge-versus-human agreement. Those numbers come from Quillon's own family count and their raters' committed hours, not from our template.\n\n**Measured baseline, loss of 24.** Scored 1. Nothing is measured. The 42-case spreadsheet passes 42 of 42 and did not move through a regression that cost them a top-ten account's confidence, so it is not a baseline, it is a decoration.\n\n**Headroom over prompting and workflow design, loss of 20.** Scored 2, and worth being precise about, because it decides whether this engagement leads anywhere. Their prompt authors are actively rewriting the retrieval and grounding layer, and the 42-case sheet stayed green straight through it. Prompting headroom here is untested, not exhausted: nobody can tell whether better prompting closes the gap until the harness exists. A 2 is what \"we cannot distinguish\" is worth.\n\n## What is not wrong\n\nTask definability 3, traces 3, environment constructibility 3: they can replay any historical document through any pipeline version deterministically, having built that harness for support debugging. Expansion surface 3, with obligations extraction and renewal triage named as adjacent workflows with named owners.\n\n## The commercial items\n\nMetric owner scored 2. Rosalind Danquah owns acceptance rate and has cancelled twice, on 9 and 16 July, without rescheduling. Asha will not state a threshold on Rosalind's behalf and is right not to. This grazes the \"nobody states the number\" disqualifier. It does not fire yet, because Asha has committed to getting Rosalind's threshold in writing before we propose; if that sentence does not exist by 31 August, it fires and this score becomes irrelevant.\n\nExecutive sponsor scored 3. Tom is a co-founder, was in every call, and asked for the work himself; he has just never spent money on anything shaped like it.\n\nSecurity and legal path scored 2. The MSA permits service-improvement use in 44 of 60 accounts and 16 have opted out, which fixes the sampling frame at 40 accounts before we start. Counsel has never been asked anything, and nobody at Quillon owns that question.\n\nForcing function scored 2. They intend to migrate Quill's core model by 15 November and cannot responsibly do it blind. Real and dated, but self-imposed, and nobody has costed a slip.\n\n## The action the band prescribes\n\nScope down: do not sell the workflow. The sub-task that stands on its own is the measurement, meaning the held-out set, the verifier and the agreement number. Training stays out of the conversation until the verifier exists with an agreement figure attached.",
"kind": "qualification"
},
{
"templateSlug": null,
"stage": "qualification",
"title": "DEMO — Why the 42-case spreadsheet always says 100%",
"status": "final",
"authoredDaysAgo": 15,
"body": "*Written after Radek Wilczek sent me the sheet. Shared back to him and to Asha; Radek's reply was \"yes, this is the argument I have been losing internally for a year.\"*\n\n## The artefact\n\n`quill_eval_cases_v3.xlsx`. Forty-two rows. Columns: case name, input document, expected behaviour in prose, pass or fail, notes. Last modified 14 February. Current result 42 of 42 pass. It is run by hand before major releases, by whoever remembers.\n\n## One correction to my own discovery note\n\nI wrote down nine flagged clauses per document from the call. Radek's query says 9.4 once suppressed low-severity flags are counted, which the log does and his verbal answer did not. That is about 291,000 suggestions a month, and it matters less for the headline than for the per-family floor below: the suppressed flags sit in the two rarest families, exactly where the sample is thinnest.\n\n## Why it always passes\n\n**It was written from fixed bugs.** Every case reconstructs something that went wrong and was fixed. A suite built backwards from resolved defects cannot fail unless a fix regresses, and cannot see a quality change in the behaviour nobody wrote a case for.\n\n**Pass is a human verdict rendered by the person who wants to ship.** The expected-behaviour column says things like \"should flag the uncapped indemnity and explain why\". Tom reads the output and decides. That is not a verifier. It is a mood, recorded in a cell.\n\n**Forty-two cases against twelve clause families is under four per family.** The March regression moved accepted-as-written from 58% to 39% across roughly 14,000 data-processing suggestions a month, a family below the twelve-family average because only the 38% of contracts touching personal data carry a DPA clause at all. The sheet holds three data-processing cases. All three passed on 10 March, the day after the change, because all three sit at the easy end of the distribution.\n\n**There is no held-out set and no split.** Every case has been visible to every prompt author for over a year, and prompts have been tuned until the sheet is green, which is the definition of fitting to the eval.\n\n**It has no notion of severity or of disagreement.** A wrongly inverted DPA suggestion and a redundant assignment-clause comment are the same single row of \"fail\".\n\n## What replaces it, and how big it has to be\n\nSizing first, because everything downstream cites it. Twelve families at 20 adjudicated items each is 240. Twenty is not a round number chosen for comfort: below about fifteen the per-family agreement interval is wider than the effect we are looking for, and above twenty their raters run out of hours. Both raters have committed one day a week for three weeks, 24 hours each, and at the twelve items an hour they managed on a warm-up batch of thirty, double-labelled, that is roughly 290 items of capacity. So 240 target, 200 hard floor. The two rarest families sit near 2% of volume, so 20 apiece means oversampling them about 8:1 against their natural rate.\n\nThen, in order:\n\n1. A held-out set drawn from the live log after the last prompt edit on 2 July, stratified by clause family and tenant, frozen on day one and withheld from prompt authors thereafter. Their prompts are tuned continuously, so no historical window is untainted; freezing forward is the only honest split available here.\n2. Adjudicated labels from the two ex-procurement solutions engineers, with a written adjudication rule and both independent verdicts kept rather than merged away. Rater-pair agreement reported as Cohen's kappa: below 0.6 the labelling rule is the defect, not the judge, and we stop and rewrite it.\n3. A judge measured against those labels, reported as kappa with an interval, per family. Acceptance telemetry stays as a cheap continuous proxy, but only once we know what it correlates with, and with the four auto-accept tenants excluded.\n\nUntil step three exists, the 15 November model migration is a coin toss they will not be able to score.",
"kind": "narrative"
},
{
"templateSlug": null,
"stage": "qualification",
"title": "DEMO — Scope-down recommendation: a six-week eval harness, no training",
"status": "draft",
"authoredDaysAgo": 6,
"body": "*Draft. Not sent. Written to have something concrete on the table before Tom's next call, and deliberately narrower than what he asked for.*\n\n## What Tom asked for, and what he is getting instead\n\nTom opened the last call asking whether we could \"train a model on our accept data\". Not yet, and the scorecard says so at 54.5% with a measured-baseline score of 1: you cannot train against a signal nobody has validated. What we propose instead is the measurement, delivered as a working harness their engineers own and run.\n\n## Scope\n\nSix weeks. One forward deployed lead, plus their two ex-procurement solutions engineers as raters for one day a week each.\n\n**Weeks 1 to 2, the held-out pool.** Sample 1,200 clause suggestions from the live log since the last prompt edit on 2 July, stratified across the twelve clause families that make up 78% of volume and across tenants, excluding the four auto-accept tenants and the 16 accounts that opted out of service-improvement use, leaving 40 of 60 accounts in the frame. Frozen on day one and withheld from prompt authors thereafter; their prompts are tuned continuously, so a forward freeze is the only real split available.\n\n**Weeks 2 to 4, adjudicated labels.** 240 items drawn from the frozen 1,200, 20 per clause family, the two rarest oversampled about 8:1 against their natural rate; the rest of the pool is held for a second round once the judge is calibrated, so we never re-label the set the judge was fitted against. Both raters label independently against the customer playbook, with a written rule for what correct means at clause level and a recorded adjudication for every disagreement. Rater-pair agreement is reported as Cohen's kappa; below 0.6 we stop and rewrite the labelling rule. Hard floor 200 items; the sizing derivation is in the spreadsheet note.\n\n**Weeks 4 to 5, the judge.** A model-based verifier scored against the adjudicated labels, reported as kappa with an interval, per clause family. If it does not clear the bar on a family, we say so and that family stays human-scored rather than being quietly averaged in.\n\n**Weeks 5 to 6, the harness.** Runs in their CI against any pipeline version and reports per-family deltas. Its acceptance test replays the March regression from the historical log, a separate slice from the held-out set used only for that replay. Plus a written note on what the acceptance telemetry actually correlates with, the cheap signal they live on afterwards.\n\n## Where Tom pushed back\n\nHe wants four weeks, not six, and before the November migration work starts rather than alongside it. He is also unconvinced two raters at a day a week is worth the disruption, and suggested Radek label the set alone over a weekend. I have no good answer on four weeks beyond \"then the labelling is what gets cut\", and single-rater labelling removes the only agreement number that makes the rest defensible. Settle on the next call; I expect to lose a week somewhere.\n\n## Acceptance\n\nThe engagement is accepted when the harness, run by Radek on the 9 March change with no help from us, shows the data-processing regression as a significant negative delta. If it cannot see a regression we already know happened, we have not delivered.\n\n## The two things blocking this becoming a proposal\n\n**No budget line.** Not a soft no, an absent yes. Tom must produce a code and name what this displaces. The indicative figure carried on the demand record is $135,475.\n\n**Rosalind Danquah has not stated a threshold.** She owns acceptance rate and has cancelled twice. I want one sentence from her in writing: what accept-as-written rate on the top twelve families would make her comfortable shipping the November migration. The scorecard gives that until 31 August, eighteen days from today, before the \"nobody states the number\" disqualifier fires.\n\nNot in scope, and I want this sentence in the proposal verbatim: no training, no fine-tuning, no environment, no weights. Those are a second conversation that this engagement earns the right to have.",
"kind": "proposal"
}
],
"scores": [
{
"daysAgo": 20,
"note": "DEMO — First scoring, straight off the discovery call. Technical 132 of 228 and commercial 86 of 172 puts this at 54.5%, scope down: the traces and the task are real, but nothing is measured, no verifier has ever been checked against a human, prompting headroom is untested either way, and there is no budget code at all. Still open: Rosalind Danquah has not stated a threshold and has cancelled twice, on 9 and 16 July.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 3
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 2
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
}
],
"promote": null
}
@@ -0,0 +1,191 @@
{
"key": "tessellate-burst-inference",
"dealName": "DEMO — Inference burst capacity",
"accountName": "DEMO — Tessellate Labs",
"stage": "proposal",
"status": "open",
"summary": "DEMO — Proposal is drafted for Marek Sowa at $178,675 for a committed inference floor plus burst overflow, and goes out once Ilse Fossum has had the forecast call; engineering wants it on the strength of an 11-miss quarter on archive windows. Finance has not conceded: against the incumbent's counter of $60,000 in credits and a 12% per-token cut we are about $8,500 dearer on forecast volume and about $10,400 cheaper if volume stays flat, and the forecast that settles it has not been produced.",
"openedDaysAgo": 112,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — First technical discovery: burst inference for drawing extraction",
"status": "final",
"authoredDaysAgo": 96,
"body": "# First technical discovery: burst inference for drawing extraction\n\nCall with Marek Sowa (co-founder and CTO) and Devi Prasad (staff engineer, serving layer). 55 minutes, no slides.\n\n## What Tessellate actually does\n\nTessellate Labs sells structured quantity takeoffs to architecture and construction firms. A customer uploads a drawing set as PDF; a fine-tuned 12B vision-language model reads each sheet and emits structured line items. 41 paying accounts today.\n\nThe unit of work, because everything below depends on it: a drawing sheet is tiled into roughly twenty crops, so a sheet is not one forward pass. One replica sustains about **120 sheets an hour** at batch 8, call it 30 GPU-seconds a sheet. Devi's figure, from their own dashboards, and the first thing I want to re-measure on our hardware.\n\nTwo load shapes share one endpoint:\n\n- The **interactive path**, a few sheets at a time while someone is at their desk. Median day is about 17,800 sheets across all accounts, which is 148 replica-hours. Their end-to-end p95 target is 45 seconds a sheet and they run at 38.\n- The **archive path**, where a new customer dumps their entire back catalogue. The window they sell is 90 minutes for any archive up to 5,000 sheets. That is 42 replica-hours of work, so it needs about 28 replicas held for the whole hour and a half. Largest observed day was 62,000 sheets, one account migrating eleven years of projects; they wanted it done over an afternoon and it finished at 04:00.\n\nConcurrency across all hours: p50 is 5.8 replica-equivalents, one top-tier accelerator per replica. p90 is 11. p99 is 34. That ratio is the whole engagement.\n\n## What is going wrong\n\nThey serve on per-token serverless endpoints. Two constraints bite. The provider caps them at 40 concurrent replicas, which is only just above the 28 an archive needs, and cold starts on their custom weights run 90 to 210 seconds. When two archives land in the same afternoon the queue backs up behind cold starts rather than behind compute, and the 90-minute window slips.\n\nDevi says they missed it 11 times in Q2. Ferrand and Coe, their second-largest account, have written a service credit into their renewal as a direct result. Marek's words: \"we are not compute-poor, we are compute-shaped-wrong.\"\n\n## Measurement\n\nExtraction quality is measured per sheet against adjudicated labels produced by two internal raters with a third adjudicating disagreements, on a held-out set of 3,400 sheets. That number is stable and is not what this deal is about, and nobody could tell me the inter-rater agreement rate when I asked. The metric that moves here is archive completion within 90 minutes, which they track but nobody currently owns as a target.\n\n## Spend today\n\nTrailing twelve months on the incumbent is $224,000. The last quarter run-rate is $21,800 a month, so $261,600 annualised. Three of the last twelve months were under $9,000. Both facts are true and Marek and their finance lead read them differently, which I expect to be the central argument of this deal.\n\n## Open questions\n\n1. Who owns the 90-minute number as a target rather than a dashboard? Marek said \"probably Devi\" and Devi did not agree in the room.\n2. Ilse Fossum joined as Head of Finance six weeks ago and has not been in a call yet. Nobody has told me what budget line inference sits on.\n3. Their customers' drawings are commercially sensitive. Retention, training rights and who owns the fine-tuned adapter need writing down before anything else moves.\n\nNext: get 90 days of concurrency traces from Devi, and get Ilse into a room.",
"kind": "discovery"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Data and security brief: drawing archives, retention and adapter custody",
"status": "final",
"authoredDaysAgo": 74,
"body": "# Data and security brief: drawing archives, retention, and adapter custody\n\nWritten with Devi Prasad and reviewed by Tessellate's outside counsel (Halbeck Freen). This covers what crosses the boundary, what may be retained, what may not be learned from, and who owns the weights when this ends.\n\n## What the data is\n\nArchitectural and structural drawing sets belonging to Tessellate's customers, not to Tessellate. A single set may contain unbuilt designs, client names, tender pricing on the title block, and in two accounts the layouts of buildings with physical security requirements. Tessellate's own customer agreements grant them a processing licence only. They have no right to sublicense the content for model training, and they said so plainly rather than making us find out at redline.\n\n## Consequences for the deal\n\n- **No training on customer content, by either party.** Written into the order form, not left to policy. It closes off any future post-training expansion that depends on their production traffic, and both sides should stop pretending otherwise.\n- **Retention is 24 hours.** Sheets are rendered to images, inferred, and the images deleted. Only the structured output and a request-level log survive. Our serving layer must not persist request or response bodies beyond completion of the request, and the log we keep is metadata: timestamp, account hash, sheet count, latency, GPU-seconds.\n- **Metadata we may keep and use.** Concurrency, queue depth, token counts, error classes. Enough to run capacity planning and to build a demand forecast; not enough to reconstruct a drawing, which is the test counsel applied.\n- **Region.** Two accounts, both UK practices, require processing in the EEA or UK. Everything else is unrestricted. We commit to a UK region for the whole workload rather than splitting it, because splitting it doubles the operational surface for 4% of volume.\n\n## Weights ownership, exit, and the base licence\n\nThe base model is open-weight and licensed to each party independently. The fine-tuned adapter is **Tessellate's property**, created before us and unchanged by us; we hold it as a processor and gain no licence to it beyond serving their traffic. Three things follow, and only the first is currently drafted:\n\n1. Adapter artefacts are exportable on request at any time, in the format supplied, within five working days.\n2. On non-renewal or termination we delete resident copies within 30 days and confirm in writing. Egress of their stored adapter versions is free; there is no exit fee and I want that written rather than assumed.\n3. If we ever produce a derived artefact — a quantised or compiled build of their adapter for our runtime — it is theirs too, and it should be named as such. This is the clause counsel has not yet seen and it is the one I expect to take a round.\n\n## Position on subprocessors\n\nTessellate publish a subprocessor list to their own customers with 30 days' notice. Adding us triggers that clock. Marek started it on day 27 without being asked, which is the single most useful thing anyone on their side has done for the schedule.\n\n## Where we landed\n\nDPA agreed on day 36 with two edits: deletion confirmation on request rather than on a fixed report, and 72-hour breach notification rather than 24, which counsel accepted. No penetration test was requested. No SOC 2 report was requested, which surprised me; Ilse may yet ask, and if she does it arrives as a procurement delay rather than a technical one.\n\n## Residual risk\n\nThe 24-hour retention promise is ours to keep in the serving layer, and our default request logging is more verbose than that. It needs a named configuration, verified once before go-live and again after any serving upgrade. Owner: our side, not theirs. I have asked Sunil Nandakumar on the platform team for it and it is not done. If we breach this quietly it shows up in no metric either party watches, which is exactly the kind of risk that surfaces at renewal.",
"kind": "discovery"
},
{
"templateSlug": "reference-architectures",
"stage": "scoping",
"title": "DEMO — Serving architecture: committed floor with burst overflow",
"status": "final",
"authoredDaysAgo": 61,
"body": "# Serving architecture: committed floor with burst overflow\n\nReviewed with Devi Prasad on day 49. This is the shape we are proposing, why it is that shape, and what it does not solve.\n\n## The load, restated in capacity terms\n\nFrom 90 days of Devi's traces, at their measured 120 sheets per replica-hour:\n\n| Measure | Value |\n| --- | --- |\n| Median concurrent replicas, all hours | 5.8 |\n| p90 | 11 |\n| p99 | 34 |\n| Peak sustained, 60 minutes | 31 |\n| Hours per week above 12 replicas | 9.4 |\n| Sheets, median day | 17,800 |\n| Sheets, largest day observed | 62,000 |\n| Replica-hours a year, both paths | about 60,000 |\n\nNine and a half hours a week above 12 replicas. That is the entire problem in one row: paying for 34 replicas continuously is absurd, and having 40 available only after a three-minute cold start is what is losing them archive windows.\n\n## Proposed shape\n\n**A committed floor of ten accelerators, held warm, plus on-demand burst above it.**\n\n- The floor is reserved capacity with the base model and their adapter already resident. No cold start, ever, for the first ten replicas. It absorbs the interactive path outright and the first third of any archive.\n- Burst above the floor is on-demand, autoscaled on queue depth rather than on GPU utilisation, because the queue signals an archive before utilisation does. Devi's dashboards already emit queue depth, but at 30-second granularity, which is too coarse to scale on; taking it to five seconds is about a week of their work and is not yet scheduled. This is currently the largest unowned item in the plan.\n- Burst instances start from a pre-baked image with weights on local NVMe. Measured start on our side is 38 to 55 seconds, against the 90 to 210 they see today. That is the number the technical case rests on and it is the one they should test in week one.\n- A 5,000-sheet archive is 42 replica-hours. Inside a 90-minute window that is 28 replicas: ten warm from the floor, eighteen burst, of which the burst arrives inside a minute rather than inside four. One region, UK, for the whole workload. See the security brief.\n\n## What this does not solve\n\nBurst start is 38 to 55 seconds, not zero. An archive that arrives with no warning still has a first minute where the queue grows. If Ferrand and Coe's service credit clause triggers on the first sheet rather than the last, this architecture does not fully protect them and we should not imply that it does. Devi and I both think the fix is a 60-second pre-announce hook in their upload flow, which is their work, not ours, and it is not in scope here.\n\nSecond, the floor is sized on concurrency, not on utilisation. Against roughly 60,000 replica-hours of demand a year, of which about 6,400 land above the floor, the floor supplies 87,480 and runs at about 61%. Roughly 39% of floor hours are unused. That is deliberate — it is what buys the reserved rate and the warm start — and it is the exact fact their finance lead will fasten onto. The pricing artefact has to meet it head on rather than hide it in a blended number.\n\nThird, none of this improves extraction quality. Their held-out accuracy is what it is. The headroom in this engagement is in the serving shape and nowhere else. If someone at Tessellate is quietly hoping this deal also makes the model better, that expectation needs killing now, and the no-training clause in the brief means it cannot be revisited later either.",
"kind": "architecture"
},
{
"templateSlug": "pricing-and-packaging",
"stage": "proposal",
"title": "DEMO — Pricing and packaging: what Tessellate pays today and what changes",
"status": "final",
"authoredDaysAgo": 24,
"body": "# Pricing and packaging: what Tessellate pays today and what changes\n\n## What they pay today\n\nPer-token serverless. Three numbers, all true, all as at day 88:\n\n- Trailing twelve months, re-pulled with Devi this week: **$250,700**. At discovery on day 16 it was $224,000. Two sub-$9,000 months have rolled off and two months above $21,800 have rolled on, so the metric moves about $13,000 a month in our favour and only one of the last twelve is now under $9,000.\n- Last quarter run-rate: **$21,800 a month, $261,600 annualised**.\n- The incumbent's counter, made on day 81: **$60,000 in credits over twelve months and a 12% per-token reduction**. On the same volume that is about $170,200 next year, all of it variable.\n\nIlse Fossum used the trailing twelve until the re-pull; she now uses the counter, which is a better argument and I should say so out loud. Marek and Devi use the run-rate. Nobody is being dishonest.\n\n## Proposed shape\n\nA committed floor with burst above it, twelve-month term.\n\n| Line | Basis | Annual |\n| --- | --- | --- |\n| Committed floor | $12,250 a month, 7,290 GPU-hours a month at the reserved rate of $1.68 per GPU-hour, which is ten accelerators held continuously | $147,000 |\n| Burst above the floor | Forecast 6,400 GPU-hours at $2.95, against a public on-demand rate of $3.40 | $18,880 |\n| Migration and onboarding | Two weeks, image bake, autoscaler integration, one-off | $12,795 |\n| **Total contract value** | | **$178,675** |\n\nThe contracted minimum is **$159,795**. The burst line is a forecast and must be labelled as one in the proposal; if their volume flattens they pay the minimum and nothing else. Blended across compute the forecast is **$1.77 per GPU-hour** over 93,880 hours; $1.90 if you amortise the one-off migration fee over year one, which is a presentation, not a rate.\n\n## What is being traded, not given away\n\nThe reserved rate of $1.68 is 51% off our on-demand rate, and burst at $2.95 is 13% off. Both are bought with five things, each written into the order form:\n\n1. **Twelve-month term, not six.** They asked for six. Six at this rate is a discount with no consideration behind it.\n2. **Unused floor rolls forward one month, capped at one month's value.** This answers the 39% idle problem directly. It costs us the option value of that capacity and is worth more to them than another ten cents on the rate.\n3. **Burst at $2.95 exists only above a live floor and only for the term.** Cancel the floor and burst reverts to $3.40 list on the next invoice. Burst is the capacity that is expensive to hold, so it is the discount least defensible on its own.\n4. **A named public reference and a joint write-up after 90 days of production**, subject to their customers' consent. Joris Talbot, who runs their customer marketing, has agreed in principle.\n5. **Two engineers on their side for the migration fortnight**, named in the order form. Devi plus one. Without this the two weeks becomes six and the fee is fiction.\n\nNot moving: no rate below $1.68, and no month-to-month.\n\n## The argument we have not yet won\n\nAgainst the run-rate we save them $82,925. Against the re-pulled trailing twelve, $72,025. Against the incumbent's counter we are **$8,467 more expensive** on forecast volume — and $10,413 cheaper if volume never grows and they pay the minimum. So the finance case inverts on the volume question, and Ilse is right that nobody has shown her the forecast on paper. That forecast is what I owe them, and it should come from Devi's ingestion numbers rather than from us.",
"kind": "pricing"
},
{
"templateSlug": "proposal-blocks",
"stage": "proposal",
"title": "DEMO — Proposal to Marek Sowa: burst inference capacity",
"status": "draft",
"authoredDaysAgo": 9,
"body": "# Proposal: burst inference capacity for Tessellate Labs\n\nFor Marek Sowa, CTO. Copied to Devi Prasad, Ilse Fossum and Joris Talbot.\n\n## The problem, as you described it\n\nTessellate serves two workloads through one endpoint. The interactive path is steady, about 17,800 sheets on an ordinary day at roughly 120 sheets per replica-hour, and it works. The archive path is not steady: a single account migrating eleven years of drawings produced 62,000 sheets in a day, and your concurrency goes from a median of 5.8 replicas to 34. The 90-minute window you sell on a 5,000-sheet archive needs 28 replicas held for the whole hour and a half. On per-token serverless you are capped at 40 and your custom weights cold-start in 90 to 210 seconds, so when two archives land together the queue grows behind cold starts rather than behind compute. You missed that window 11 times in Q2, and Ferrand and Coe have put a service credit into their renewal because of it. In your words, you are not compute-poor, you are compute-shaped-wrong.\n\n## What we propose\n\nA committed floor of ten accelerators, held warm with your adapter resident, and on-demand burst above it that scales on queue depth rather than GPU utilisation. Nothing in the first ten replicas ever cold-starts. Burst instances start from a pre-baked image with weights on local NVMe; we measure 38 to 55 seconds, against the 90 to 210 you see now. Single UK region, on the terms in the data brief we agreed with Halbeck Freen on day 36, including the adapter custody and exit-egress clauses.\n\n## What this does and does not fix\n\nIt removes cold start as the cause of a missed archive window and it removes the 40-replica ceiling. It does not make burst start instantaneous. An unannounced archive still has a first minute where the queue grows, and if Ferrand and Coe's credit clause triggers on the first sheet rather than the last, this alone will not protect you. Devi and I think the answer is a short pre-announce hook in your upload flow, plus queue-depth metrics at five-second granularity rather than thirty. Both are your work and neither is in this proposal. Nor does any of this change extraction accuracy; your held-out numbers stay where they are.\n\n## What it costs\n\n$12,250 a month committed, 7,290 GPU-hours a month at $1.68 per GPU-hour, which is 51% off our on-demand rate. Burst above the floor at $2.95 against a $3.40 list, for the term and above a live floor. A one-off $12,795 for the migration fortnight. On Devi's forecast volumes that is $178,675 for the year, blended $1.77 per GPU-hour across compute; the contracted minimum, if your volume never grows, is $159,795. Unused floor rolls forward one month, capped at one month's value, which is our answer to the fact that your median load leaves about 39% of the floor unused on an ordinary week.\n\n## The number Ilse should see\n\nAgainst your last quarter's run-rate this saves $82,925 a year. Against your trailing twelve, re-pulled a fortnight ago at $250,700, it saves $72,025. Against your incumbent's counter of $60,000 in credits and a 12% per-token cut, we are about $8,500 more expensive on forecast volume, and about $10,400 cheaper if your volume stays flat and you pay the minimum. I am not going to pretend that first comparison away. The deal turns on your own ingestion forecast for the two accounts that onboarded in March, not on our arithmetic.\n\n## What we need from you\n\nDevi plus one engineer for the migration fortnight, named. A named owner for the 90-minute archive number — a person, not a dashboard. Ilse in a 30-minute call on the forecast. A decision by day 124, because the floor we have held for you releases then and the reserved rate goes with it.",
"kind": "proposal"
}
],
"scores": [
{
"daysAgo": 92,
"note": "DEMO — First pass after discovery: 184 of 400, 46.0, Scope down and only just. The load shape is clear and the pain is real, but there are no traces yet, no named owner for the 90-minute archive window, no inter-rater agreement figure for the 3,400-sheet held-out set, and no idea which budget line inference sits on now that a Head of Finance has started six weeks ago. Headroom is a 1 deliberately: the headroom in this engagement is in the serving shape, not in extraction quality, which is already where it is going to be, and scoring it as though a post-training upside exists would be the easiest way to mis-sell this deal.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 1
},
{
"id": "env_constructibility",
"weight": 8,
"score": 2
},
{
"id": "metric_owner",
"weight": 9,
"score": 2
},
{
"id": "budget_source",
"weight": 9,
"score": 2
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 2
},
{
"id": "security_legal_path",
"weight": 8,
"score": 1
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 2
}
]
},
{
"daysAgo": 20,
"note": "DEMO — Re-scored after the traces, the DPA and the incumbent's counter: 238 of 400, 59.5, still Scope down, which is the honest reading: the offer on the table is the scoped-down shape, a floor of ten and burst, inference only, no post-training. Up: baseline 1 to 4 on 90 days of concurrency traces at a measured 120 sheets per replica-hour; traces and rights 2 to 3 because the metadata we need for capacity planning is explicitly ours to keep; environment 2 to 4 on a pre-baked burst start measured at 38 to 55 seconds on our own hardware; security 1 to 3 on the DPA signed day 36 with adapter custody drafted, though the exit-egress and derived-artefact clauses have not been through counsel; sponsor 2 to 3 because Marek put his own name on the internal case and took it to their board's operating review. Down: budget 2 to 1, since Ilse Fossum has reclassified inference as run-rate cost and the incumbent has come back with $60,000 of credits and a 12% per-token cut, so nobody at Tessellate will yet state a number they will defend; expansion 2 to 1, because the no-training clause removes the post-training path entirely and what is left is more inference volume and a second region, which is a smaller surface than we scored at discovery. Held: verifier at 2 (nobody has produced an agreement rate or an adjudication rate for the held-out set, and this deal has not asked them to), metric owner at 2 (still nobody owns the 90-minute number, which is why the proposal asks for a person by name), forcing function at 2 (the Ferrand and Coe renewal credit was already known at discovery and nothing new has landed), headroom at 1. The no-rights-to-train disqualifier is triggered on the page and does not bite here, because this is an inference engagement; it would kill any post-training follow-on, which is exactly why expansion drops. Still open on our side: the 24-hour retention logging configuration asked of Sunil Nandakumar on day 34 and not done as of today, day 92. One dimension moves this into Qualified conditional and it is budget: at a 3 the total is 256 of 400, 64.0, and everything else can stay exactly where it is.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 3
},
{
"id": "baseline_measured",
"weight": 8,
"score": 4
},
{
"id": "headroom",
"weight": 10,
"score": 1
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 2
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 2
},
{
"id": "expansion_surface",
"weight": 5,
"score": 1
}
]
}
],
"promote": null
}
@@ -0,0 +1,341 @@
{
"key": "verity-eu-fine-tuning",
"dealName": "DEMO — EU-resident fine-tuning",
"accountName": "DEMO — Verity Health AI",
"stage": "procurement",
"status": "open",
"summary": "DEMO — Second demand record at Verity, opened out of the same May discovery that produced the inference endpoint: a clinical-coding agent trained against payer-settled DRGs, $1,252,154, technically resolved and commercially stuck. The bundled requisition was rejected on category grounds; the fix is two requisitions, and half of it is raised while the infrastructure half waits on a silent budget holder and an unsettled compute ceiling.",
"openedDaysAgo": 90,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — Second discovery: DRG coding inside the Scribe business",
"status": "final",
"authoredDaysAgo": 86,
"body": "## Who was in the room\n\nNinety minutes on 25 May, Verity's Amsterdam office, nineteen days after the Scribe discovery in Lisbon. Their side: Dr Margriet Ruijs (VP Clinical Coding Operations, owns the coding-accuracy number reported to the board), Bram Kooij (staff engineer, built and maintains the coding pipeline), Sabine Brandt (lead clinical coder, twelve years in DRG coding, the only person present who has done the work). Priya Raghunathan sat in for the first twenty minutes and left. Her platform programme and Margriet's coding programme run alongside each other and share a tenancy, a CTO and nothing else. Our side: Sofia Lindgren and Theo Aguirre. No lawyers, deliberately.\n\nMarisol Okonjo asked for this call on 18 May. We are not an incumbent here: the EU inference endpoint is still in redline, nothing of ours is in production at Verity, and I said so in the room rather than let anyone assume a relationship we have not earned yet.\n\n## The workflow\n\nVerity Code is the older half of the business and the one Scribe was built out of. It codes inpatient discharge summaries for nine hospital groups: five German, three Dutch, one Irish. Rheinklinik Verbund — the fourteen-hospital group whose 12 January go-live is driving the endpoint deal — is the largest of them and has been a coding customer since 2023.\n\nVolume is 61,000 cases a month, growing roughly 4% a quarter; it was 58,600 a year ago. Each case produces a primary diagnosis, ordered secondaries, procedures and a resulting DRG, which determines what the hospital is paid. About 12.4% of coded cases are challenged by the payer's medical service, roughly 7,600 a month. A challenge requires a written justification citing the documentation. Sabine's team of nineteen coders spends most of its week on those justifications rather than on coding.\n\n## Repeatable, traced, checkable\n\nRepeatable: yes, and unusually so. Same input shape, same output schema, same rulebook, revised annually.\n\nTraced: yes. Every case since 2023 carries the pipeline's proposed codes, the coder's accepted or corrected codes, the challenge letter where one exists, the justification the coder wrote, and the payer's settled DRG. That last field is the interesting one and Bram had never thought of it as a label.\n\nCheckable: this is the finding of the call. The settled DRG after appeal is an adjudicated label produced by a process that has nothing to do with us and cannot be gamed by the model. Agreement is computable exactly at DRG level and hierarchically at ICD-10-GM code level. The justification text needs a judge; the code assignment does not.\n\n## The failed attempt\n\nBram has run a prompted pipeline on a frontier API for seven months. Primary-DRG agreement moved from 62.3% to 68.1% and has been flat for eleven weeks despite retrieval over the coding rulebook and a two-pass self-critique. Sabine's coders sit at 88.9% against the same settled labels. Bram volunteered, unprompted, that he has run out of ideas. That is the headroom argument made by the customer rather than by us.\n\n## Constraints found\n\nResidency is inherited, not negotiable: the Rheinklinik DPA that forces the Scribe endpoint into the EU covers coding data too, and Verity's contracts with all nine groups permit processing for coding and are silent on training. Margriet believes an amendment is needed and does not know how many of the nine will sign. Largest open question, and it is legal rather than technical.\n\nThey will not accept a hosted model whose weights they do not hold. Consistent with what Marisol said in Lisbon, so this is settled ground rather than an argument.\n\n## What we do not have\n\nNo baseline on a held-out set — 68.1% is computed over rolling production traffic with no frozen split. No named budget: Margriet says the money \"would come from the platform side\", which is a direction, not a source. Katrin Vogel, the fractional DPO who gates the endpoint deal, was not in the room and will gate this one too.",
"kind": "discovery"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Data and security brief: training rights across nine hospital groups",
"status": "final",
"authoredDaysAgo": 62,
"body": "## Scope of this brief\n\nPost-training on clinical documentation from nine hospital groups across three jurisdictions. Verity is processor to each group; Prime Intellect becomes subprocessor to Verity. Filled in with Bram Kooij and Owen McCabe on 18 June and handed to Katrin Vogel, Verity's fractional DPO, the same afternoon.\n\nThis is an amendment to the DPA Ana Beltrán is negotiating for the inference endpoint, not a fresh negotiation — which sounds like six weeks saved and is not, because that DPA is not signed. Ana is drafting one document to serve two deals. Efficient while it moves; total blockage when it stops. Katrin's endpoint queue comes first and she works two days a week.\n\n## The gap between policy and practice\n\nThirty minutes with Bram found what the policy document does not say. Discharge summaries have never left Verity's Frankfurt tenancy, not because a policy forbids it but because nobody has ever asked. The prohibition Margriet described in Amsterdam turned out to be a habit with a plausible justification attached, and that distinction is the reason this deal is doable at all.\n\nWhat is a genuine constraint: the hospital-group contracts permit Verity to process documentation for the purpose of coding and say nothing about training. Katrin's position, which we accept without argument, is that silence is not permission in this jurisdiction and an amendment is required from each group before its cases enter a training corpus.\n\nStatus: amendments drafted and issued to all nine on 22 May. Seven executed — Rheinklinik Verbund, Sankt-Vitus Klinikgruppe, Moselhöhe Kliniken, Zuiderpoort Ziekenhuizen, Stichting Maasdelta Zorg, Hoogveld Medisch Centrum, St Brendan's Group. Two outstanding, both German university hospitals with their own ethics committees: Universitätsklinikum Bergwald has a committee date in October, Universitätsklinikum Nordharz has not answered in five weeks. Between them they are about 21% of monthly volume and most of the cardiology case mix.\n\nDecision taken: the POC corpus is built from the seven. This costs case-mix breadth and it is stated in the proposal rather than discovered in week four.\n\n## Permitted use\n\nStated by us before Katrin raised it: customer data trains the customer's model and nothing else. No cross-customer training, no shared base, no de-identified reuse. Adapter weights are Verity's property on creation, not on payment. Katrin accepted the clause and asked for the deletion certificate to name the storage system, which we agreed.\n\n## Residency, retention, deletion\n\nTraining and inference run in the Frankfurt region. No failover — the endpoint DPA forbids cross-region replication and a US read replica, which means a Frankfurt outage stops training and stops coding. That constrains our recovery story and we say so in the questionnaire rather than let a security reviewer find it.\n\nTraces used for training are retained for the campaign plus 90 days, then deleted with certificate. Evaluation sets persist for the life of the agreement, because a held-out set that is deleted cannot be re-run. Written as a named exception rather than left implicit.\n\n## Roles, pseudonymisation, subprocessors\n\nDirect identifiers are stripped in Verity's tenancy before anything reaches a training pipeline — the same hash-based method Katrin has already flagged as possibly insufficient on the endpoint deal. If she rules against it there, it moves here too. Free-text re-identification risk is real and neither side pretends otherwise; the mitigation is that the environment never egresses text. Subprocessor list issued unprompted: three named, all EU-resident, fifteen working days to object.\n\n## Open\n\nThe DPIA amendment cannot close while the two groups are outstanding, because the DPIA enumerates the controllers and Katrin will not sign one with a footnote. This does not block a POC scoped to the seven. It does block any expansion to full volume, and it is why the security and legal path will not score full marks on this deal at any point.",
"kind": "discovery"
},
{
"templateSlug": "reference-architectures",
"stage": "scoping",
"title": "DEMO — Coding environment, corpus derivation and the three-layer verifier stack",
"status": "final",
"authoredDaysAgo": 48,
"body": "## What we are building\n\nOne environment, one verifier stack, one held-out set. Nothing here needs tool use, browser control or a live system, which is why environment construction is cheap on this deal and the risk sits in the judge.\n\n## The environment\n\nSingle-turn with an optional second turn. Input: a de-identified discharge summary from a signed group, the structured admission record, and the ICD-10-GM and OPS catalogue version in force for the case year. Output: JSON with primary diagnosis, ordered secondaries, procedures and the derived DRG, plus a justification paragraph citing document spans where a challenge letter exists in the trace.\n\nCases are keyed to the catalogue in force at coding time, not the current one. Getting this wrong silently poisons about a fifth of the corpus. It is the failure Bram's own pipeline made in 2024 and took five weeks to find.\n\n## The corpus, and why it is 214,000 and not 1.25M\n\nSeven signed groups are about 49,300 cases a month; over 26 months that is roughly 1.25M. We are training on 214,000 of them, and the derivation matters because Piotr Sawicki will ask.\n\nAll 148,000 settled challenged cases — challenged by a payer, resolved, settled DRG recorded — because the challenge is where the signal is. Plus a 66,000-case stratified draw from the settled unchallenged remainder, stratified by group, case year and DRG family. Unchallenged cases are 88% of volume; taking all of them would bury the 148,000 that teach anything. Cases still inside the payer's six-month challenge window are excluded because they are not settled and therefore not labelled.\n\nDev split 12,000, held-out 4,000, both stratified the same way. The held-out set is frozen and sealed before the first training run and read at most twice.\n\n## The verifier stack\n\nThree layers, in descending order of trust.\n\nLayer one, deterministic and free: the DRG grouper. Verity licenses the same certified grouper the payers use; given a code set it returns a DRG with no model in the loop. Exact DRG match against the settled label is the primary reward, and it is not gameable because the label is produced outside this system.\n\nLayer two, deterministic: hierarchical code agreement, partial credit for a correct three-character category with a wrong fourth, because a pure DRG signal is too sparse early in training.\n\nLayer three, judged: the justification paragraph. Span citation is checkable deterministically; clinical adequacy is not.\n\nThe failure mode we designed against is the model learning to propose whatever DRG pays most, which correlates with the settled label often enough to look like learning. Mitigation: an upcoding rate reported alongside accuracy on every run, defined as cases whose proposed DRG exceeds the settled DRG in reimbursement weight. If accuracy rises and upcoding rises with it, the run failed regardless of the headline.\n\n## Two numbers Bram measured for us\n\nOn the 12,000-case dev split, 30 June, so that neither appears first in a proposal as if we had invented it. Current pipeline upcoding rate 3.1%, 372 of 12,000. First-pass acceptance of the pipeline's drafted justification by a senior coder, 34%, on a 300-letter sample adjudicated by Sabine Brandt's team.\n\n## Base model and serving\n\nOpen-weights 70B, the same base `verity-scribe-v4` was SFT'd from, so the trained adapter serves on the endpoint Priya is contracting for rather than needing a second migration. Adapter weights are Verity's, held in their Frankfurt tenancy.\n\n## What Applied Research flagged\n\nThe judge is the weak joint. Justification quality is what Sabine cares about most and the only objective without a deterministic check. Pre-committed rule, written here so it cannot be argued about in week six: if the judge's Cohen's kappa against an adjudicated set falls below 0.75 — the human-to-human ceiling on this material is 0.84, measured across Sabine's own seniors — we drop the justification objective from the POC threshold and report it as a secondary observation.\n\nCalibration cannot use customer traces before a PO exists. The set is being assembled instead from published German DRG appeal case law and non-customer teaching cases, adjudicated by Sabine's team on their own time. Any result off it is provisional.",
"kind": "architecture"
},
{
"templateSlug": "proposal-blocks",
"stage": "proposal",
"title": "DEMO — POC proposal: primary-DRG agreement from 67.4% to 80.0%",
"status": "final",
"authoredDaysAgo": 34,
"body": "## Problem restated\n\nVerity Code handles 62,400 inpatient cases a month across nine hospital groups, up from 61,000 in May; payers challenge about 7,700 of them. Your prompted pipeline reaches 67.4% primary-DRG agreement on the sealed 4,000-case split. Your coders reach 88.9% on the same split.\n\nThe 21.5-point gap is absorbed by 11.8 full-time equivalents out of a nineteen-person coding team writing justification letters instead of coding. It also caps onboarding: at 4% growth a quarter you are adding roughly 2,400 cases a month to a team you have already decided not to grow proportionally, which is why two groups came out of the FY26 plan.\n\nSeven months of prompt engineering moved the number 5.8 points and it has now been flat for five months. That is not a criticism of the work. It is evidence about where the remaining gain lives.\n\n## What we will build\n\nFour artefacts, all yours: the coding environment, the three-layer verifier stack, the frozen 4,000-case held-out set, and adapter weights over an open 70B base — the base your Scribe model is already built on. The weights are your property on creation. You hold them in your Frankfurt tenancy and can serve them without us.\n\n## Success and measurement\n\nThe POC succeeds if, on the sealed held-out set, primary-DRG agreement reaches 80.0% or above with the upcoding rate no higher than the pipeline's current 3.1%. Both numbers were measured by Bram Kooij on your data, not by us.\n\nSecondary, reported but not gating: first-pass acceptance of the drafted justification by a senior coder, 34% today, target 60%. It is secondary because its verifier is a judge. If the judge's Cohen's kappa against an adjudicated set comes in below 0.75, against a human-to-human ceiling of 0.84 on the same material, we drop it and say so rather than defend a number we do not trust.\n\n## Measuring it honestly\n\nThe held-out set is frozen before the first training run and read at most twice. Every eval reports accuracy and upcoding together, because a model that has learned to bias upward shows a rising headline and a rising upcoding rate at the same time.\n\n## Data handling\n\nTraining corpus drawn from the seven groups whose amendments are executed: 214,000 settled cases over 26 months. Universitätsklinikum Bergwald and Universitätsklinikum Nordharz are excluded, which removes about 21% of volume and most of your cardiology mix. We would rather state that here than discover it in week four. Everything stays in the Frankfurt region with no failover, per the endpoint DPA.\n\n## What we need from you\n\nBram Kooij for four days across weeks one and two on trace extraction. Sabine Brandt or a delegate for six half-days of adjudication. A named contact for the grouper licence. This is the block most often ignored and the commonest cause of a POC finishing late.\n\n## Commercial structure\n\nTotal $1,252,154 in three deliberately separable lines.\n\nEngagement fee, $520,000. Applied Research, environment construction, verifier and eval build, run triage.\n\nTraining compute, $432,154. A committed floor of 8,000 H100-hours, a blended $54.02 an hour, with on-demand burst above it at $57.75 rather than a block sized to the ceiling.\n\nProduction inference for the coding workload, $300,000 a year: 49,300 cases a month across the seven signed groups, at the per-case rate on the endpoint order form now in redline. The two outstanding groups add roughly $80,000 when they sign. This line attaches to that order form and cannot be ordered before it is signed.\n\n## Timeline and decision\n\nTwelve weeks from PO. New requisitions freeze on 12 September, your financial year closes on 30 September, and the FY27 planning round runs to mid-October — so a compute line not committed against existing spend this month is a January conversation. A PO on 5 September reports on 27 November. Margriet's board paper is due 1 December. Four days is not margin, and we would rather write that than shorten the plan to fit it.",
"kind": "proposal"
},
{
"templateSlug": "pricing-and-packaging",
"stage": "proposal",
"title": "DEMO — Pricing and packaging: three lines, two funding sources, one ceiling",
"status": "final",
"authoredDaysAgo": 30,
"body": "## What this prices\n\nTotal $1,252,154 across three lines that must be separable, because they are not going to be funded from the same place. That is a procurement decision disguised as a pricing decision, and getting it wrong here is what a rejected requisition looks like six weeks later.\n\n| Line | Amount | Category | Recurs |\n| --- | --- | --- | --- |\n| Engagement fee | $520,000 | Professional services | No |\n| Training compute | $432,154 | Infrastructure | Per campaign |\n| Production inference, coding workload | $300,000 | Infrastructure | Yes, annually |\n\n## The compute forecast\n\nBuilt so Piotr Sawicki can rebuild it. He is a finance business partner, not an ML engineer, and a unit he cannot forecast is a unit he escalates.\n\n| Component | Low | Expected | High |\n| --- | --- | --- | --- |\n| Training campaigns (4 x 1,800 h) | 5,400 | 7,200 | 9,000 |\n| Restart factor | x1.20 | x1.35 | x1.55 |\n| Campaigns after restarts | 6,480 | 9,720 | 13,950 |\n| Eval sweeps | 1,900 | 2,400 | 3,400 |\n| Environment iteration | 1,420 | 1,600 | 2,050 |\n| Total H100-hours | 9,800 | 13,720 | 19,400 |\n\nThe restart factor applies to training campaigns only. Eval sweeps and environment iteration do not restart; they are already ranged.\n\nCommercial shape: a committed floor of 8,000 hours at $432,154, a blended $54.02 an hour, with on-demand burst above it at $57.75. The not-to-exceed ceiling on the order form is the high column, 19,400 hours — 11,400 on-demand hours above the floor, $658,350. So the ceiling exposure is $1,090,504 of compute, not $432,154, and any requisition raised at the floor is raised at the wrong number.\n\nVerity has never run a training campaign and cannot defend a commitment, which is why the floor is 8,000 and not 13,720. Our own margin is modelled against the full 8,000 whether consumed or not, because cost is charged on the commitment.\n\n## The anchoring question\n\nAsked of Margriet Ruijs on 25 May: what does the gap cost you today? She had it already, built for a March board paper, and she walked through the derivation rather than quoting the total.\n\nNineteen coders, timesheeted, so this part is measured: 11.8 full-time equivalents spent on challenge justifications rather than coding in the twelve months to April. At €96,000 loaded, €1.13m.\n\nThen the part she computed herself. Universitätsklinikum Bergwald and one group she would not name were deferred out of the FY26 onboarding plan because she could not staff the coding. Contribution roughly €340,000 each, so €680,000. Total €1.81m.\n\nHer board believes the €1.13m — it is in the timesheet system and Greta Halvorsen's team can pull it. It does not believe the €680,000, because deferred contribution is a number a VP can construct and Greta has said so in writing. The defensible anchor is therefore €1.13m a year. $1,252,154 sits above one year of that and below two, which is a survivable place to be but not a comfortable one.\n\n## The marketplace route and its consequence\n\nJan Kowalczyk holds a €4.1m commitment with their hyperscaler, nineteen months remaining. A private offer against it is a drawdown rather than a purchase, which changes three things: the contracting counterparty becomes the marketplace operator, the offer must be listed and accepted before any drawdown counts, on a five working day listing lead time, and the marketplace fee is a margin input rather than a discount. The fee is priced into the two infrastructure lines above and not into the engagement fee.\n\nThat is the load-bearing point. Professional services cannot draw down that commitment. No infrastructure category in Verity's chart of accounts touches them, so the $520,000 has to come from Margriet's operating line.\n\n**Recommendation: two requisitions, not one.** A bundled requisition is rejected by whichever category owner does not expect the other half, and the rejection arrives without a human attached to it.",
"kind": "pricing"
},
{
"templateSlug": null,
"stage": "procurement",
"title": "DEMO — Pre-PO work log: sole-source, questionnaire, and the judge calibration result",
"status": "final",
"authoredDaysAgo": 5,
"body": "## Why this is a separate note\n\nEverything Nadia Osei's list permits before a PO, in one place, so nobody reconstructs it from a requisition thread in October. Two of these changed the deal: the judge calibration returned a result that removes an objective from the POC threshold, and the sole-source page is the only document a procurement analyst will read end to end.\n\n## Sole-source justification\n\nFour paragraphs, sent to Margriet on 11 August to paste under her own name. Paragraph two carries it, verbatim: \"The adapter weights sit over the same open-weights base the EU inference endpoint order form already specifies, so the trained model serves on that endpoint without a second migration. The held-out set was built against our own certified grouper licence and the payers' settled DRGs, and no other supplier has had access to either. Re-deriving both for a competitive process is nine weeks of Bram Kooij's time, which we do not have before the January Rheinklinik go-live.\"\n\n## Vendor security review\n\nRasmus Holm (Vendor Risk Lead, reporting to Owen McCabe) opened the review on 6 August. Twenty working days is typical at Verity, so 3 September, which is inside the 12 September freeze with a week of slack. Not blocking, and we hold Nadia Osei's written list of what may proceed alongside it: supplier onboarding, supplier record creation, the questionnaire, the sole-source page, and the marketplace listing. Everything except work touching a real customer trace.\n\n## The questionnaire\n\nThe questionnaire lives in a portal that opens only once a supplier record exists, so Nadia released the previous vendor's redacted copy instead and we answered against that. Eight of the twelve topics came straight from the standing library. Four went to Applied Research as one batch rather than four emails: base model licence lineage for the open 70B; whether any Verity artefact enters a shared training mix, answered no with the contractual clause cited rather than a policy sentence; judge model provenance and where it runs; and which artefacts cross the customer boundary in each direction. Returned in six days.\n\nOne answer we volunteered because a reviewer will find it anyway: the Frankfurt region has no failover, because the endpoint DPA forbids cross-region replication. A regional outage stops training and stops coding. Rasmus asked a follow-up about recovery time and we gave him a number rather than a paragraph.\n\n## Judge calibration, and the objective we dropped\n\nNadia's rule means no customer traces before a PO, so the judge could not be calibrated against Sabine Brandt's production adjudications. The set was built instead from 900 justifications synthesised from published German DRG appeal decisions and 300 written by Sabine's seniors against non-customer teaching cases, adjudicated twice.\n\nResult on 12 August: Cohen's kappa 0.68 against a human-to-human ceiling of 0.84 measured across the same seniors on the same material. The pre-committed floor in the architecture note was 0.75. It is below it.\n\nSo the justification objective comes out of the POC success threshold and becomes a secondary observation, per the rule written in July precisely so this would not be an argument. Told to Margriet Ruijs and Sabine Brandt on 12 August, before either asked. Sabine's reaction is the part worth recording: justification quality is the number she cares about most, and she now has a POC that does not promise it. She has not withdrawn her six half-days, but she asked what a second calibration on real traces would cost after the PO, which is the right question.\n\nTwo caveats on the 0.68. It is provisional: case law and teaching cases are cleaner than production discharge summaries, so the number could move either way once real traces are in scope. And a single kappa on a three-point clinical-adequacy scale is a coarse instrument. We are not arguing either point into a pass.",
"kind": "pricing"
},
{
"templateSlug": "procurement-and-budget-path",
"stage": "procurement",
"title": "DEMO — Procurement path: splitting REQ-2026-04471 before the 12 September freeze",
"status": "draft",
"authoredDaysAgo": 2,
"body": "## Where this stands\n\nProposal accepted verbally by Margriet Ruijs on 21 July. No PO. REQ-2026-04471 was raised on 30 July for the full $1,252,154 and rejected on 5 August, eight working days ago, by the category system with no human attached to the notice. Margriet read it as \"legal are looking at it\". They are not. The 20 July pricing note recommended two requisitions in bold; it was bundled anyway, which is on us for leaving that in a last paragraph rather than saying it on the phone.\n\n## Budget source\n\nTwo sources, and the failure was assuming one. Jan Kowalczyk (Director of Infrastructure) owns the €4.1m hyperscaler commitment, nineteen months to run. It funds GPU-hours and inference without new approval, because a private offer against it is a drawdown, and it cannot fund professional services. Margriet's business-unit operating line funds the $520,000 engagement fee, and it is the durable source: expansion next year is a line increase, not a fresh fight.\n\n## The unsticking move\n\n**REQ-B**, category 6420, professional services external: $520,000 against Margriet's line, sole-source attached at raise. Raised 13 August. Below the $750,000 CFO threshold, so it needs Margriet and Piotr Sawicki and nothing else — no CFO, no second legal pass, no board. That is the point of the split: half the money moves this week.\n\n**REQ-A**, category 7310, infrastructure committed drawdown, raised at the ceiling rather than the floor: 8,000 committed hours at $432,154, plus 11,400 on-demand at $57.75, $658,350, plus $300,000 of coding inference. $1,390,504. Not raised. It crosses $750,000, so it goes to the Investment Review Board, and the private offer must be listed today, Wednesday 19 August, to be accepted in five working days and counted on 26 August.\n\nNadia Osei (Senior Procurement Manager) was told the split is by funding category, not to duck a threshold. She agreed, then said she would have asked why we were bundling in July had anyone shown her the requisition first. Fair.\n\n## Approval chain\n\nBelow $25,000, budget owner alone. Above, Piotr Sawicki joins as finance business partner. Above $750,000, Greta Halvorsen (CFO), a second legal pass despite the MSA, and the fortnightly Investment Review Board: 26 August, then 9 September. The cadence, not the decision, costs the days.\n\nPiotr rebuilt the ceiling and came out at 22,600 hours, having applied the restart factor to eval sweeps and environment iteration as well as to campaigns. He wants REQ-A at $1,575,304 — $184,800 of ceiling nobody will use and a worse board paper. He thinks a ceiling that binds is not a ceiling. Unresolved, and it is what is stopping REQ-A today. The sole-source page, the security review, the questionnaire and the judge calibration are in the pre-PO work log; none of them is blocking.\n\n## Champion's homework\n\nFour things; she has done two. The sole-source page went in under her name on 12 August, and REQ-B was raised on 13 August with 6420 attached. Not done: raise REQ-A at 7310 once Piotr's ceiling settles, and send Jan the sentence we drafted on 6 August — \"Jan, the coding programme needs $1,390,504 of your hyperscaler commitment this year across compute and inference, as one infrastructure requisition; it is a drawdown, not new spend, and I need a yes or no by Thursday to make the 26 August board.\"\n\n## Where it is stuck\n\nJan has not replied in seven working days. He was told in July this was one requisition against his line and it is now a different, larger one. Margriet has not chased him because she still believes the rejection was legal. The move is three lines from Marisol Okonjo to Jan confirming the drawdown is expected — drafted, sent for signature on Friday, unsigned; she has missed two reviews since the Series C process started. If the offer is not listed today, REQ-A lands on 9 September, three days before the 12 September freeze, with no slack.\n\nWhat REQ-A cannot fix: the $300,000 inference line attaches to an endpoint order form that is not signed. Nadia says a requisition may be approved against an unexecuted contract; Ana Beltrán says it may not. Unresolved, and the second-largest risk here.",
"kind": "pricing"
}
],
"scores": [
{
"daysAgo": 84,
"note": "DEMO — Scored straight off the 25 May Amsterdam call. 62.5, Qualified conditional. The verifier is unusually good for healthcare because the payer's settled DRG is an adjudicated label produced outside the system, and Bram Kooij made the headroom argument himself. Against that: no frozen split, so the 68.1% is rolling production traffic and baseline scores a 1; no named budget beyond \"the platform side\"; and nine hospital-group contracts that are silent on training.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 3
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 58,
"note": "DEMO — One move, downward. Amendments went to all nine groups on 22 May; seven executed, and two German university hospitals have not, so the corpus is seven groups and about 21% of volume is out. Katrin Vogel will not sign a DPIA that enumerates controllers who have not signed, so trace and data rights fall 2 to 1. Nothing else changed: still no frozen split, still no budget owner, and the DPA we are amending is itself still in redline on the endpoint deal.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 1
},
{
"id": "baseline_measured",
"weight": 8,
"score": 1
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 3
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 28,
"note": "DEMO — The technical side moved at once on the architecture and proposal work: environment designed and catalogue keying resolved (task definability 4), Bram measured 67.4% on the 4,000-case split before sealing it (baseline 4), the prompted pipeline is flat five months rather than eleven weeks (headroom 4), no tool use or live system needed (environment 4), and Margriet signed the 80.0% target in writing (metric owner 4). Commercially, pricing named two funding sources for the first time — budget source 1 to 2, and no higher until a PO exists — Ana Beltrán agreed the training-amendment language for the seven signed groups (security and legal 2 to 3, which is the ceiling on this deal), and the 12 September freeze became a hard date (forcing function 4). Data rights recover one point because the seven-group corpus is contractually clean and the POC is scoped to it; verifier holds at 3 until the judge is calibrated.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 4
},
{
"id": "headroom",
"weight": 10,
"score": 4
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 2
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 4
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 1,
"note": "DEMO — 80.75, still Build, and two dimensions fall. Budget source 2 to 1: REQ-2026-04471 was rejected on category grounds on 5 August and Jan Kowalczyk has not replied to the 6 August note in eight working days, so two sources are named and neither has produced a PO. Exec sponsor 3 to 2: Marisol Okonjo has missed two reviews since the Series C process started and has not signed the note to Jan. Holds that are not ratchets — verifier stays at 3 because the judge calibrated at kappa 0.68 against a 0.84 human ceiling and the justification objective has been dropped from the threshold per the pre-committed rule; expansion surface stays at 3 because the DPIA blocks any move to full volume; data rights stay at 2 with Bergwald's ethics committee in October.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 3
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 4
},
{
"id": "headroom",
"weight": 10,
"score": 4
},
{
"id": "env_constructibility",
"weight": 8,
"score": 4
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 1
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 2
},
{
"id": "security_legal_path",
"weight": 8,
"score": 3
},
{
"id": "forcing_function",
"weight": 8,
"score": 4
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
}
],
"promote": null
}
@@ -0,0 +1,257 @@
{
"key": "verity-eu-inference",
"dealName": "DEMO — EU inference endpoint",
"accountName": "DEMO — Verity Health AI",
"stage": "scoping",
"status": "open",
"summary": "DEMO — Verity needs EU-resident inference for a fourteen-hospital German pilot going live 12 January 2027; we scoped down from the fine-tuning programme they asked for and sold the endpoint that has to exist first. The architecture is drafted and defensible at sixteen GPUs rather than the eight I first quoted, and Katrin Vogel, the fractional DPO, has been silent for the nine days since she flagged the hash-based pseudonymisation on 10 August — she is the gate on every EU conversation at this account.",
"openedDaysAgo": 126,
"artefacts": [
{
"templateSlug": "technical-discovery",
"stage": "qualification",
"title": "DEMO — First technical discovery: Verity Scribe in the EU",
"status": "final",
"authoredDaysAgo": 105,
"body": "DEMO — *Playbook stage: qualification.* Ninety minutes, 6 May 2026, Verity's Lisbon office. Present: Priya Raghunathan (VP Platform Engineering, champion), Tomas Lindqvist (Director of Clinical NLP, owns the metric), Owen McCabe (Head of Security and Compliance). Marisol Okonjo (CTO) joined for the last twenty minutes.\n\n## The workflow\n\nVerity Scribe turns an ambient consultation recording into a structured clinical note. One encounter is one unit of work: roughly 9,400 input tokens of transcript, 1,100 output tokens of structured note against their own schema. The clinician accepts, edits, or rejects. Acceptance — first pass, no edit beyond typographical — is the number the business runs on, and it is instrumented per speciality and per site.\n\nRepeatable: yes, 1.9M encounters a year in the US. Traced: yes, every request and the clinician's disposition, retained 400 days. Checkable: only by a human today. Tomas has no computable verifier, and said so without being asked twice, which is a good sign.\n\n## The failed attempt and the real problem\n\nThey are not here because the model is bad. They are here because of geography. Rheinklinik Verbund, a fourteen-hospital group, signed in March for a pilot going live 12 January 2027. The DPA Rheinklinik's counsel wrote forbids any processing of patient data outside the EU, including inference, including transient. Verity currently serves `verity-scribe-v4` — a 70B open-weights base SFT'd on 240,000 consented US encounters — from a single US region. Their separate coding pipeline runs on a prompted frontier API rather than on anything of their own, which will matter when we get to the training conversation. There is no EU serving path today and no in-house capacity to build one before January.\n\nSecond problem, discovered in the same hour: the model is worse in German. Their internal pilot on 412 consented and synthetic German encounters returned 58.2% acceptance against 73.6% in US English. Tomas has not published this number internally.\n\n## Data\n\nPHI throughout. Pseudonymisation happens in Verity's own tenant before anything leaves it — a hash-based identifier replacement Owen described as \"good enough for the US, untested against a German DPO\". Rights to use EU transcripts for model improvement are not in the Rheinklinik MSA at all; Priya believes they are addable, Owen does not.\n\n## Environment and economics\n\nServing, not training, is what they are buying first. Load: 6,000 encounters in pilot month one, 38,000 a month at full rollout in month six. Peak concurrency estimated at 140 requests in flight between 08:00 and 11:00 CET, extrapolated from US clinic patterns — Rheinklinik will not share scheduling data before signature, so treat this as ±40%. Latency budget: p95 time-to-first-token 700 ms, p95 complete note 25 s. Their current US p95 is 19 s on shorter US-English transcripts, so do not assume it transfers.\n\n## Org and counterfactual\n\nPriya owns uptime and latency. Tomas owns acceptance. Marisol wants the weights running inside Verity's own Azure EU tenant eventually and said so unprompted — flag this, it is a deployment-expectation risk, not a preference. Budget sits on an EU expansion line Priya could not name the owner of. Katrin Vogel, who took the Data Protection Officer role in February when Elena Duarte left, is fractional at two days a week and was not in the room.\n\nThe counterfactual is the incumbent's managed endpoint, which Priya priced at roughly twice our per-token rate, and no route to the fine-tuning they will need once the 15-point German gap becomes visible to Rheinklinik.",
"kind": "discovery"
},
{
"templateSlug": "trainability-qualification",
"stage": "qualification",
"title": "DEMO — Qualification read-out: why we scoped down to inference first",
"status": "final",
"authoredDaysAgo": 102,
"body": "DEMO — *Playbook stage: qualification.* Scored from the 6 May discovery notes, three days after the call. Result 52 of 100 — Scope down.\n\n## What the score says\n\nScored as the fine-tuning programme Verity walked in asking for, this deal does not clear. Three dimensions carry the damage and all three are real:\n\n**Verifier availability (1).** Clinician acceptance is a human disposition. There is no computable check on a clinical note today and no adjudicated set to build a judge against. Not a disqualifier — there is a visible path — but it is a quarter of work nobody has started.\n\n**Traces and rights (1).** The Rheinklinik MSA grants no model-improvement rights over EU transcripts. Owen McCabe reads the omission as deliberate on their counsel's part. Until that is amended we have no lawful training corpus in the EU, and the US corpus cannot be used to fix a German-language gap under a DPA that forbids the data crossing at all.\n\n**Security and legal path (1).** No DPA with us, no EU processing today, and the person who has to approve any of it — Katrin Vogel, DPO — works two days a week and has not yet been introduced.\n\n## What is strong\n\nForcing function (3) is the best I have seen this quarter: a signed customer contract with a fixed 12 January go-live and an EU-residency clause that Verity cannot engineer around. Metric owner (3) is Tomas Lindqvist, who volunteered a number that makes his own team look bad, which is the behaviour you want. Exec sponsor (3) is Marisol. Expansion surface (3) is the fine-tuning work the German gap will force within two quarters.\n\n## What the score does not cover\n\nBaseline (2) and headroom (2) are scored here on the serving side, not the model side. What is measured is a US p95 of 19 s on US-English transcripts. Nothing has been measured in region, at German transcript lengths, against the 700 ms and 25 s budgets in Rheinklinik's schedule. The 58.2% German acceptance figure is not evidence for this deal — the endpoint does not fix it — it is evidence for the expansion surface.\n\n## The decision\n\nScope down, and be specific about what to. Split the engagement in two. Sell the EU-resident inference endpoint now — it is the only thing that has to exist by January, it is the only thing the score supports, and it puts our infrastructure inside the boundary that every later conversation depends on. Open the fine-tuning conversation as a separate demand record once the German acceptance gap is published internally and the MSA amendment is on someone's desk.\n\nThis is not a consolation prize. The endpoint deal is $433,382 and it makes us the incumbent inside the compliance boundary before anyone competes for the training work.\n\n## What must be true before legal\n\nPriya names the budget owner. Tomas commits to a German-language baseline measurement on our endpoint in pilot month one. Owen introduces Katrin Vogel. If Katrin cannot be got into a room within three weeks, that is the finding, not a scheduling problem — she gates this deal and every EU conversation after it.",
"kind": "qualification"
},
{
"templateSlug": "data-and-security-brief",
"stage": "legal",
"title": "DEMO — Data and security brief: the PHI boundary for the EU endpoint",
"status": "final",
"authoredDaysAgo": 60,
"body": "DEMO — *Playbook stage: legal.* Filled in with Priya Raghunathan and Owen McCabe on 12 June, taken into the first joint counsel call on 19 June and written up the following day. Fifteen decisions; eleven resolved, four named below as open.\n\n## Legal basis and roles\n\nVerity is controller for encounter data and processor for Rheinklinik. Prime Intellect is a sub-processor, appointed under Verity's existing sub-processor notification obligation to Rheinklinik — which requires thirty days' notice, so the notice must go out by 13 December for a 12 January go-live. Our paper starts. Verity's counsel (Ana Beltrán, external, Lisbon) accepted the standard DPA with two redlines, both on breach-notification timing.\n\n## Permitted use, and what Owen's questionnaire actually forced\n\nOwen's questionnaire runs to 214 items. Three wrote this section.\n\nItem 3.11 asks whether any vendor employee can read a payload, including during a live incident. Our support terms allow a named engineer to, with customer consent. Owen refused a consent-based carve-out, because Rheinklinik's DPA does not give Verity standing to grant that consent. So it is cut: no human review of payloads under any circumstance including support. It costs us — a customer-reported bad note is reproducible only from Verity's audit log — and Priya accepted that trade in writing on 12 June.\n\nItem 3.14 asks whether payloads train anything. No model improvement, no retention of request or response bodies. The rights a future fine-tuning engagement needs are explicitly NOT granted here and must be a separate amendment; do not let anyone conflate the two documents.\n\nItem 9.6 asked for Rheinklinik's auditor to have a right of physical inspection of our Frankfurt racks on five business days' notice. We refused it — we cannot grant a third party site access on that notice in a shared facility. Counter-offer: audit exercisable through Verity, once a year, thirty days' notice, at Verity's cost, plus our C5 attestation. Ana Beltrán accepted. Rheinklinik's counsel has not answered. That is open decision four and it is ours, not theirs.\n\n## Data classes and boundary\n\nPseudonymised consultation transcripts: free text, containing residual PHI by construction — no pseudonymisation survives a clinician saying a patient's name mid-sentence. Treat every payload as special-category health data under Article 9. Verity applies hash-based identifier replacement in their own Azure EU tenant before egress.\n\nOpen decision one: Katrin Vogel, Verity's fractional DPO, has not validated that method, and Rheinklinik's own DPO may hold it to a stricter standard. If it is judged insufficient nothing changes technically — the boundary already assumes residual PHI — but Verity's controller analysis for the German entity shifts and their notice to Rheinklinik has to be rewritten.\n\n## Residency and egress\n\nAll inference in the Frankfurt region. No cross-region failover, no US read replica, no support access from outside the EU. Egress limited to aggregate metrics: request counts, token counts, latency histograms, error codes.\n\n## Retention and deletion\n\nZero retention of payloads, in-memory only. Owen treated that as table stakes and spent his time on error paths instead: item 5.9 asks what a stack trace contains. Our default triage log keeps the first 200 characters of a failing request. Off for this tenant — that is a code change in the Frankfurt gateway, not a setting, and it sits with Jonas Neubauer for the September build. Aggregate metrics retained thirteen months for billing dispute; Owen asked for six, we held at thirteen because it is our own dispute window, and he took it.\n\n## Certifications\n\nRheinklinik's schedule 4 enumerates two by name: ISO/IEC 27001 with a Statement of Applicability covering cloud processing, and BSI C5 Type 2. We hold both, C5 attested March 2026 for the Frankfurt region. Verity holds neither. Owen is mid ISO 27001, stage 1 audit booked November 2026, certification expected February 2027 — one month after go-live. Open decision two: he is planning to ask Rheinklinik for a waiver. Our exposure is a slipped go-live, not a compliance failure, but a slipped go-live moves our revenue start.\n\n## Ownership\n\nVerity owns `verity-scribe-v4` and every derivative. We serve it, we do not hold rights in it. Open decision three: Marisol wants a contractual exit that lets Verity take the serving stack in-house at renewal. Commercially answerable; not answered.",
"kind": "discovery"
},
{
"templateSlug": "technical-narratives",
"stage": "scoping",
"title": "DEMO — Narrative for Marisol Okonjo: you are buying a boundary, not GPUs",
"status": "final",
"authoredDaysAgo": 15,
"body": "DEMO — *Playbook stage: scoping, used wherever it is needed.* Written for Marisol Okonjo, CTO. Delivered on the 4 August call after she asked, for the third time, why she cannot just run the weights herself.\n\n**Audience.** Marisol Okonjo, CTO, Verity Health AI. She is judged on gross margin per encounter and on not being locked in. She has been through one vendor migration that cost her team a quarter, and she is currently raising, which makes every recurring line item on a slide an argument she has to have with someone else. What she loses by agreeing with me: optionality, in her framing, and a line on the infrastructure slide.\n\n**The belief, in her words.** \"It is our checkpoint and open weights. Serving a 70B model is a solved problem. If we are going to end up running this in our own EU tenant anyway, paying you to run it for a year is a detour with a bill attached.\"\n\n**The narrative.** She is right that serving a 70B model is solved. She is wrong about what she is buying, and the date is what makes the difference.\n\nYou have 161 days until 12 January. In that window your team has to stand up EU-resident capacity, pass Rheinklinik's schedule 4, get an ISO 27001 stage 1 audit through in November, and rewrite the pseudonymisation path if Katrin rules against the hash method. Three of those four are things only your team can do. The fourth is the only one you can buy, and it is the one with the longest lead time if you do it yourself, because the hard part is not the inference server — it is being able to answer a German hospital group's DPO in writing about single-tenancy, payload retention, and support access, with an audit trail, in December.\n\nWhat you are buying is a compliance boundary that already exists, with our name on the sub-processor notice thirty days before go-live. Not GPUs.\n\nOn lock-in: the weights are yours, the checkpoint is yours, and the exit is a DNS change and a config push, because the gateway that talks to us is code your team wrote and controls. There is no proprietary format in the path. If at renewal you want it inside your tenant, we should write that path into the contract now rather than argue about it in eleven months.\n\nAnd the thing I would not say if I were only trying to close this: the endpoint is not what fixes your German acceptance rate. 58.2% against 73.6% is a model problem, not an infrastructure problem, and it becomes Rheinklinik's problem around pilot month two. Getting the endpoint live in January is what buys you the time to deal with it.\n\n**The tell that it landed.** She stops arguing about price and asks who writes the renewal exit clause. She did, on 4 August. Piotr Sawicki was asked to model both paths the following week.",
"kind": "narrative"
},
{
"templateSlug": "reference-architectures",
"stage": "scoping",
"title": "DEMO — Architecture selection: EU-resident inference endpoint, Frankfurt",
"status": "draft",
"authoredDaysAgo": 8,
"body": "DEMO — *Playbook stage: scoping.* Draft for the 27 August architecture review with Priya and Owen. Not shown to Marisol.\n\n## Which shape this is\n\nThe serving edge of **architecture 2, SFT on traces**, training loop absent because the rights do not exist yet, with **architecture 1, eval-first** alongside at Verity's cost. This is the substrate the fine-tuning deal attaches to.\n\n## The diagram\n\n```mermaid\nflowchart LR\n subgraph EU[Verity Azure EU tenant]\n C[Clinic app]\n Ps[Pseudonymise]\n Gw[Gateway]\n Au[Audit log]\n Ev[Eval harness]\n end\n subgraph PI[PI Frankfurt region]\n Ep[Dedicated endpoint]\n W[v4 weights]\n Mt[Metrics only]\n end\n subgraph US[Verity US]\n Cp[Control plane]\n Bi[Billing]\n end\n C --> Ps --> Gw --> Ep\n W --> Ep\n Ep -- note --> Gw\n Gw --> C\n Gw --> Au\n Au --> Ev\n Ep --> Mt --> Bi\n Cp -. config only .-> Gw\n```\n\n## Component ownership\n\n| Component | Who runs it | Why it is there |\n| --- | --- | --- |\n| Gateway | Verity | The only thing that talks to us; holds the audit log, so a bad note is reproducible their side, not ours. |\n| Dedicated endpoint | PI | Single-tenant, sixteen reserved H200s as eight TP2 replicas, Frankfurt. Single-tenancy is questionnaire item 3.2, not an upsell. |\n| v4 weights | Verity, hosted by PI | Their IP; copy deleted on termination. |\n| Metrics only | PI | Counts, tokens, latency, errors. No payload fragments. |\n| Eval harness | Verity | German acceptance, measured against the endpoint. Tomas owns it. |\n\n## Where PHI crosses a boundary\n\nOnce, in one direction, never leaving the EU. Pseudonymisation and clinic-to-gateway are internal to Verity; gateway to endpoint is the only external hop, and the one the DPA governs; the return path carries the note, also PHI. The US control plane pushes config and reads nothing — keep that line dotted and one-way, Owen will interrogate it.\n\n## Sizing, including the number I got wrong\n\nRun-rate: 38,000 encounters a month, 9,400 in and 1,100 out — 456,000 a year, about 500M output tokens and 4.3B in. Year one rides the ramp (6k to 38k over pilot months one to six): 355,000 encounters, ~390M out. Reserve on run-rate; forecast on the ramp.\n\nMy first model said eight GPUs. It was wrong: I took the US p95 of 19 s and assumed it transferred, so I sized on decode alone. Decode is the easy half — memory-bound at ~68 tokens/s per in-flight request, so 1,100 output tokens is ~16 s, ~8 s under the 25 s note budget at 140 concurrent. TTFT binds: a 9,400-token German transcript is prefill-bound. At 140 in flight, ~16.5 s each, we retire ~8.5 requests/s — ~80,000 prefill tokens/s. A TP2 H200 replica sustains ~11,000 with chunked prefill. Four replicas is 44,000 and TTFT never converges. Eight is 88,000, 91% loaded at peak, modelled p95 640 ms — modelled, not measured. Load test booked 3 September; under 9,000 per replica this is wrong again. At 196 concurrent, demand is ~112,000: eleven replicas, six GPUs beyond the reservation.\n\nOwn goal, stated because Priya will ask: the eight-GPU figure sits inside the $433,382 quoted in May. I am not repricing at scoping over my own arithmetic. The delta comes out of our margin; Nils Ekström signed it off as boundary-entry cost.\n\nWho pays for burst: $32,102 of the $433,382 is held as pre-purchased on-demand — 7,465 GPU-hours at our $4.30 rate, six extra GPUs across the CET peak for about 400 working days. If concurrency lands nearer 196, that pool absorbs two quarters and Verity tops up at list after. Clause drafted; Priya has seen it.\n\n## Three decisions that redraw this\n\n**Payload logging.** Off, in-memory only. On, it puts a retention node holding Article 9 data in our region and reopens the DPA. Off; we debug blind.\n\n**Where pseudonymisation sits.** If Katrin rules the hash method insufficient it moves into the clinic app and Verity's German controller analysis changes. The diagram survives; their notice does not.\n\n**Hosted versus weights in Verity's tenant.** Marisol wants the second at renewal. That deletes the PI subgraph and makes this a support deal at another price. Not answered here.",
"kind": "architecture"
}
],
"scores": [
{
"daysAgo": 102,
"note": "DEMO — First score, off the 6 May discovery. 52, Scope down: no computable verifier, no EU training rights in the Rheinklinik MSA, and no DPA with us. Baseline and headroom are scored on the serving side only — a US p95 of 19 s and nothing measured in region. Splitting the engagement: sell the inference endpoint now, open fine-tuning as a separate record once the German acceptance gap is published internally.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 1
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 1
},
{
"id": "baseline_measured",
"weight": 8,
"score": 2
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 2
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 2
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 1
},
{
"id": "forcing_function",
"weight": 8,
"score": 3
},
{
"id": "expansion_surface",
"weight": 5,
"score": 3
}
]
},
{
"daysAgo": 58,
"note": "DEMO — Rescored 22 June, three days after the joint counsel call. 65, Qualified conditional — over the line for the first time. Piotr Sawicki named the $2.3M FY27 EU expansion line, Rheinklinik's go-live is contractually fixed with a service credit attached, Ana Beltrán has a model-improvement amendment drafted and sitting with Rheinklinik's counsel, and the fine-tuning conversation is now a separate demand record with its own scorecard, which is what moves expansion surface. Baseline and headroom do not move: Tomas publishing 58.2% is evidence for the training deal, not this one, and we still have no in-region latency measurement. Still open: DPO validation of the pseudonymisation method.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 3
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 2
},
{
"id": "baseline_measured",
"weight": 8,
"score": 2
},
{
"id": "headroom",
"weight": 10,
"score": 2
},
{
"id": "env_constructibility",
"weight": 8,
"score": 2
},
{
"id": "metric_owner",
"weight": 9,
"score": 3
},
{
"id": "budget_source",
"weight": 9,
"score": 3
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 3
},
{
"id": "security_legal_path",
"weight": 8,
"score": 2
},
{
"id": "forcing_function",
"weight": 8,
"score": 4
},
{
"id": "expansion_surface",
"weight": 5,
"score": 4
}
]
},
{
"daysAgo": 6,
"note": "DEMO — 71, Qualified conditional. Architecture drafted, Tomas has signed the German acceptance target in writing, and the Frankfurt single-tenant shape is now drawn and costed against measured German token lengths, so task definability, metric owner, environment constructibility, baseline and headroom all move — 17 s modelled against a 25 s budget is the first serving evidence this deal has had. Three things got worse. My first sizing model undercounted the GPUs by half and the correction comes out of our margin. Katrin Vogel flagged the hash-based pseudonymisation as possibly insufficient on 10 August and has not answered since, which stalls the MSA amendment and leaves the ISO waiver unresolved with the 13 December sub-processor notice inside the window — security and legal path drops to 1 and traces and rights back to 1. Marisol has missed two reviews since the Series C process started.",
"dimensions": [
{
"id": "task_definability",
"weight": 10,
"score": 4
},
{
"id": "verifier_quality",
"weight": 13,
"score": 2
},
{
"id": "trace_data_rights",
"weight": 8,
"score": 1
},
{
"id": "baseline_measured",
"weight": 8,
"score": 3
},
{
"id": "headroom",
"weight": 10,
"score": 3
},
{
"id": "env_constructibility",
"weight": 8,
"score": 3
},
{
"id": "metric_owner",
"weight": 9,
"score": 4
},
{
"id": "budget_source",
"weight": 9,
"score": 3
},
{
"id": "exec_sponsor",
"weight": 4,
"score": 2
},
{
"id": "security_legal_path",
"weight": 8,
"score": 1
},
{
"id": "forcing_function",
"weight": 8,
"score": 4
},
{
"id": "expansion_surface",
"weight": 5,
"score": 4
}
]
}
],
"promote": null
}
+9 -2
View File
@@ -350,9 +350,16 @@ async function seed() {
// ------------------------------------------------ the Motion starter library
const motion = await seedMotionLibrary(db);
console.log(
` ${motion.total} Motion starter template(s) (${motion.added} new) — authored content, ` +
'shared and system-owned, version 1 of their lineages.',
` ${motion.total} Motion starter template(s) (${motion.added} new, ${motion.refreshed} ` +
'refreshed) — authored content, shared and system-owned, version 1 of their lineages.',
);
if (motion.held.length) {
console.log(
` ${motion.held.length} starter template(s) have newer wording here and were LEFT ALONE, ` +
`because they are in use or have been adopted: ${motion.held.join(', ')}. ` +
'Publish a new version rather than editing one an engagement was cut from.',
);
}
for (const bad of motion.rejected) {
console.error(` SKIPPED a starter template with a value outside the ontology: ${bad}`);
}
File diff suppressed because one or more lines are too long
+102 -9
View File
@@ -19,12 +19,22 @@
* against this to confirm the attribute syntax is understood by each, since
* the server runs the TypeScript directly.
*
* Idempotency is `onConflictDoNothing({ target: [slug, version] })`, which
* works **only** because of `motion_templates_slug_version_key`. AGENTS.md §5
* is blunt about what that clause does without a constraint to fire on — it
* silently duplicated seed data here twice — so `assertNoDuplicates` below
* checks the outcome rather than trusting the schema, and CI counts this table
* in its idempotency gate alongside `contacts`.
* Idempotency is keyed on `(slug, version)`, which is enforceable **only**
* because of `motion_templates_slug_version_key`. AGENTS.md §5 is blunt about
* what `onConflictDoNothing` does without a constraint to fire on — it silently
* duplicated seed data here twice — so `assertNoDuplicates` below checks the
* outcome rather than trusting the schema, and CI counts this table in its
* idempotency gate alongside `contacts`.
*
* A second run REFRESHES a starter row rather than skipping it, but only while
* that row is still ours: `is_system`, nobody has forked or instantiated it
* (`usage_count = 0`) and it has no owner. Insert-only was the first version and
* is wrong for the one thing this library is for — the content improves, and a
* deployment seeded in August would otherwise be frozen on August's wording for
* ever, with no upgrade path short of hand-editing production rows. The
* pristine test is what keeps that from becoming a write over somebody's work:
* the moment a template has been used, §7a says it is never edited in place,
* and this respects that with the same condition the API enforces.
*/
import { DEMAND_STAGES, isMotionKind, type DemandStage, type MotionKind } from '@pig/core';
import { and, eq, inArray } from 'drizzle-orm';
@@ -89,6 +99,29 @@ const isDemandStage = (value: string): value is DemandStage =>
* inside `fields` means promotion carries it forward with the rest of the
* structured payload without any special handling in the promote path.
*/
/*
* Key order is not preserved by `jsonb`: Postgres stores an object with its
* keys sorted by length and then bytewise, so the value that comes back is
* rarely the value that went in. A plain `JSON.stringify` comparison therefore
* reported every starter template as changed on every run, and the seed
* rewrote nine rows each time while claiming to be idempotent. Found by
* running it twice and reading the count, not by reading the code.
*/
function canonical(value: unknown): string {
const order = (node: unknown): unknown => {
if (Array.isArray(node)) return node.map(order);
if (node && typeof node === 'object') {
return Object.fromEntries(
Object.keys(node as Record<string, unknown>)
.sort()
.map((key) => [key, order((node as Record<string, unknown>)[key])]),
);
}
return node;
};
return JSON.stringify(order(value));
}
function toRow(
template: StarterTemplate,
kind: MotionKind,
@@ -144,9 +177,17 @@ async function assertNoDuplicates(db: Database, slugs: readonly string[]): Promi
export async function seedMotionLibrary(
db: Database,
): Promise<{ total: number; added: number; rejected: readonly string[] }> {
): Promise<{
total: number;
added: number;
refreshed: number;
held: readonly string[];
rejected: readonly string[];
}> {
const rejected: string[] = [];
const held: string[] = [];
let added = 0;
let refreshed = 0;
for (const template of STARTER_LIBRARY) {
/*
@@ -161,12 +202,64 @@ export async function seedMotionLibrary(
continue;
}
const row = toRow(template, template.kind, template.stage);
const [existing] = await db
.select({
id: motionTemplates.id,
isSystem: motionTemplates.isSystem,
usageCount: motionTemplates.usageCount,
ownerUserId: motionTemplates.ownerUserId,
title: motionTemplates.title,
summary: motionTemplates.summary,
body: motionTemplates.body,
fields: motionTemplates.fields,
kind: motionTemplates.kind,
stage: motionTemplates.stage,
})
.from(motionTemplates)
.where(and(eq(motionTemplates.slug, template.slug), eq(motionTemplates.version, 1)))
.limit(1);
if (!existing) {
const [created] = await db
.insert(motionTemplates)
.values(toRow(template, template.kind, template.stage))
.values(row)
.onConflictDoNothing({ target: [motionTemplates.slug, motionTemplates.version] })
.returning({ id: motionTemplates.id });
if (created) added += 1;
continue;
}
const pristine =
existing.isSystem && existing.usageCount === 0 && existing.ownerUserId === null;
const changed =
existing.title !== row.title ||
existing.summary !== row.summary ||
existing.body !== row.body ||
existing.kind !== row.kind ||
existing.stage !== row.stage ||
canonical(existing.fields) !== canonical(row.fields);
if (pristine && changed) {
await db
.update(motionTemplates)
.set({
title: row.title,
summary: row.summary,
body: row.body,
fields: row.fields,
kind: row.kind,
stage: row.stage,
updatedAt: new Date(),
})
.where(eq(motionTemplates.id, existing.id));
refreshed += 1;
} else if (changed) {
// Reported rather than forced: the row has been used or adopted, so
// overwriting it would edit a template a live engagement was cut from.
held.push(template.slug);
}
}
const total = await assertNoDuplicates(
@@ -174,5 +267,5 @@ export async function seedMotionLibrary(
STARTER_LIBRARY.map((template) => template.slug),
);
return { total, added, rejected };
return { total, added, refreshed, held, rejected };
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+365
View File
@@ -0,0 +1,365 @@
/*
* Build a /learn video from a script file.
*
* node scripts/learn-film.mjs docs/learn-films/<slug>.json
*
* The five original platform videos were cut by an ad-hoc process that was
* never committed, so the first time the UI moved nobody could re-shoot them.
* This is that pipeline, written down. It reuses the two hard-won lessons from
* `scripts/screenshots.mjs`: the appearance preference is stored SERVER-side
* and adopted after hydration, so seeding localStorage is not enough and the
* profile response has to be rewritten; and per-device layout state
* (`pig.sidebarOpen`, `pig.piggyDockOpen`) has to be forced or the framing is
* whatever a human last left behind.
*
* ORDER IS LOAD-BEARING. The narration is rendered FIRST and its measured
* duration drives every shot length. A shot list timed by guess leaves the
* narrator talking over a frozen frame, or gets cut mid-sentence.
*
* THE CAMERA. Landscape, and still by default. An earlier version drifted a
* slow zoom across every shot, which reads as restless rather than as emphasis
* the whole frame moving all the time is the thing people mean when they call
* a screen recording "annoying". This one holds still, and where a line points
* at one thing it eases IN on that element, holds, and eases back OUT, the way
* Cap zooms toward a click. The easing is a smoothstep either side of a hold,
* so there is no visible start or stop.
*
* Typography is composed in the browser, not in ffmpeg: the product's face is
* Manrope Variable and drawtext would fall back to DejaVu, which reads as a
* different company. Playwright renders a transparent caption layer per shot
* and ffmpeg only moves pixels.
*
* Requires: a running API + web (PIG_WEB_URL), Chatterbox reachable, ffmpeg.
*/
import { chromium } from 'playwright';
import { execFile } from 'node:child_process';
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';
import { promisify } from 'node:util';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
const run = promisify(execFile);
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO = resolve(HERE, '..');
const WEB = process.env.PIG_WEB_URL ?? 'http://localhost:5179';
const TTS = process.env.PIG_TTS_URL ?? 'http://100.127.247.67:8095';
const VOICE = process.env.PIG_TTS_VOICE ?? 'karti';
/* 1.20x is the rate docs/learn-scripts.md is written to. */
const TEMPO = Number(process.env.PIG_TTS_TEMPO ?? 1.2);
const MUSIC = process.env.PIG_FILM_MUSIC ?? join(REPO, 'apps/web/public/audio/pig-ambient.mp3');
const W = 1920;
const H = 1080;
const SCALE = 2;
const FPS = 30;
/* The zoom envelope, as fractions of the shot: hold, ease in, hold, ease out. */
const ZOOM_IN_START = 0.16;
const ZOOM_IN_END = 0.42;
const ZOOM_OUT_START = 0.76;
const ZOOM_OUT_END = 0.98;
const THEME = {
dark: { bg: '#0a0a0b', fg: '#fafafa', pill: 'rgba(18,18,20,.92)', rule: 'rgba(255,255,255,.14)', accent: '#22c55e' },
light: { bg: '#ffffff', fg: '#0a0a0b', pill: 'rgba(255,255,255,.94)', rule: 'rgba(0,0,0,.12)', accent: '#16a34a' },
};
async function ffprobeDuration(path) {
const { stdout } = await run('ffprobe', ['-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', path]);
return Number(stdout.trim());
}
async function narrate(text, out) {
const response = await fetch(`${TTS}/generate`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ text, voice: VOICE }),
});
if (!response.ok) throw new Error(`Chatterbox ${response.status}: ${await response.text()}`);
const wav = Buffer.from(await response.arrayBuffer());
/* Raw WAV bytes, not JSON. A JSON.parse here reads as a TTS outage. */
if (wav.subarray(0, 4).toString() !== 'RIFF') {
throw new Error(`Chatterbox did not return WAV: ${wav.subarray(0, 120).toString()}`);
}
await writeFile(out, wav);
return ffprobeDuration(out);
}
/** Shot lengths are proportional to the words spoken over them. */
function shotFrames(shots, total) {
const words = shots.map((shot) => shot.line.trim().split(/\s+/).length);
const sum = words.reduce((a, b) => a + b, 0);
const frames = words.map((n) => Math.max(FPS, Math.round(((n / sum) * total) * FPS)));
const target = Math.round(total * FPS);
frames[frames.length - 1] += target - frames.reduce((a, b) => a + b, 0);
return frames;
}
/**
* A smoothstep ramp between two fractions of the shot, as an ffmpeg expression.
*
* Written out rather than accumulated with zoompan's own `zoom` variable, which
* refers to the PREVIOUS frame and drifts: expressing the curve purely as a
* function of `on` means the zoom is the same on every re-render and lands
* exactly on 1.0 at both ends.
*/
function ramp(from, to, n) {
const u = `clip((on/${n - 1}-${from})/${(to - from).toFixed(4)},0,1)`;
return `(pow(${u},2)*(3-2*${u}))`;
}
function zoomExpression(scale, n) {
const rise = ramp(ZOOM_IN_START, ZOOM_IN_END, n);
const fall = ramp(ZOOM_OUT_START, ZOOM_OUT_END, n);
return `1+${(scale - 1).toFixed(4)}*(${rise}-${fall})`;
}
const CAPTION_HTML = (o) => `<!doctype html><html><head><meta charset="utf-8"><style>
@font-face{font-family:'Manrope';src:url('file://${o.font}') format('woff2');font-weight:200 800;font-display:block}
*{margin:0;padding:0;box-sizing:border-box}
html,body{width:${W}px;height:${H}px;background:transparent}
body{font-family:'Manrope',system-ui,sans-serif;-webkit-font-smoothing:antialiased}
.pill{position:absolute;left:56px;bottom:66px;display:flex;align-items:center;gap:16px;
padding:16px 26px;border-radius:999px;background:${o.pill};border:1px solid ${o.rule};
backdrop-filter:blur(12px);box-shadow:0 10px 40px rgba(0,0,0,.28)}
.dot{width:11px;height:11px;border-radius:50%;background:${o.accent};flex:none}
.text{font-size:29px;font-weight:600;letter-spacing:-.01em;color:${o.fg};white-space:nowrap}
.track{position:absolute;left:0;right:0;bottom:0;height:5px;background:${o.rule}}
.fill{height:5px;background:${o.accent};width:${(o.progress * 100).toFixed(2)}%}
</style></head><body>
<div class="pill"><span class="dot"></span><span class="text">${o.caption}</span></div>
<div class="track"><div class="fill"></div></div>
</body></html>`;
async function main() {
const scriptPath = process.argv[2];
if (!scriptPath) throw new Error('usage: learn-film.mjs <script.json>');
const film = JSON.parse(await readFile(scriptPath, 'utf8'));
const work = join(REPO, '.film', film.slug);
await rm(work, { recursive: true, force: true });
await mkdir(work, { recursive: true });
const font = join(
REPO,
'node_modules/.pnpm/@fontsource-variable+manrope@5.3.0/node_modules/@fontsource-variable/manrope/files/manrope-latin-wght-normal.woff2',
);
// 1 -------------------------------------------------------- the narration
const narration = film.shots.map((s) => s.line.trim()).join(' ');
const words = narration.split(/\s+/).length;
const rawVoice = join(work, 'voice-raw.wav');
const spoken = await narrate(narration, rawVoice);
const voice = join(work, 'voice.wav');
await run('ffmpeg', ['-v', 'error', '-y', '-i', rawVoice, '-filter:a', `atempo=${TEMPO}`, voice]);
const duration = await ffprobeDuration(voice);
console.log(` narration ${words} words -> ${spoken.toFixed(2)}s at 1.0x -> ${duration.toFixed(2)}s at ${TEMPO}x`);
const frames = shotFrames(film.shots, duration);
// 2 ----------------------------------------------------------- the routes
const engagements = await fetch(`${WEB}/api/motion/engagements`).then((r) => r.json());
const templates = await fetch(`${WEB}/api/motion/templates`).then((r) => r.json());
const byStage = (stage) => engagements.engagements.find((e) => e.stage === stage);
const alias = {
/* By slug, not by kind: two shipped templates are `playbook` kind, and
* which one `find` returned depended on list order. */
PLAYBOOK: templates.templates.find((t) => t.slug === 'strategic-deployment-playbook')?.id,
SCORECARD: templates.templates.find((t) => t.slug === 'trainability-qualification')?.id,
HALCYON: byStage('poc')?.id,
NORTHWIND: byStage('expansion')?.id,
};
for (const [name, id] of Object.entries(alias)) {
if (!id) throw new Error(`route alias ${name} resolved to nothing — is the demo book seeded?`);
}
const browser = await chromium.launch({ channel: 'chrome' });
const contexts = {};
for (const theme of ['dark', 'light']) {
const context = await browser.newContext({
/*
* A real 1920 desktop viewport, captured at 2x and delivered at 1920.
* Setting the viewport to W/SCALE was wrong in a way that looked right:
* it produced a 1920px image, but the PAGE believed it was 960 wide, so
* the sidebar collapsed and the app filmed its compact layout. The
* viewport is what the product responds to; the scale factor is only how
* much detail there is to zoom into.
*/
viewport: { width: W, height: H },
deviceScaleFactor: SCALE,
colorScheme: theme,
});
await context.addInitScript(`
localStorage.setItem('pig.themeMode', '${theme}');
localStorage.setItem('pig.sidebarOpen', 'true');
localStorage.setItem('pig.piggyDockOpen', 'false');
localStorage.setItem('pig.audio.enabled', 'false');
`);
await context.route('**/api/me/profile', async (route) => {
const response = await route.fetch();
const body = await response.json().catch(() => ({}));
await route.fulfill({ json: { ...body, themeMode: theme } });
});
contexts[theme] = context;
}
const captionPage = await browser.newPage({ viewport: { width: W, height: H } });
const segments = [];
for (const [index, shot] of film.shots.entries()) {
let route = shot.route;
for (const [name, id] of Object.entries(alias)) route = route.replace(name, id);
const page = await contexts[shot.theme].newPage();
await page.goto(`${WEB}${route}`, { waitUntil: 'networkidle' });
/* Lazy route chunks sit behind a Suspense boundary; a screenshot taken
* before the fallback resolves photographs the spinner. */
await page.waitForTimeout(1400);
const painted = await page.evaluate(() => document.documentElement.getAttribute('data-theme'));
if (painted !== shot.theme) throw new Error(`shot ${index} painted ${painted}, wanted ${shot.theme}`);
if (shot.scrollTo) {
const ok = await page.evaluate((text) => {
const node = [...document.querySelectorAll('h1,h2,h3')].find((el) => el.textContent?.trim() === text);
if (!node) return false;
(node.closest('section,[class*="card"]') ?? node).scrollIntoView({ block: 'center' });
return true;
}, shot.scrollTo);
if (!ok) throw new Error(`shot ${index}: no heading "${shot.scrollTo}" on ${route}`);
await page.waitForTimeout(600);
}
/* The zoom target, in captured pixels. Smallest element containing the
* text, so naming "82.8%" lands on the figure rather than on the card. */
let focus = null;
if (shot.zoom) {
/*
* Target selection is "smallest element already in frame", not "last
* match in document order", which is what this did first and got wrong in
* a way that only shows up on a long page. The Halcyon engagement prints
* the same phrases twice once in the score note and again inside the
* artefact bodies further down so "kappa 0.54" resolved to a paragraph
* two and a half thousand pixels below the fold, and no amount of
* scrolling brought it into view because an ancestor clips it.
*
* Preferring what is already on screen keeps the zoom in the frame the
* `scrollTo` chose; preferring the smallest match makes naming a figure
* land on the figure rather than on the card around it. Only if nothing
* matches in frame does it scroll, and then it re-measures.
*/
/* Inlined in both evaluates rather than injected as a string: the page
* is the only place this can run, and `new Function` is one CSP away from
* failing in an environment that is not the dev server. */
const locate = ({ text, scale, move }) => {
const matches = [...document.querySelectorAll('body *')].filter(
(el) => el.textContent?.includes(text) && el.getBoundingClientRect().width > 0,
);
if (!matches.length) return { found: false };
const area = (el) => {
const r = el.getBoundingClientRect();
return r.width * r.height;
};
const inFrame = matches.filter((el) => {
const r = el.getBoundingClientRect();
return r.bottom > 0 && r.top < window.innerHeight;
});
const el = (inFrame.length ? inFrame : matches).sort((a, b) => area(a) - area(b))[0];
const r = el.getBoundingClientRect();
if (r.bottom < 0 || r.top > window.innerHeight) {
if (move) {
el.scrollIntoView({ block: 'center' });
return { found: true, offScreen: true };
}
return { found: true, offScreen: true };
}
return {
found: true,
offScreen: false,
cx: (r.left + r.width / 2) * scale,
cy: (r.top + r.height / 2) * scale,
};
};
const first = await page.evaluate(locate, { text: shot.zoom.on, scale: SCALE, move: true });
if (!first.found) throw new Error(`shot ${index}: zoom target "${shot.zoom.on}" is not on ${route}`);
if (first.offScreen) await page.waitForTimeout(500);
focus = first.offScreen
? await page.evaluate(locate, { text: shot.zoom.on, scale: SCALE, move: false })
: first;
if (focus.offScreen) focus = null;
if (!focus) {
throw new Error(`shot ${index}: zoom target "${shot.zoom.on}" would not come into view on ${route}`);
}
}
const framePng = join(work, `frame-${index}.png`);
await page.screenshot({ path: framePng });
await page.close();
const palette = THEME[shot.theme];
await captionPage.setContent(
CAPTION_HTML({ ...palette, font, caption: shot.caption, progress: (index + 1) / film.shots.length }),
{ waitUntil: 'load' },
);
await captionPage.waitForTimeout(250);
const captionPng = join(work, `caption-${index}.png`);
await captionPage.screenshot({ path: captionPng, omitBackground: true });
// 3 ---------------------------------------------- one shot -> one clip
const n = frames[index];
const camera = focus
? `zoompan=z='${zoomExpression(shot.zoom.scale ?? 1.6, n)}':` +
`x='max(0,min(iw-iw/zoom,${focus.cx.toFixed(1)}-(iw/zoom)/2))':` +
`y='max(0,min(ih-ih/zoom,${focus.cy.toFixed(1)}-(ih/zoom)/2))':` +
`d=${n}:s=${W}x${H}:fps=${FPS}`
: `scale=${W}:${H}`;
const segment = join(work, `seg-${index}.mp4`);
await run('ffmpeg', [
'-v', 'error', '-y',
'-loop', '1', '-framerate', String(FPS), '-t', String(n / FPS), '-i', framePng,
'-i', captionPng,
'-filter_complex', `[0:v]${camera},format=yuv420p[base];[base][1:v]overlay=0:0:format=auto,format=yuv420p[v]`,
'-map', '[v]', '-frames:v', String(n),
/* CRF 23 on a still frame with an occasional slow zoom. The originals
* ship at about 420 kbps for 1440x810; this lands in the same class. */
'-c:v', 'libx264', '-preset', 'slow', '-crf', '23', segment,
]);
segments.push(segment);
console.log(
` ${String(index + 1).padStart(2)}/${film.shots.length} ${shot.theme.padEnd(5)} ` +
`${(n / FPS).toFixed(2)}s ${focus ? `zoom x${shot.zoom.scale ?? 1.6} on "${shot.zoom.on}"` : 'still'}`,
);
}
await browser.close();
// 4 ------------------------------------------------ cut, score, normalise
const list = join(work, 'segments.txt');
await writeFile(list, segments.map((s) => `file '${s}'`).join('\n'));
const silent = join(work, 'silent.mp4');
await run('ffmpeg', ['-v', 'error', '-y', '-f', 'concat', '-safe', '0', '-i', list, '-c', 'copy', silent]);
const out = join(work, 'film.mp4');
await run('ffmpeg', [
'-v', 'error', '-y',
'-i', silent, '-i', voice, '-stream_loop', '-1', '-i', MUSIC,
'-filter_complex',
`[2:a]volume=0.075,afade=t=in:st=0:d=1.2,afade=t=out:st=${(duration - 1.6).toFixed(2)}:d=1.6[bed];` +
`[1:a]adelay=250|250,apad[vox];` +
`[vox][bed]amix=inputs=2:duration=first:dropout_transition=0,` +
`atrim=0:${duration.toFixed(3)},loudnorm=I=-16:TP=-1.5:LRA=11[a]`,
'-map', '0:v', '-map', '[a]',
'-c:v', 'copy', '-c:a', 'aac', '-b:a', '160k', '-movflags', '+faststart', '-shortest', out,
]);
const finalDuration = await ffprobeDuration(out);
const hash = createHash('sha256').update(await readFile(out)).digest('hex').slice(0, 8);
const named = join(work, `${film.slug}.${hash}.mp4`);
await run('cp', [out, named]);
const poster = join(work, `${film.slug}.${hash}.jpg`);
await run('ffmpeg', ['-v', 'error', '-y', '-ss', '1.2', '-i', named, '-frames:v', '1', '-q:v', '3', poster]);
console.log(`\n ${film.slug}: ${finalDuration.toFixed(2)}s, ${words} words, ${W}x${H}`);
console.log(` ${named}\n ${poster}`);
}
await main();
+41 -6
View File
@@ -49,12 +49,31 @@ for (const theme of ['light', 'dark']) {
isMobile: vp.name === 'mobile',
hasTouch: vp.name === 'mobile',
});
// The sidebar and the dock are per-device states in localStorage; left to
// whatever a human last set, the measurement is not reproducible.
/*
* The sidebar and the dock are per-device states in localStorage; left to
* whatever a human last set, the measurement is not reproducible. The key
* names are `pig.sidebarOpen` and `pig.piggyDockOpen` an earlier version
* of this file invented `pig.sidebar` and `pig.piggy.dock`, which set two
* keys nothing reads and left the real state untouched, so the run looked
* pinned and was not.
*/
await ctx.addInitScript(`
localStorage.setItem('pig.sidebar', 'expanded');
localStorage.setItem('pig.piggy.dock', 'closed');
localStorage.setItem('pig.sidebarOpen', 'true');
localStorage.setItem('pig.piggyDockOpen', 'false');
localStorage.setItem('pig.themeMode', '${theme}');
`);
/*
* colorScheme alone only sets prefers-color-scheme, and the appearance
* preference is stored SERVER-side and adopted after hydration so the
* dark pass rendered light a beat after first paint and measured the wrong
* theme. Rewriting the profile response is what screenshots.mjs does, and
* for the same reason.
*/
await ctx.route('**/api/me/profile', async (route) => {
const response = await route.fetch();
const body = await response.json().catch(() => ({}));
await route.fulfill({ json: { ...body, themeMode: theme } });
});
for (const route of ROUTES) {
const page = await ctx.newPage();
@@ -90,11 +109,27 @@ for (const theme of ['light', 'dark']) {
.split(' ')
.slice(0, 4)
.join('.')}`);
return { overflow, wide, text: document.body.innerText.slice(0, 120) };
return {
overflow,
wide,
theme: doc.getAttribute('data-theme'),
chars: document.body.innerText.trim().length,
text: document.body.innerText.slice(0, 120),
};
});
const label = `${route.slug} ${vp.name} ${theme}`;
if (measured.overflow !== 0) {
if (measured.theme !== theme) {
// Without this the dark pass silently measured the light palette, and a
// dark-only overflow — a wider border, a different font fallback —
// would have gone unseen while the run reported ok.
failures++;
console.log(`FAIL ${label}: rendered data-theme=${measured.theme}, not ${theme}`);
} else if (measured.chars < 120) {
// A page that throws inside its own body still measures zero overflow.
failures++;
console.log(`FAIL ${label}: rendered only ${measured.chars} characters — the page is blank`);
} else if (measured.overflow !== 0) {
failures++;
console.log(`FAIL ${label}: overflows by ${measured.overflow}px — ${measured.wide.join(', ')}`);
} else if (problems.length) {