Rebuild Piggy's interface, and give the demo book a business to describe

Piggy answered in raw markdown, threw away every tool result it streamed,
and fought the reader's scroll on every token. The three surfaces that
made it worth having — what it read, how it reasoned, what it cost — were
all on the wire and none of them reached the screen.

The transcript is now composed of five parts under components/piggy:
answers render through streamdown, the container sticks to the bottom
without pinning the reader there, tool steps say what they read and link
to the record, and each turn carries its model and token count. Three
lifecycle bugs went with them: Stop left a permanent spinner, a truncated
stream was indistinguishable from thinking, and a failed send destroyed
the message it failed to send.

Underneath, the inference path grew timeouts, jittered retries on 429 and
5xx, tolerance of the malformed frames a 30B model emits, and an
agent_runs row per turn so chat spend is observable. The system prompt now
states that a field ending in Cents is cents — without it nemotron renders
costPerGpuHourCents: 189 as "$189 per GPU-hour", which is a 100x error on
the most scrutinised number in the room.

The demo book was arithmetically incoherent: every deal's value
contradicted its own allocation revenue by up to 3.6x, nothing had ever
closed, no customer had any paper, and the marketplace was empty. Deal
value is now derived from the allocation, the book clears 5.3% across five
blocks with one deliberately underwater, and the renewal, compliance and
agent-provenance machinery finally has rows to act on. A --clear that
deleted every obligation, SLA term and capacity request in the database
regardless of origin is scoped to the demo's own ids.

Around that: accounts have a detail page, ⌘K searches the book, Settings
can mint the API keys it always claimed to, and deploy.sh actually ships
the agent instead of silently skipping its compose profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

apps/piggy/test/chat.test.ts

@@ -26,6 +26,84 @@ function eventStream(events: unknown[]): Response {
   );
 }
 
+/** Frames verbatim, so a test can send something no `JSON.stringify` would. */
+function rawEventStream(frames: string[]): Response {
+  const encoder = new TextEncoder();
+  return new Response(
+    new ReadableStream({
+      start(controller) {
+        for (const frame of frames) controller.enqueue(encoder.encode(`${frame}\n\n`));
+        controller.close();
+      },
+    }),
+    { headers: { 'content-type': 'text/event-stream' } },
+  );
+}
+
+/** One frame, then silence: the shape of an upstream that has stopped talking. */
+function stallingEventStream(frame: string): Response {
+  const encoder = new TextEncoder();
+  return new Response(
+    new ReadableStream({
+      start(controller) {
+        controller.enqueue(encoder.encode(`${frame}\n\n`));
+        // Never closed, and no pull, so the next read waits for ever.
+      },
+    }),
+    { headers: { 'content-type': 'text/event-stream' } },
+  );
+}
+
+/** Frames spaced in time, to prove a long answer is not a stalled one. */
+function pacedEventStream(frames: string[], gapMs: number): Response {
+  const encoder = new TextEncoder();
+  const remaining = [...frames];
+  return new Response(
+    new ReadableStream({
+      async pull(controller) {
+        const frame = remaining.shift();
+        if (frame === undefined) {
+          controller.close();
+          return;
+        }
+        await new Promise((resolve) => setTimeout(resolve, gapMs));
+        controller.enqueue(encoder.encode(`${frame}\n\n`));
+      },
+    }),
+    { headers: { 'content-type': 'text/event-stream' } },
+  );
+}
+
+function jsonResponse(status: number, headers: Record<string, string> = {}): Response {
+  return new Response(JSON.stringify({ error: { message: `upstream said ${status}` } }), {
+    status,
+    headers: { 'content-type': 'application/json', ...headers },
+  });
+}
+
+const finalAnswer = { choices: [{ delta: { content: 'Idle is $12,000.' }, finish_reason: 'stop' }] };
+
+function contentOf(events: PiggyChatEvent[]): string {
+  return events
+    .filter((event): event is Extract<PiggyChatEvent, { type: 'content_delta' }> =>
+      event.type === 'content_delta',
+    )
+    .map((event) => event.delta)
+    .join('');
+}
+
+function readTool(onCall?: () => void) {
+  return defineTool({
+    name: 'pig_get_idle_capacity',
+    description: 'Read idle capacity.',
+    inputSchema: z.object({}).strict(),
+    execute: async () => {
+      onCall?.();
+      return { totalIdleCostCents: 1_200_000 };
+    },
+  });
+}
+
 test('interactive streaming keeps reasoning, tools and final content as separate events', async () => {
   const bodies: Record<string, unknown>[] = [];
   let call = 0;
@@ -179,3 +257,270 @@ test('ambient coding tools are rejected before inference', async () => {
   );
   assert.equal(fetched, false);
 });
+
+test('the system prompt states the units rule and the margin definitions', async () => {
+  let systemPrompt = '';
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    fetchImpl: async (_input, init) => {
+      const body = JSON.parse(String(init?.body)) as { messages: { role: string; content: string }[] };
+      systemPrompt = body.messages.find((message) => message.role === 'system')?.content ?? '';
+      return eventStream([finalAnswer]);
+    },
+  });
+
+  await collect(provider.run({ message: 'What is idle costing us?', tools: [readTool()] }));
+
+  // The whole point: 189 spoken as "$189 per GPU-hour" is a hundredfold error
+  // on the number everyone in the room is watching.
+  assert.match(systemPrompt, /ends in Cents is an integer number of US cents/i);
+  assert.match(systemPrompt, /costPerGpuHourCents: 189 is \$1\.89 per GPU-hour/);
+  assert.match(systemPrompt, /ends in Pct, and utilisation, is a share between 0 and 1/);
+  // Margin against sold hours only would report a losing block as healthy.
+  assert.match(systemPrompt, /revenue minus the FULL cost of the commitment/);
+  assert.match(systemPrompt, /REMAINING unsold hours must fetch/);
+  assert.match(systemPrompt, /null break-even means the block is fully allocated/);
+});
+
+test('an unparseable frame is discarded rather than ending the turn', async () => {
+  const warnings: string[] = [];
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    onWarning: (message) => warnings.push(message),
+    fetchImpl: async () =>
+      rawEventStream([
+        'data: {"choices":[{"delta":{"content":"Idle is "}}]}',
+        // Truncated mid-object, and then a frame that is JSON but not a chunk.
+        'data: {"choices":[{"delta":',
+        'data: {"choices":"not an array"}',
+        'data: {"choices":[{"delta":{"content":"$12,000."},"finish_reason":"stop"}]}',
+        'data: [DONE]',
+      ]),
+  });
+
+  const events = await collect(provider.run({ message: 'What is idle?', tools: [readTool()] }));
+
+  assert.deepEqual(events.map((event) => event.type), [
+    'meta',
+    'content_delta',
+    'content_delta',
+    'done',
+  ]);
+  assert.equal(contentOf(events), 'Idle is $12,000.');
+  assert.equal(warnings.length, 2);
+});
+
+test('a tool call that arrived without an id is handed back to the model, not thrown', async () => {
+  const bodies: Record<string, unknown>[] = [];
+  let executed = false;
+  let call = 0;
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    onWarning: () => {},
+    fetchImpl: async (_input, init) => {
+      bodies.push(JSON.parse(String(init?.body)) as Record<string, unknown>);
+      call += 1;
+      return call === 1
+        ? eventStream([
+            {
+              choices: [{
+                delta: {
+                  tool_calls: [{
+                    index: 0,
+                    function: { name: 'pig_get_idle_capacity', arguments: '{}' },
+                  }],
+                },
+                finish_reason: 'tool_calls',
+              }],
+            },
+          ])
+        : eventStream([finalAnswer]);
+    },
+  });
+
+  const events = await collect(
+    provider.run({ message: 'What is idle?', tools: [readTool(() => { executed = true; })] }),
+  );
+
+  assert.deepEqual(events.map((event) => event.type), [
+    'meta',
+    'tool_call',
+    'tool_result',
+    'content_delta',
+    'done',
+  ]);
+  const result = events[2];
+  assert.equal(result?.type === 'tool_result' && result.ok, false);
+  assert.match(
+    (result?.type === 'tool_result' && result.error) || '',
+    /arrived without its id/,
+  );
+  // A call with no id must not run: the model never asked for a specific
+  // invocation, and the reply would have nothing to attach to.
+  assert.equal(executed, false);
+
+  // The correction only reaches the model if the tool reply matches the
+  // synthesised id on the assistant message that preceded it.
+  const messages = bodies[1]?.messages as {
+    role: string;
+    tool_calls?: { id: string }[];
+    tool_call_id?: string;
+    content?: string;
+  }[];
+  const assistant = messages.find((message) => message.role === 'assistant');
+  const toolReply = messages.find((message) => message.role === 'tool');
+  assert.equal(toolReply?.tool_call_id, assistant?.tool_calls?.[0]?.id);
+  assert.match(toolReply?.content ?? '', /arrived without its id/);
+});
+
+test('tool arguments that are not valid JSON come back as a tool result the model can fix', async () => {
+  let executed = false;
+  let call = 0;
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    onWarning: () => {},
+    fetchImpl: async () => {
+      call += 1;
+      return call === 1
+        ? eventStream([
+            {
+              choices: [{
+                delta: {
+                  tool_calls: [{
+                    index: 0,
+                    id: 'call_1',
+                    function: { name: 'pig_get_idle_capacity', arguments: '{"unclosed": ' },
+                  }],
+                },
+                finish_reason: 'tool_calls',
+              }],
+            },
+          ])
+        : eventStream([finalAnswer]);
+    },
+  });
+
+  const events = await collect(
+    provider.run({ message: 'What is idle?', tools: [readTool(() => { executed = true; })] }),
+  );
+
+  const result = events[2];
+  assert.equal(result?.type, 'tool_result');
+  assert.match(
+    (result?.type === 'tool_result' && result.error) || '',
+    /were not valid JSON/,
+  );
+  assert.equal(executed, false);
+  // The turn continued, which is the difference between a tool that failed
+  // once and a conversation that stopped.
+  assert.equal(events.at(-1)?.type, 'done');
+  assert.equal(call, 2);
+});
+
+test('a rate-limited turn is retried, honouring the Retry-After it was given', async () => {
+  const retries: { attempt: number; delayMs: number; reason: string }[] = [];
+  let calls = 0;
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    maxBackoffMs: 5,
+    onRetry: (info) => retries.push(info),
+    fetchImpl: async () => {
+      calls += 1;
+      return calls === 1 ? jsonResponse(429, { 'retry-after': '0' }) : eventStream([finalAnswer]);
+    },
+  });
+
+  const events = await collect(provider.run({ message: 'What is idle?', tools: [readTool()] }));
+
+  assert.equal(calls, 2);
+  assert.deepEqual(retries.map((retry) => retry.delayMs), [0]);
+  assert.match(retries[0]?.reason ?? '', /429/);
+  assert.deepEqual(events.map((event) => event.type), ['meta', 'content_delta', 'done']);
+});
+
+test('a 5xx exhausts the attempt budget; a 4xx spends exactly one attempt', async () => {
+  let serverErrors = 0;
+  const failing = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    maxAttempts: 3,
+    maxBackoffMs: 1,
+    fetchImpl: async () => {
+      serverErrors += 1;
+      return jsonResponse(500);
+    },
+  });
+  await assert.rejects(
+    collect(failing.run({ message: 'What is idle?', tools: [readTool()] })),
+    /Piggy inference 500/,
+  );
+  assert.equal(serverErrors, 3);
+
+  let badRequests = 0;
+  const rejected = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    maxAttempts: 3,
+    maxBackoffMs: 1,
+    fetchImpl: async () => {
+      badRequests += 1;
+      return jsonResponse(400);
+    },
+  });
+  await assert.rejects(
+    collect(rejected.run({ message: 'What is idle?', tools: [readTool()] })),
+    /Piggy inference 400/,
+  );
+  // A malformed request fails identically however often it is sent, and every
+  // repeat spends credit to learn nothing.
+  assert.equal(badRequests, 1);
+});
+
+test('an upstream that never sends headers is abandoned on the attempt deadline', async () => {
+  const provider = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    maxAttempts: 1,
+    timeoutMs: 25,
+    fetchImpl: (_input, init) =>
+      new Promise((_resolve, reject) => {
+        // Only the deadline can end this, which is also the proof that the
+        // deadline reaches the request at all.
+        init?.signal?.addEventListener('abort', () => reject(init.signal?.reason));
+      }),
+  });
+
+  await assert.rejects(
+    collect(provider.run({ message: 'What is idle?', tools: [readTool()] })),
+    /did not respond within 25ms/,
+  );
+});
+
+test('a stream that goes quiet is abandoned, a slow one is not', async () => {
+  const stalled = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    streamIdleTimeoutMs: 25,
+    fetchImpl: async () => stallingEventStream('data: {"choices":[{"delta":{"content":"Idle "}}]}'),
+  });
+  await assert.rejects(
+    collect(stalled.run({ message: 'What is idle?', tools: [readTool()] })),
+    /stalled for 25ms/,
+  );
+
+  // Six times the gap in total, and never a gap longer than the deadline: a
+  // flat deadline would have killed this answer for being long.
+  const slow = new PrimeOpenAIChatProvider({
+    apiKey: 'test',
+    streamIdleTimeoutMs: 60,
+    fetchImpl: async () =>
+      pacedEventStream(
+        [
+          ...['Idle ', 'is ', '$12,000 ', 'across ', 'four ', 'blocks.'].map(
+            (word) => `data: ${JSON.stringify({ choices: [{ delta: { content: word } }] })}`,
+          ),
+          'data: [DONE]',
+        ],
+        15,
+      ),
+  });
+  const events = await collect(slow.run({ message: 'What is idle?', tools: [readTool()] }));
+  assert.equal(contentOf(events), 'Idle is $12,000 across four blocks.');
+  assert.equal(events.at(-1)?.type, 'done');
+});