Rebuild Piggy's interface, and give the demo book a business to describe

Piggy answered in raw markdown, threw away every tool result it streamed, and fought the reader's scroll on every token. The three surfaces that made it worth having — what it read, how it reasoned, what it cost — were all on the wire and none of them reached the screen. The transcript is now composed of five parts under components/piggy: answers render through streamdown, the container sticks to the bottom without pinning the reader there, tool steps say what they read and link to the record, and each turn carries its model and token count. Three lifecycle bugs went with them: Stop left a permanent spinner, a truncated stream was indistinguishable from thinking, and a failed send destroyed the message it failed to send. Underneath, the inference path grew timeouts, jittered retries on 429 and 5xx, tolerance of the malformed frames a 30B model emits, and an agent_runs row per turn so chat spend is observable. The system prompt now states that a field ending in Cents is cents — without it nemotron renders costPerGpuHourCents: 189 as "$189 per GPU-hour", which is a 100x error on the most scrutinised number in the room. The demo book was arithmetically incoherent: every deal's value contradicted its own allocation revenue by up to 3.6x, nothing had ever closed, no customer had any paper, and the marketplace was empty. Deal value is now derived from the allocation, the book clears 5.3% across five blocks with one deliberately underwater, and the renewal, compliance and agent-provenance machinery finally has rows to act on. A --clear that deleted every obligation, SLA term and capacity request in the database regardless of origin is scoped to the demo's own ids. Around that: accounts have a detail page, ⌘K searches the book, Settings can mint the API keys it always claimed to, and deploy.sh actually ships the agent instead of silently skipping its compose profile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 00:33:41 -07:00
parent 76e3caa1cb
commit 99d165b5e5
81 changed files with 21780 additions and 2250 deletions
@@ -0,0 +1,392 @@
+/**
+ * The Learn library: the illustrative demo rows, and the PIG-hosted real ones.
+ *
+ * They sit in one module because they write to one table and share the
+ * idempotency key, but they are not the same kind of data: the first set is
+ * invented and prefixed, the second is genuine product footage and survives
+ * `--clear`.
+ */
+import { LEARN_MEDIA_PATH_PREFIX, isLearnMediaFilename, learnMediaContentType } from '@pig/core';
+import { eq } from 'drizzle-orm';
+import { readdir } from 'node:fs/promises';
+import { resolve } from 'node:path';
+import { learnResources, users } from '../../schema/index';
+import type { DemoContext } from './index';
+
+/**
+ * Every id below is a REAL public recording on the Cap instance at
+ * video.karti.ai, and every duration is the MEASURED length of the file that
+ * embed plays. Both were re-checked against the instance — `/s/<id>` and
+ * `/embed/<id>` answer 200, and the recordings are 60.4s and 6.2s — because
+ * this list previously stated neither truthfully.
+ *
+ * It used to carry four rows, two of them sharing the id `0n6n9p83efnxbs2`
+ * under different titles and different stated lengths. That id is not a
+ * recording at all — it 404s, and the only other place it appears in this
+ * repository is as the fixture string in `apps/api/test/learn.test.ts`, which
+ * is almost certainly where it was copied from. So two cards each promised a
+ * quarter of an hour of teaching and played nothing, on the one page that is
+ * deliberately shown to outsiders. A card that lies about its own content is
+ * worse than a track with one card in it, and there are exactly two public
+ * recordings on that instance — so there are exactly two rows here, one per
+ * concept track.
+ *
+ * The titles remain illustrative and prefixed, which is the standing bargain
+ * for demo rows: the concepts they name are the ones this business actually
+ * teaches, and the footage behind them is whatever genuinely exists. The
+ * bargain only holds while the stated LENGTH is true, since that is the one
+ * claim a viewer can check before pressing play.
+ *
+ * PLATFORM rows are not seeded here. The five real recordings in
+ * HOSTED_LEARN_MANIFEST cover that track, and illustrative Cap rows beneath
+ * genuine ones made the page read as half-placeholder to the exact audience it
+ * is meant to convince.
+ *
+ * The concept rows stay: `supply` and `demand` have no purpose-shot recordings
+ * yet, and an empty track hides the shape of the page. They are
+ * `members`-visible, which the CHECK constraint enforces anyway — only
+ * `platform` may be `code`.
+ */
+export async function seedLearn(
+  context: DemoContext,
+  ownerUserId: string | null,
+): Promise<{ total: number; added: number }> {
+  const { db, prefix } = context;
+
+  const LEARN_RESOURCES = [
+    {
+      track: 'supply' as const,
+      title: `${prefix}How neocloud capacity is actually priced`,
+      summary: 'Reserved versus on-demand, commitment length, and where the spread comes from.',
+      externalId: '1rqq9rk4dpp71fd',
+      visibility: 'members' as const,
+      // 60.4s on the wire, rounded down: a duration that overstates by a
+      // second is the same class of claim as one that overstates by minutes.
+      durationSeconds: 60,
+      sortOrder: 10,
+    },
+    {
+      track: 'demand' as const,
+      title: `${prefix}What a hold takes off the board`,
+      summary:
+        'A hold reserves hours nobody else can be quoted, and it is not revenue until it converts.',
+      externalId: 'sjqqvthbfma27bm',
+      visibility: 'members' as const,
+      durationSeconds: 6,
+      sortOrder: 10,
+    },
+  ];
+
+  /*
+   * The unique key is (track, provider, external_id), so the SAME recording on
+   * TWO tracks inserts perfectly happily — which is how one id came to sit
+   * behind two different titles and two different stated durations for as long
+   * as it did. The database cannot catch that; this can. Thrown before any
+   * insert, and loudly, because it is a typo in a literal rather than a
+   * condition of the environment: there is nothing for an operator to fix at
+   * run time and nothing worth continuing past.
+   */
+  const ids = LEARN_RESOURCES.map((resource) => resource.externalId);
+  if (new Set(ids).size !== ids.length) {
+    throw new Error(
+      'Two demo learn resources share an external id — one of them would be a lie about its own content.',
+    );
+  }
+
+  let learnAdded = 0;
+  for (const resource of LEARN_RESOURCES) {
+    // Idempotent on the unique key rather than an existence check, which is
+    // the whole reason that constraint exists: onConflictDoNothing without one
+    // is a silent no-op and has duplicated seed data here twice before.
+    const inserted = await db
+      .insert(learnResources)
+      .values({
+        track: resource.track,
+        title: resource.title,
+        summary: resource.summary,
+        url: `https://video.karti.ai/s/${resource.externalId}`,
+        provider: 'cap',
+        externalId: resource.externalId,
+        visibility: resource.visibility,
+        durationSeconds: resource.durationSeconds,
+        sortOrder: resource.sortOrder,
+        addedByUserId: ownerUserId,
+      })
+      .onConflictDoNothing({
+        target: [learnResources.track, learnResources.provider, learnResources.externalId],
+      })
+      .returning({ id: learnResources.id });
+    if (inserted.length) learnAdded += 1;
+  }
+
+  return { total: LEARN_RESOURCES.length, added: learnAdded };
+}
+
+// ------------------------------------------------------- PIG-hosted learn
+//
+// Real videos, served by PIG itself from PIG_MEDIA_DIR — not demo data. They
+// carry no `DEMO — ` prefix precisely because they are genuine product
+// walkthroughs, which also means `--clear` leaves them alone; `--clear-hosted`
+// is their own switch.
+//
+// **A row is written only when its file is actually on disk.** A learn row
+// whose media 404s is worse than a missing row: the card renders, the play
+// button does nothing, and the feature reads as broken. So the manifest below
+// declares the curriculum, and the seed inserts the entries it can find.
+// Running it before the videos are generated is a no-op with a printed list,
+// and running it again afterwards fills them in.
+//
+// That choice was re-examined, because the audience for the absence is not an
+// operator: the anonymous share-code view of /learn shows this track and
+// nothing else, so a missing file is a stranger's first impression of the
+// product. A "coming soon" ROW was rejected. It would need a filename to point
+// at, the card would carry a play button, and pressing it would fail — which
+// is the one outcome worse than an empty section, and the same reason the
+// access hero draws redacted bars rather than invented thumbnails. The absence
+// is handled where it belongs instead: `apps/web/src/pages/Learn.tsx` gives a
+// code-holder with nothing published a finished panel that says so and offers
+// a way on, rather than the admin empty state.
+//
+// `media/` is gitignored — hundreds of megabytes of rendered MP4 are a release
+// artefact, not source — so a fresh clone and every git worktree start without
+// it, and that is the ordinary case rather than a fault. Point PIG_MEDIA_DIR
+// at a directory holding the renders (the API route reads the same variable)
+// and the five rows appear.
+//
+// **The filename is discovered, not written down.** Files are
+// content-addressed — `<slug>.<hash>.mp4` — so the hash changes every time a
+// video is re-rendered, and a manifest carrying the hash would be a file that
+// has to be edited in lockstep with a render. Instead the slug is the stable
+// identity and the directory supplies the rest. Idempotency then rests on the
+// unique key (track, provider, external_id) exactly as the DEMO rows do.
+//
+// A re-render produces a NEW hash and therefore a new row; the old row keeps
+// pointing at a file that is no longer there. That is reported rather than
+// resolved automatically, because deleting rows on the strength of a missing
+// file would empty the curriculum the first time someone ran this with the
+// media directory unmounted.
+
+interface HostedLearnEntry {
+  /** Stable identity. Also the filename stem: `<slug>.<hash>.<ext>`. */
+  slug: string;
+  title: string;
+  summary: string;
+  track: 'supply' | 'demand' | 'platform';
+  visibility: 'members' | 'code';
+  /**
+   * Measured from the render, to the nearest second — not estimated from the
+   * script. Every one below was re-checked against the file it names, because
+   * a duration is the only claim a card makes that a viewer can verify before
+   * pressing play.
+   */
+  durationSeconds: number;
+  sortOrder: number;
+}
+
+export const HOSTED_LEARN_MANIFEST: readonly HostedLearnEntry[] = [
+  {
+    slug: 'overview-and-margin',
+    title: 'Overview and the margin question',
+    summary:
+      'Which contracted capacity is sold, at what margin, and what is idle right now — and why cost is charged against the full commitment.',
+    track: 'platform',
+    visibility: 'code',
+    durationSeconds: 29,
+    sortOrder: 1,
+  },
+  {
+    slug: 'quarterly-calendar',
+    title: 'The quarterly calendar',
+    summary:
+      'What closes, what renews, what expires and when capacity lands — with export authorisations and compliance artefacts at the top.',
+    track: 'platform',
+    visibility: 'code',
+    durationSeconds: 30,
+    sortOrder: 2,
+  },
+  {
+    slug: 'capacity-to-allocations',
+    title: 'Capacity to allocations',
+    summary:
+      'Joining a commitment you bought to a deal you sold — the availability book, the matcher, and the allocation the ledger is built on.',
+    track: 'platform',
+    visibility: 'code',
+    durationSeconds: 26,
+    sortOrder: 3,
+  },
+  {
+    slug: 'importing-your-book',
+    title: 'Importing your book',
+    summary:
+      'Getting off the spreadsheet — CSV, Excel, Notion or a bounded Google Sheets range, with a dry run you review before anything is written.',
+    track: 'platform',
+    visibility: 'code',
+    durationSeconds: 28,
+    sortOrder: 4,
+  },
+  {
+    slug: 'piggy-and-its-boundary',
+    title: 'Piggy, and what it will not do',
+    summary:
+      'The docked agent reads through scoped, page-specific PIG tools — and has no shell, no filesystem, and no ability to write CRM records.',
+    track: 'platform',
+    visibility: 'code',
+    durationSeconds: 28,
+    sortOrder: 5,
+  },
+];
+
+/**
+ * Where the served files live. The same variable the API route reads.
+ *
+ * The default is resolved from this file's location, not from `process.cwd()`,
+ * because pnpm runs the seed with the working directory at `packages/db` while
+ * the server runs at the repository root — so a relative default would mean
+ * two different directories, and the seed would write rows for files the API
+ * cannot find.
+ *
+ * Five levels up, because this file sits one deeper than the seed it was split
+ * out of: `packages/db/src/seed/demo` → repository root.
+ */
+function mediaDirectory(): string {
+  const configured = process.env.PIG_MEDIA_DIR?.trim();
+  if (configured && configured.length > 0) return resolve(configured);
+  return resolve(import.meta.dirname, '../../../../../media');
+}
+
+/**
+ * Find the one VIDEO belonging to a slug.
+ *
+ * Two videos for one slug is an error rather than a choice: picking the newest
+ * would silently publish whichever render happened to finish last, and the
+ * operator has an old file to delete.
+ *
+ * Video extensions only, and that is the whole point of the second predicate.
+ * A poster is `<slug>.<hash>.jpg` — deliberately the video's own stem, so it
+ * cannot go stale against the clip it shows (`learnPosterFilename`) — and it
+ * sits in this same directory. Matching on the slug alone therefore found two
+ * files for every complete entry and declared each of them an ambiguous stale
+ * render, so a directory containing five finished walkthroughs AND their
+ * posters seeded exactly nothing, and the share-code page it feeds rendered
+ * empty on the one box that had the media.
+ */
+function mediaFileFor(
+  slug: string,
+  filenames: readonly string[],
+): { filename: string | null; ambiguous: readonly string[] } {
+  const matches = filenames.filter(
+    (name) =>
+      isLearnMediaFilename(name) &&
+      name.startsWith(`${slug}.`) &&
+      learnMediaContentType(name)?.startsWith('video/'),
+  );
+  /*
+   * Reported and skipped, not thrown.
+   *
+   * This runs from the middle of seedDemo(), so throwing on a duplicate took
+   * out every later section — facts, activities, the lot — and left the
+   * operator working out why `pnpm db:demo` died on two MP4s sharing a slug.
+   * A stale render is a condition of one directory entry; its blast radius
+   * should be that entry. Missing files are already handled this way.
+   */
+  if (matches.length > 1) return { filename: null, ambiguous: matches };
+  return { filename: matches[0] ?? null, ambiguous: [] };
+}
+
+export async function seedHostedLearn(context: DemoContext): Promise<{
+  present: number;
+  added: number;
+  missing: number;
+}> {
+  const { db } = context;
+
+  const directory = mediaDirectory();
+  let filenames: string[] = [];
+  try {
+    filenames = await readdir(directory);
+  } catch {
+    // No directory is the normal state of a fresh checkout, not a failure.
+    console.log(
+      `  (no media directory at ${directory} — skipping PIG-hosted learn videos; ` +
+        'set PIG_MEDIA_DIR if the renders live elsewhere)',
+    );
+    return { present: 0, added: 0, missing: HOSTED_LEARN_MANIFEST.length };
+  }
+
+  const [owner] = await db.select({ id: users.id }).from(users).limit(1);
+  let present = 0;
+  let added = 0;
+  const absent: string[] = [];
+
+  for (const entry of HOSTED_LEARN_MANIFEST) {
+    const { filename, ambiguous } = mediaFileFor(entry.slug, filenames);
+    if (ambiguous.length > 0) {
+      console.warn(
+        `  ! ${entry.slug}: ${ambiguous.length} files match (${ambiguous.join(', ')}) — ` +
+          'content-addressed names mean one is a stale render. Delete it and re-run. Skipped.',
+      );
+      absent.push(entry.slug);
+      continue;
+    }
+    if (!filename) {
+      absent.push(entry.slug);
+      continue;
+    }
+    present += 1;
+
+    const inserted = await db
+      .insert(learnResources)
+      .values({
+        track: entry.track,
+        title: entry.title,
+        summary: entry.summary,
+        // The same path the resolver builds, from the same constant — so a
+        // seeded row and a row created through the API are indistinguishable.
+        url: `${LEARN_MEDIA_PATH_PREFIX}${filename}`,
+        provider: 'pig',
+        externalId: filename,
+        visibility: entry.visibility,
+        durationSeconds: entry.durationSeconds,
+        sortOrder: entry.sortOrder,
+        addedByUserId: owner?.id ?? null,
+      })
+      .onConflictDoNothing({
+        target: [learnResources.track, learnResources.provider, learnResources.externalId],
+      })
+      .returning({ id: learnResources.id });
+    if (inserted.length) added += 1;
+  }
+
+  if (absent.length) {
+    console.log(
+      `  (no file yet in ${directory} for: ${absent.join(', ')} — ` +
+        'drop <slug>.<hash>.mp4 there, or point PIG_MEDIA_DIR at the renders, and run this again)',
+    );
+  }
+
+  // Rows whose file has gone: reported, never deleted. See the note above.
+  const orphans = (
+    await db
+      .select({ externalId: learnResources.externalId, title: learnResources.title })
+      .from(learnResources)
+      .where(eq(learnResources.provider, 'pig'))
+  ).filter((row) => !filenames.includes(row.externalId));
+  for (const orphan of orphans) {
+    console.log(`  ! "${orphan.title}" points at ${orphan.externalId}, which is not on disk`);
+  }
+
+  return { present, added, missing: absent.length };
+}
+
+/**
+ * `--clear-hosted`. Kept beside the seed it undoes rather than in `clear.ts`,
+ * because it is the opposite of a different command: `--clear` removes the
+ * invented book and must leave these genuine rows exactly where they are.
+ */
+export async function clearHostedLearn(context: DemoContext): Promise<void> {
+  const removed = await context.db
+    .delete(learnResources)
+    .where(eq(learnResources.provider, 'pig'))
+    .returning({ id: learnResources.id });
+  console.log(`Removed ${removed.length} PIG-hosted learn resource(s). Files on disk are untouched.`);
+}