Files
pig/apps/api/src/lib/media.ts
T
claude 99d165b5e5
CI / verify (push) Successful in 4m57s
CI / publish (push) Has been skipped
Rebuild Piggy's interface, and give the demo book a business to describe
Piggy answered in raw markdown, threw away every tool result it streamed,
and fought the reader's scroll on every token. The three surfaces that
made it worth having — what it read, how it reasoned, what it cost — were
all on the wire and none of them reached the screen.

The transcript is now composed of five parts under components/piggy:
answers render through streamdown, the container sticks to the bottom
without pinning the reader there, tool steps say what they read and link
to the record, and each turn carries its model and token count. Three
lifecycle bugs went with them: Stop left a permanent spinner, a truncated
stream was indistinguishable from thinking, and a failed send destroyed
the message it failed to send.

Underneath, the inference path grew timeouts, jittered retries on 429 and
5xx, tolerance of the malformed frames a 30B model emits, and an
agent_runs row per turn so chat spend is observable. The system prompt now
states that a field ending in Cents is cents — without it nemotron renders
costPerGpuHourCents: 189 as "$189 per GPU-hour", which is a 100x error on
the most scrutinised number in the room.

The demo book was arithmetically incoherent: every deal's value
contradicted its own allocation revenue by up to 3.6x, nothing had ever
closed, no customer had any paper, and the marketplace was empty. Deal
value is now derived from the allocation, the book clears 5.3% across five
blocks with one deliberately underwater, and the renewal, compliance and
agent-provenance machinery finally has rows to act on. A --clear that
deleted every obligation, SLA term and capacity request in the database
regardless of origin is scoped to the demo's own ids.

Around that: accounts have a detail page, ⌘K searches the book, Settings
can mint the API keys it always claimed to, and deploy.sh actually ships
the agent instead of silently skipping its compose profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 00:34:18 -07:00

232 lines
11 KiB
TypeScript

/**
* Serving the Learn videos PIG hosts itself.
*
* ## The trade, stated plainly
*
* **The files are unauthenticated. The listing is not.** `/api/learn` and
* `/api/learn/public` decide who learns that a video exists, what it is called
* and which track it belongs to; this route hands the bytes to anyone who can
* name the file. That is the same shape every video platform has — a gated
* manifest in front of segments on a public CDN — and it is the shape a
* `<video>` element actually wants, because a media element re-requests ranges
* on every seek and does not carry a bearer token while doing it.
*
* What it costs: **a URL, once shared, is a permanent public link to that
* video**. Someone who has been given the share code can copy the `src` out of
* the page and post it, and revoking the Learn access code will not close it.
* The only remedies are renaming the file (a new hash) or deleting it. We are
* taking that deliberately, because these are product demos meant to be
* shareable with the code — the material that must never leak is the concept
* tracks, and those are gated by the listing, which is where the boundary
* genuinely is.
*
* The mitigation is the filename. Names are **content-addressed** — a hash in
* the middle — so a URL is unguessable and enumerating the directory over HTTP
* is not possible: there is no index, and a wrong guess is a flat 404. That
* makes the exposure "whoever was given the link" rather than "the internet",
* which is exactly what an unlisted video is.
*
* ## Ranges are not optional
*
* A `<video>` that cannot be range-requested cannot be scrubbed: the browser
* asks for `bytes=…` when the user drags the scrubber, and a server that
* answers 200 with the whole file makes seeking either impossible or a
* re-download. So `Accept-Ranges: bytes` is advertised and a single range is
* honoured with a 206. This is also why the route streams from a file
* descriptor rather than reading the file into memory: these are hundreds of
* megabytes and several viewers may be seeking at once.
*
* ## Why the path cannot escape the directory
*
* The filename is validated by the same allowlist that decides whether a row
* may exist at all (`isLearnMediaFilename` in `@pig/core`), so the pattern
* that admits a video source and the pattern that admits a file read are one
* pattern rather than two that drift. It permits no slash and no `..`. The
* resolved path is then checked to be inside the root anyway, because a
* defence that rests on a single regular expression rests on nobody ever
* editing that regular expression.
*/
import { Hono } from 'hono';
import { createReadStream } from 'node:fs';
import { realpath, stat } from 'node:fs/promises';
import { dirname, join, resolve, sep } from 'node:path';
import { Readable } from 'node:stream';
import { fileURLToPath } from 'node:url';
import { isLearnMediaFilename, learnMediaContentType, LEARN_MEDIA_PATH_PREFIX } from '@pig/core';
/**
* Where the files live.
*
* Read from the environment here rather than through `loadConfig` because the
* media root is an operational detail of one route, and a missing value is not
* a reason to refuse to boot — an install with no videos should serve 404s and
* work in every other respect.
*
* A relative path — including the default — is resolved against the REPOSITORY
* ROOT, not the working directory. It used to be the working directory, and
* that was wrong in the one case it had to be right: `pnpm -F @pig/api dev`
* runs with the cwd set to `apps/api`, so the documented `PIG_MEDIA_DIR=./media`
* resolved to `apps/api/media`, which does not exist, and every Learn video
* 404'd while the poster fell back to a placeholder that looks deliberate. The
* container copies the tree to `/app`, so the root is `/app` there and the
* default lands on `/app/media` — exactly where docker-compose bind-mounts the
* host directory read-only, and what it sets `PIG_MEDIA_DIR` to anyway.
*/
export const LEARN_MEDIA_DIR_ENV = 'PIG_MEDIA_DIR';
const DEFAULT_MEDIA_DIR = './media';
// apps/api/src/lib/media.ts — four levels up is the repository root.
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..', '..', '..');
export function learnMediaRoot(env: NodeJS.ProcessEnv = process.env): string {
const configured = env[LEARN_MEDIA_DIR_ENV]?.trim();
// `resolve` ignores the base when the second argument is already absolute,
// so an absolute PIG_MEDIA_DIR is honoured untouched.
return resolve(REPO_ROOT, configured && configured.length > 0 ? configured : DEFAULT_MEDIA_DIR);
}
/** The mount path, exported so `app.ts` and the resolver cannot disagree. */
export const LEARN_MEDIA_ROUTE = `${LEARN_MEDIA_PATH_PREFIX}:filename`;
interface ParsedRange {
start: number;
end: number;
}
/**
* Parse a single byte range against a known size, or say what to do instead.
*
* `null` means "serve the whole thing with 200" — the correct answer for no
* header, a syntactically odd one, or a multi-range request, all of which a
* server is permitted to ignore. `'unsatisfiable'` is the one case that must
* NOT become a 200: a range starting past the end is a client with a stale
* idea of the file, and answering it with the whole file would splice the
* beginning of the video into the middle of its buffer.
*/
export function parseByteRange(header: string | undefined, size: number): ParsedRange | null | 'unsatisfiable' {
if (!header) return null;
const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
if (!match) return null;
const [, rawStart, rawEnd] = match;
if (rawStart === '' && rawEnd === '') return null;
// A suffix range — `bytes=-500`, the last 500 bytes. Browsers use it to read
// the MP4 moov atom when it sits at the end of the file, so a player that
// cannot start at all is often this branch missing.
if (rawStart === '') {
const suffix = Number(rawEnd);
if (!Number.isSafeInteger(suffix) || suffix <= 0) return 'unsatisfiable';
if (size === 0) return 'unsatisfiable';
return { start: Math.max(0, size - suffix), end: size - 1 };
}
const start = Number(rawStart);
if (!Number.isSafeInteger(start) || start < 0) return 'unsatisfiable';
if (start >= size) return 'unsatisfiable';
const end = rawEnd === '' ? size - 1 : Math.min(Number(rawEnd), size - 1);
if (!Number.isSafeInteger(end) || end < start) return 'unsatisfiable';
return { start, end };
}
export function createMediaRoutes(options: { root?: string } = {}) {
const app = new Hono();
const root = options.root ? resolve(options.root) : learnMediaRoot();
// GET and HEAD both, because a player probes with HEAD before it commits to
// downloading, and an unrouted HEAD would 404 a file that is plainly there.
app.on(['GET', 'HEAD'], LEARN_MEDIA_ROUTE, async (c) => {
const filename = c.req.param('filename');
if (!filename || !isLearnMediaFilename(filename)) return c.notFound();
const contentType = learnMediaContentType(filename);
if (!contentType) return c.notFound();
const path = join(root, filename);
// Belt and braces over the pattern: if this ever fails, the pattern has
// been loosened and the loosening is a traversal.
if (path !== resolve(path) || !path.startsWith(root + sep)) return c.notFound();
let size: number;
let modified: Date;
try {
/*
* realpath BEFORE the containment check, not just resolve().
*
* resolve() is lexical: it collapses `..` in the string but cannot see
* through a symlink, and stat() follows one. So a link planted in the
* media directory pointing at /etc/passwd passed the check above and was
* served in full, while this file's own header claimed containment. The
* directory is operator-populated and mounted read-only, so this was
* hardening rather than a live hole — but it becomes real the moment the
* directory is filled by an rsync or a tarball unpack.
*
* BOTH sides are resolved, though. Comparing a real file path against a
* LEXICAL root rejects the entire directory the moment the media root is
* itself reached through a symlink — a symlinked checkout, or a data
* volume under /var that is a link into /mnt — and the symptom is a
* blanket 404 on every video with nothing in the log to say why.
* Resolving the root the same way the file is resolved keeps the defence
* exactly as strict: the file still has to sit inside the real
* directory, so a link planted among the videos and pointing at
* /etc/passwd is still refused.
*/
const realRoot = await realpath(root);
const real = await realpath(path);
if (!real.startsWith(realRoot + sep)) return c.notFound();
const info = await stat(real);
if (!info.isFile()) return c.notFound();
size = info.size;
modified = info.mtime;
} catch {
// Missing, unreadable, a dangling symlink — all one answer. Telling the
// difference tells a prober which names exist, and unguessable names are
// the only thing standing between these files and enumeration.
return c.notFound();
}
const headers = new Headers({
'content-type': contentType,
'accept-ranges': 'bytes',
// Content-addressed: the bytes behind a name never change, so a year is
// safe and `immutable` stops the revalidation round trip on every seek.
'cache-control': 'public, max-age=31536000, immutable',
'last-modified': modified.toUTCString(),
etag: `"${size.toString(16)}-${modified.getTime().toString(16)}"`,
// These are downloads to a media element, never documents. Without it a
// browser that sniffs its way to text/html on a truncated file would
// treat same-origin bytes as a page.
'x-content-type-options': 'nosniff',
});
const range = parseByteRange(c.req.header('range'), size);
if (range === 'unsatisfiable') {
headers.set('content-range', `bytes */${size}`);
// Explicit zero rather than an absent header: without it Node falls back
// to chunked encoding for a body that does not exist.
headers.set('content-length', '0');
return new Response(null, { status: 416, headers });
}
const start = range ? range.start : 0;
const end = range ? range.end : Math.max(0, size - 1);
const length = size === 0 ? 0 : end - start + 1;
headers.set('content-length', String(length));
if (range) headers.set('content-range', `bytes ${start}-${end}/${size}`);
// A HEAD answers with the headers and no body — including the 206 status
// and Content-Range, so the player learns the file is seekable without
// fetching a byte of it.
if (c.req.method === 'HEAD') {
return new Response(null, { status: range ? 206 : 200, headers });
}
const stream = createReadStream(path, size === 0 ? undefined : { start, end });
return new Response(Readable.toWeb(stream) as ReadableStream, {
status: range ? 206 : 200,
headers,
});
});
return app;
}