516685526c
The ledger answers which contracted capacity is sold, to whom, at what margin. It says nothing about the motion — the repeatable practice that turns a customer conversation into a scoped deployment, and turns that deployment into something the next one reuses. Motion is deliberately not a parallel entity tree. DEMAND_STAGES already is the motion, so Motion binds reusable artefacts to the stages of a demand deal that already exists: an engagement hangs off one deal, cascade deleted, one per deal by unique constraint. Nine closed kinds, each declaring which stages it serves, and a starter library of twelve templates covering all eight open stages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
334 lines
16 KiB
Markdown
334 lines
16 KiB
Markdown
# Working on PIG
|
||
|
||
Onboarding for an agent or engineer joining this codebase cold. Read this
|
||
first, then [`docs/build-plan.md`](./docs/build-plan.md) for what to build.
|
||
|
||
Named `AGENTS.md` because that is the file coding agents look for by
|
||
convention. Everything here applies equally to humans.
|
||
|
||
---
|
||
|
||
## 1. What PIG is, in three sentences
|
||
|
||
A company that aggregates GPU capacity and resells it runs **two pipelines**,
|
||
and its business is the spread between them. Generic CRMs model one pipeline
|
||
against companies — they have no concept of inventory, no concept of a
|
||
commitment you already paid for, and so cannot answer the only question that
|
||
matters: *which contracted capacity is sold, to whom, at what margin, and what
|
||
is idle right now.*
|
||
|
||
PIG is built around that question, it is Apache 2.0, and it is designed to be
|
||
self-hosted by the customer.
|
||
|
||
**The load-bearing table is `allocations`**, which joins a `capacity_commitment`
|
||
(what we bought, at a known cost) to a `demand_deal` (what we sold, at a known
|
||
price). Margin, utilisation and idle capacity all fall out of that one join.
|
||
Everything else is plumbing that exists to keep that ledger honest.
|
||
|
||
---
|
||
|
||
## 2. Orientation
|
||
|
||
```
|
||
packages/core Ontology (stages, tiers, enums) + permissions + margin + palette
|
||
+ motion.ts (template kinds, integer qualification scoring)
|
||
packages/db Drizzle schema (51 tables), migrations, seeds
|
||
schema/motion.ts — the library, engagements and the loop
|
||
packages/prime Typed client for the Prime Intellect compute API
|
||
apps/api Hono HTTP API, auth, capacity/contract/calendar/motion services
|
||
apps/web React + Vite + Tailwind + shadcn-idiom components
|
||
apps/piggy The agent — lease-based queue worker + private chat server
|
||
apps/mcp MCP server (stdio) — 10 tools
|
||
apps/cli `pig`, the HTTP surface for scripts and agent kernels
|
||
docs/ ontology.md, motion.md, build-plan.md, agents.md, seed-data.md
|
||
deploy/ README.md (deployment), Caddyfile example, autodeploy units
|
||
```
|
||
|
||
~48,000 lines including tests. 382 tests across five packages
|
||
(core 78, prime 24, api 212, piggy 63, cli 5), plus a critical-path E2E suite
|
||
under `apps/api/e2e`. Node 22+.
|
||
|
||
| | |
|
||
|---|---|
|
||
| Repo | `PIG/pig` on git.karti.ai (Gitea) |
|
||
| Live | https://primeintellectgrowth.com |
|
||
| CI | Gitea Actions, `.gitea/workflows/ci.yml`, ~2 min, must stay green |
|
||
| Deploy | Push a `release-*` tag; CI publishes the image and the host's poller pulls it. A push to `main` deploys nothing. `bash scripts/deploy.sh` on the host is the manual path |
|
||
|
||
---
|
||
|
||
## 3. Running it
|
||
|
||
**PIG uses pnpm**, pinned by the `packageManager` field. Do not run `npm
|
||
install` — it will write a `package-lock.json` that nothing reads and resolve a
|
||
dependency tree that neither CI nor the image uses. Corepack ships with Node and
|
||
installs the pinned version for you:
|
||
|
||
```bash
|
||
corepack enable
|
||
|
||
pnpm install
|
||
|
||
# Postgres. PIG needs its own database — never point it at a shared one.
|
||
docker run -d --name pig-dev -p 5432:5432 \
|
||
-e POSTGRES_USER=pig -e POSTGRES_PASSWORD=pig -e POSTGRES_DB=pig \
|
||
postgres:16-alpine
|
||
|
||
export DATABASE_URL=postgres://pig:pig@localhost:5432/pig
|
||
pnpm run db:migrate
|
||
pnpm run db:seed # sourced, cited people — optional
|
||
pnpm run db:demo # a plausible demo book — optional, prefixed "DEMO — "
|
||
|
||
pnpm run dev:api # :8920
|
||
pnpm run dev:web # :5173, proxies /api to 8920
|
||
```
|
||
|
||
With no `SUPABASE_URL` set, **authentication is disabled in development** and
|
||
every request runs as the first user in the table. `loadConfig` refuses to start
|
||
in production without it, so this cannot leak.
|
||
|
||
Before pushing:
|
||
|
||
```bash
|
||
pnpm run typecheck && pnpm test
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Rules that must not be broken
|
||
|
||
These are load-bearing. Breaking one produces a subtle failure, not an error.
|
||
|
||
**Intelligence never lives in the API.** Handlers validate, authorize, call a
|
||
service, serialise. Research, enrichment, scoring and matching heuristics live
|
||
in the service layer or in the agent. The API signals the agent by *writing a
|
||
row to `agent_tasks`*, never by calling it — so the queue survives the agent
|
||
being down and no request thread ever blocks on a model.
|
||
|
||
**There are two auth providers, behind one interface.** Supabase for the
|
||
hosted deployment, OIDC for on-premises — see `apps/api/src/lib/auth-provider.ts`.
|
||
Both reduce to "verify a bearer token, return a subject and an email", because
|
||
that is all PIG needs. Never reach for a provider SDK outside that file.
|
||
|
||
**Authentication is not authorization.** A verified JWT proves someone has an
|
||
account in an identity provider that PIG *shares with another application*. It
|
||
does not prove they belong here. Access requires a row in PIG's own `users`
|
||
table. A token without one gets `403 needs_profile`, which the front end turns
|
||
into the join flow — never a login screen they have already completed.
|
||
|
||
**Cost is charged against the full commitment, not the hours that sold.**
|
||
Unsold hours are already paid for. Any other treatment reports a healthy margin
|
||
on a block that is losing money, which is precisely the failure PIG exists to
|
||
prevent. There is a test pinning this; if you "fix" it, read the test first.
|
||
|
||
**Sold and held are different.** A live hold removes capacity from everyone
|
||
else's availability — otherwise two sellers promise the same GPUs — but it is
|
||
not revenue and must never count toward utilisation.
|
||
|
||
**Money is integer cents.** Never floats. Round, never truncate: 2.43 is
|
||
2.4299999 in binary and a lost cent compounds across millions of GPU-hours.
|
||
|
||
**Agent claims carry evidence.** Enrichment writes to `facts` with a score, a
|
||
band, a source URL and a status — never directly to the record. Only `verified`
|
||
self-applies; anything weaker waits for a human. An agent allowed to write
|
||
unattributed claims will eventually write a wrong one and nobody will be able
|
||
to tell which.
|
||
|
||
**Never invent data about real people.** Seed records carry a confidence grade
|
||
and a source URL, both shown in the UI. No email address is ever inferred.
|
||
Authorship is not employment — contributors, residency participants and alumni
|
||
are recorded as what the evidence shows. Demo data is prefixed `DEMO — ` and
|
||
its customers are fictional.
|
||
|
||
---
|
||
|
||
## 5. Traps that have already bitten
|
||
|
||
Each of these cost real time. None produced an error message.
|
||
|
||
**`onConflictDoNothing()` is a no-op without a matching unique constraint.**
|
||
It has silently duplicated seed data twice. If there is no unique index to
|
||
conflict on, do an existence check instead.
|
||
|
||
**`z.coerce.boolean()` turns the string `"false"` into `true`.** Every feature
|
||
flag set to false was silently on. Use the `envBoolean` helper in
|
||
`apps/api/src/lib/config.ts`.
|
||
|
||
**Mutations must confirm themselves.** `<Toaster />` is mounted in `App.tsx`
|
||
inside `ThemeProvider`; use `toast.success` / `toast.error` in every mutation's
|
||
`onSuccess` / `onError`. The shadcn Toaster ships wired to `next-themes`, which
|
||
PIG does not use — it was rewired to PIG's `useTheme`. Before that it was never
|
||
mounted, so toasts already written in RecordSheets fired into nothing and every
|
||
save completed in silence.
|
||
|
||
**shadcn's `accent` is a SUBTLE surface, not the brand.** shadcn uses
|
||
`bg-accent` for hover, focus and selected states — dropdown items, command
|
||
rows, ghost buttons. The brand is `primary`. In `tailwind.config.js`, `accent`
|
||
is therefore aliased to `--accent-subtle` and `primary` to `--accent`. Use
|
||
`bg-primary` for a solid brand fill; never `bg-accent`. Mapping them the other
|
||
way makes every hover state paint a full-strength brand block, which in dark
|
||
mode with the monochrome palette is a glaring white slab.
|
||
|
||
**Grid and flex children need `min-w-0`.** They default to
|
||
`min-width: auto`, meaning they refuse to shrink below their content — and a
|
||
`tabular-nums` figure, a `whitespace-nowrap` badge or a `truncate` title is all
|
||
it takes. The page then scrolls sideways on a phone and nothing reports an
|
||
error. This was fixed three separate times at individual call sites before
|
||
`Card` was given `min-w-0` on its base class; **any new container primitive
|
||
needs the same**. Check with:
|
||
|
||
```js
|
||
document.documentElement.scrollWidth - document.documentElement.clientWidth
|
||
```
|
||
|
||
It should be 0 on every route at 393px wide.
|
||
|
||
**pnpm needs `CI=true` in any non-interactive build.** When it decides a
|
||
modules directory is stale it asks before removing it; with no TTY it cannot
|
||
ask, so it aborts with `ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY`. This reads
|
||
like a pnpm bug and is not — it is pnpm refusing to delete files nobody
|
||
confirmed. Both the Dockerfile and CI set it. The usual trigger is a host
|
||
`node_modules` reaching the build context, which is why `.dockerignore` exists:
|
||
pnpm's tree is symlinks into a content-addressed store, so copying it into an
|
||
image yields dangling links and a modules directory pnpm considers corrupt.
|
||
|
||
**`tsx` is a production dependency, not a dev one.** The server runs TypeScript
|
||
directly — `pnpm exec tsx apps/api/src/server.ts` is the container's command —
|
||
so pruning it away breaks the image. It lives in `dependencies` deliberately;
|
||
moving it back to `devDependencies` because "it's a build tool" makes
|
||
`pnpm install --prod` produce an image that cannot start.
|
||
|
||
**Drizzle-generated migrations are not always valid SQL.** A `jsonb → integer`
|
||
cast was emitted without the `USING` clause Postgres requires. Always apply a
|
||
new migration to a real empty database before pushing — CI does this, but find
|
||
out before CI does.
|
||
|
||
**The proxy allows exactly one inline script, by hash.** `index.html` carries a
|
||
pre-paint theme script that prevents a white flash for dark-mode users. Editing
|
||
it changes its hash and the browser silently blocks it. CI asserts the hash;
|
||
if it fails, update the CSP in `deploy/Caddyfile.example`, on the server, and
|
||
the expected value in the workflow.
|
||
|
||
**`prices.onDemand` from the Prime Intellect API is the TOTAL FOR THE NODE.**
|
||
Verified: 1× A100 at 1.79, 2× A100 at 3.58. `gpuMemory` is likewise a node
|
||
total. `packages/prime/src/map.ts` now divides both by `gpuCount` at the
|
||
boundary and keeps the node totals in `raw` for reconciliation — this was a
|
||
real bug that made an 8-GPU node read eight times too expensive. Anything new
|
||
that reads an upstream price must normalise the same way.
|
||
|
||
**The SPA fallback must never answer an `/api/` path.** Without an explicit
|
||
guard, an unknown API route returns `200 text/html` — the app shell — and the
|
||
caller sees `response.ok === true` before failing on `JSON.parse` with
|
||
"Unexpected token '<'", a long way from the cause. Both the static-file
|
||
middleware and the SPA fallback in `apps/api/src/server.ts` carry the guard;
|
||
anything added after them needs it too.
|
||
|
||
**Prime Intellect has two API hosts.** `api.primeintellect.ai` is compute and
|
||
pods. Inference is `api.pinference.ai/api/v1`, OpenAI-compatible.
|
||
|
||
**Piggy's default model thinks aloud.** `nvidia/nemotron-3-nano-30b-a3b` is a
|
||
hybrid reasoning model; under a tight `max_tokens` it rambles and truncates.
|
||
Pass `reasoning_effort: "none"` for tool use, routing and extraction.
|
||
|
||
**A route file with green tests can still be unmounted.** Every route module is
|
||
a factory returning a `Hono` app, and `createApp` has to call it. The tests
|
||
mount the factory themselves, so they pass whether or not `app.ts` ever does.
|
||
`read-guards.ts` and `learn.ts` were in exactly that state — which is why read
|
||
authorisation went unenforced and `/learn` answered 404 from a page that was in
|
||
the navigation — and are now mounted. `hubspot.ts` and `hubspot-webhook.ts`
|
||
still are not. After adding a route file, curl the path against a running
|
||
server; the test suite cannot tell you.
|
||
|
||
**A stale dev server on :8920 makes a mounted route look unmounted.** The curl
|
||
check the entry above recommends is only as good as the process answering it. `pnpm run dev:api` prints its `EADDRINUSE` and keeps running under
|
||
the process manager, so a server started hours earlier from an older checkout
|
||
goes on answering — and every new route 404s with a perfectly plausible
|
||
`{"error":"Not found"}` JSON body. Found this way: five Motion routes that were
|
||
correctly mounted read as missing for twenty minutes. Check
|
||
`ss -lptn 'sport = :8920'` before believing a 404, and read the dev server's log
|
||
rather than only its port.
|
||
|
||
**The demo seed skips an account it has already seen, and the rows hanging off
|
||
that account never appear.** `seedDemo` is idempotent per account, so a database
|
||
carrying a partial demo book from an earlier run silently produces no demand
|
||
deals — and anything that looks a deal up by name, as `seed/demo/motion.ts`
|
||
does, then reports zero and reads exactly like a broken loader. The fix is
|
||
`pnpm db:demo -- --clear` and a reseed, not a patch to the lookup.
|
||
|
||
**Deployment traps** live in `deploy/README.md` — chiefly that every Caddy site
|
||
block on that host needs `bind 10.0.0.2`, and that the CI runner uses
|
||
`container.network: host` so dependencies must be published on `127.0.0.1`.
|
||
|
||
---
|
||
|
||
## 6. Conventions
|
||
|
||
**Comments explain *why*, never *what*.** The code says what it does. Comments
|
||
carry the reasoning that would otherwise be lost — why this treatment and not
|
||
the obvious one, what breaks if it changes, what was tried and rejected. Match
|
||
the density already in the file; do not add narration.
|
||
|
||
**Match the surrounding style.** British spelling in prose and comments
|
||
(`utilisation`, `normalise`). Types are explicit at module boundaries. No
|
||
default exports.
|
||
|
||
**Enums come from `@pig/core`.** Never retype a stage list or a tier into a zod
|
||
schema — import it, so removing a value stops validating rather than silently
|
||
persisting.
|
||
|
||
**Tests pin decisions, not implementations.** The valuable cases are the ones
|
||
that would pass under a plausible-but-wrong version. Look at
|
||
`packages/core/test/margin.test.ts` for the register.
|
||
|
||
**Commits explain the reasoning**, including what was tried and rejected, and
|
||
say plainly when something was found by running the code rather than reading
|
||
it. Read `git log` for the register.
|
||
|
||
**Verify by running, not by assuming.** Several of the worst defects here were
|
||
found only by fetching a URL from another machine or applying a migration to a
|
||
real database. "It should work" has been wrong repeatedly.
|
||
|
||
---
|
||
|
||
## 7. Where to start
|
||
|
||
**Every task in the original three-wave plan has shipped.**
|
||
[`docs/build-plan.md`](./docs/build-plan.md) is now an audited record of that
|
||
rather than a queue, and it carries the remaining work at the bottom. The two
|
||
interfaces everything else codes against — `packages/core/src/permissions.ts`
|
||
(the RBAC model) and `apps/api/src/lib/mutation.ts` (the write path) — are
|
||
settled; read them before adding any write.
|
||
|
||
**The highest-value work now, in order:**
|
||
|
||
1. **Mount `createReadGuardRoutes`.** The read half of the permission model is
|
||
written, tabulated and tested, and does nothing, because `app.ts` never
|
||
mounts it. Until it does, every authenticated member can read supplier cost
|
||
and margin. It is one line, and it must be registered *before* the handlers
|
||
it guards — Hono runs matched handlers in registration order.
|
||
2. **Mount `learn.ts`.** `/learn` is in the navigation and its API answers 404.
|
||
3. **Enqueue the other six agent task kinds.** The worker is complete; only
|
||
`enrich_account` and `enrich_contact` are ever written to `agent_tasks`, so
|
||
Piggy does far less than the ontology implies.
|
||
4. **Mount the HubSpot routes, or delete them.** Seven tables, OAuth, sync jobs
|
||
and webhook verification, all written, tested and unreachable.
|
||
|
||
`app.ts` is the one shared file. If your change needs a route mounted, a public
|
||
path allowlisted or a schema widened there, say so rather than racing another
|
||
agent for it.
|
||
|
||
---
|
||
|
||
## 8. What not to do
|
||
|
||
- Do not copy component files out of other people's repositories. Where a
|
||
primitive is a shadcn/ui original, take it from upstream, where it is
|
||
canonical and current. Compositions we write ourselves.
|
||
- Do not open self-registration on the identity provider. It is shared with
|
||
another application. PIG mints accounts itself, gated on an invite.
|
||
- Do not put a production SSH key on the CI runner. Deployment is manual on
|
||
purpose.
|
||
- Do not weaken the production guard that refuses to start without an identity
|
||
provider.
|
||
- Do not seed or infer email addresses for real people.
|