f0173440e4
Piggy was a hand-rolled OpenAI tool loop. It is now a Prime Agent session — Prime Intellect's own harness, embedded as a Node library — answering from PIG's tools and, for the first time, able to put information into the CRM rather than only read it out. The harness is a coding agent, so the first job was taking the coding agent away from it. `noTools: 'all'` plus an explicit allowlist leaves the model with PIG's ten `pig_*` tools and no bash, no filesystem, no IPython. That holds under attack: a hostile extension, a skill and a settings file planted in the agent's own directory, then `setActiveToolsByName` called with every built-in, still leaves ten tools, all ours. Both lines are load-bearing — `noTools` alone registers nothing, and the allowlist is what admits our own. Writing is gated rather than assumed. A change is proposed, not made: the tool returns a description, the transcript renders a diff card, and nothing reaches the database until someone presses Apply. Contracts, commitments, allocations and compliance always stop for a human whatever the mode. Every write runs through `executeMutation` as the calling user, so their capabilities and the audit trail apply exactly as they would to a human's. Four things about the SDK are wrong in its own documentation and cost a debugging cycle each: models.json does not resolve an env var name for `apiKey`, it sends the literal string; there is no built-in prime-inference provider in 0.84.1; a ResourceLoader you pass in is never reloaded for you; and the stock system prompt is a coding-assistant prompt that must be replaced — but replacing it also silently removes the tool list, because the harness only renders that section when it owns the prompt. AGENTS.md records all four. The expensive one was thinking level. The harness defaults to `medium`, and nemotron spent an entire 4,096-token budget reasoning and returned an empty answer. `low` was worse; `off` omits the parameter so the endpoint's default wins. An explicit `reasoning_effort: none` via `thinkingLevelMap` took a turn from 6,195 output tokens to 149. And a turn is now bounded. The harness loop is `while (true)` with no iteration cap; a runaway on a frontier model would have eaten the credit it is supposed to report on. Ceilings on model calls and tokens, enforced both through the harness hook and independently from the event stream, plus a per-user daily spend limit — and the ledger now records spend on turns that fail, which it previously discarded. Signing in lands on /piggy, which is a workspace: conversations down one side, the agent in the middle, what it did and what it cost beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
593 lines
30 KiB
Markdown
593 lines
30 KiB
Markdown
# Working on PIG
|
||
|
||
Onboarding for an agent or engineer joining this codebase cold. Read this
|
||
first, then [`docs/build-plan.md`](./docs/build-plan.md) for what to build.
|
||
|
||
Named `AGENTS.md` because that is the file coding agents look for by
|
||
convention. Everything here applies equally to humans.
|
||
|
||
---
|
||
|
||
## 1. What PIG is, in three sentences
|
||
|
||
A company that aggregates GPU capacity and resells it runs **two pipelines**,
|
||
and its business is the spread between them. Generic CRMs model one pipeline
|
||
against companies — they have no concept of inventory, no concept of a
|
||
commitment you already paid for, and so cannot answer the only question that
|
||
matters: *which contracted capacity is sold, to whom, at what margin, and what
|
||
is idle right now.*
|
||
|
||
PIG is built around that question, it is Apache 2.0, and it is designed to be
|
||
self-hosted by the customer.
|
||
|
||
**The load-bearing table is `allocations`**, which joins a `capacity_commitment`
|
||
(what we bought, at a known cost) to a `demand_deal` (what we sold, at a known
|
||
price). Margin, utilisation and idle capacity all fall out of that one join.
|
||
Everything else is plumbing that exists to keep that ledger honest.
|
||
|
||
---
|
||
|
||
## 2. Orientation
|
||
|
||
```
|
||
packages/core Ontology (stages, tiers, enums) + permissions + margin + palette
|
||
packages/db Drizzle schema (47 tables), migrations, seeds
|
||
packages/prime Typed client for the Prime Intellect compute API
|
||
apps/api Hono HTTP API, auth, capacity/contract/calendar services
|
||
apps/web React + Vite + Tailwind + shadcn-idiom components
|
||
apps/piggy The agent — a Prime Agent session over the CRM tools behind a
|
||
private chat server, plus a lease-based queue worker (§6)
|
||
apps/mcp MCP server (stdio) — 9 tools
|
||
apps/cli `pig`, the HTTP surface for scripts and agent kernels
|
||
docs/ ontology.md, build-plan.md, agents.md, seed-data.md,
|
||
screenshots.md, learn-scripts.md
|
||
deploy/ README.md (deployment), Caddyfile example, autodeploy units
|
||
```
|
||
|
||
~45,000 lines including tests. 403 unit tests across five packages
|
||
(core 62, prime 24, api 217, piggy 95, cli 5), plus E2E suites under
|
||
`apps/api/e2e` and `apps/piggy/e2e` that need a database — and, for one Piggy
|
||
case, a key. Node 22+.
|
||
|
||
| | |
|
||
|---|---|
|
||
| Repo | `PIG/pig` on git.karti.ai (Gitea) |
|
||
| Live | https://primeintellectgrowth.com |
|
||
| CI | Gitea Actions, `.gitea/workflows/ci.yml`, ~2 min, must stay green |
|
||
| Deploy | Push a `release-*` tag; CI publishes the image and the host's poller pulls it. A push to `main` deploys nothing. `bash scripts/deploy.sh` on the host is the manual path |
|
||
|
||
---
|
||
|
||
## 3. Running it
|
||
|
||
**PIG uses pnpm**, pinned by the `packageManager` field. Do not run `npm
|
||
install` — it will write a `package-lock.json` that nothing reads and resolve a
|
||
dependency tree that neither CI nor the image uses. Corepack ships with Node and
|
||
installs the pinned version for you:
|
||
|
||
```bash
|
||
corepack enable
|
||
|
||
pnpm install
|
||
|
||
# Postgres. PIG needs its own database — never point it at a shared one.
|
||
docker run -d --name pig-dev -p 5432:5432 \
|
||
-e POSTGRES_USER=pig -e POSTGRES_PASSWORD=pig -e POSTGRES_DB=pig \
|
||
postgres:16-alpine
|
||
|
||
export DATABASE_URL=postgres://pig:pig@localhost:5432/pig
|
||
pnpm run db:migrate
|
||
pnpm run db:seed # sourced, cited people — optional
|
||
pnpm run db:demo # a plausible demo book — optional, prefixed "DEMO — "
|
||
|
||
pnpm run dev:api # :8920
|
||
pnpm run dev:web # :5173, proxies /api to 8920
|
||
```
|
||
|
||
With no `SUPABASE_URL` set, **authentication is disabled in development** and
|
||
every request runs as the first user in the table. `loadConfig` refuses to start
|
||
in production without it, so this cannot leak.
|
||
|
||
Before pushing:
|
||
|
||
```bash
|
||
pnpm run typecheck && pnpm test
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Rules that must not be broken
|
||
|
||
These are load-bearing. Breaking one produces a subtle failure, not an error.
|
||
|
||
**Intelligence never lives in the API.** Handlers validate, authorize, call a
|
||
service, serialise. Research, enrichment, scoring and matching heuristics live
|
||
in the service layer or in the agent. The API signals the agent by *writing a
|
||
row to `agent_tasks`*, never by calling it — so the queue survives the agent
|
||
being down and no request thread ever blocks on a model.
|
||
|
||
**There are two auth providers, behind one interface.** Supabase for the
|
||
hosted deployment, OIDC for on-premises — see `apps/api/src/lib/auth-provider.ts`.
|
||
Both reduce to "verify a bearer token, return a subject and an email", because
|
||
that is all PIG needs. Never reach for a provider SDK outside that file.
|
||
|
||
**Authentication is not authorization.** A verified JWT proves someone has an
|
||
account in an identity provider that PIG *shares with another application*. It
|
||
does not prove they belong here. Access requires a row in PIG's own `users`
|
||
table. A token without one gets `403 needs_profile`, which the front end turns
|
||
into the join flow — never a login screen they have already completed.
|
||
|
||
**Cost is charged against the full commitment, not the hours that sold.**
|
||
Unsold hours are already paid for. Any other treatment reports a healthy margin
|
||
on a block that is losing money, which is precisely the failure PIG exists to
|
||
prevent. There is a test pinning this; if you "fix" it, read the test first.
|
||
|
||
**Sold and held are different.** A live hold removes capacity from everyone
|
||
else's availability — otherwise two sellers promise the same GPUs — but it is
|
||
not revenue and must never count toward utilisation.
|
||
|
||
**Money is integer cents.** Never floats. Round, never truncate: 2.43 is
|
||
2.4299999 in binary and a lost cent compounds across millions of GPU-hours.
|
||
|
||
**Agent claims carry evidence.** Enrichment writes to `facts` with a score, a
|
||
band, a source URL and a status — never directly to the record. Only `verified`
|
||
self-applies; anything weaker waits for a human. An agent allowed to write
|
||
unattributed claims will eventually write a wrong one and nobody will be able
|
||
to tell which.
|
||
|
||
**Never invent data about real people.** Seed records carry a confidence grade
|
||
and a source URL, both shown in the UI. No email address is ever inferred.
|
||
Authorship is not employment — contributors, residency participants and alumni
|
||
are recorded as what the evidence shows. Demo data is prefixed `DEMO — ` and
|
||
its customers are fictional.
|
||
|
||
---
|
||
|
||
## 5. Traps that have already bitten
|
||
|
||
Each of these cost real time. None produced an error message.
|
||
|
||
**`onConflictDoNothing()` is a no-op without a matching unique constraint.**
|
||
It has silently duplicated seed data twice. If there is no unique index to
|
||
conflict on, do an existence check instead.
|
||
|
||
**`z.coerce.boolean()` turns the string `"false"` into `true`.** Every feature
|
||
flag set to false was silently on. Use the `envBoolean` helper in
|
||
`apps/api/src/lib/config.ts`.
|
||
|
||
**Mutations must confirm themselves.** `<Toaster />` is mounted in `App.tsx`
|
||
inside `ThemeProvider`; use `toast.success` / `toast.error` in every mutation's
|
||
`onSuccess` / `onError`. The shadcn Toaster ships wired to `next-themes`, which
|
||
PIG does not use — it was rewired to PIG's `useTheme`. Before that it was never
|
||
mounted, so toasts already written in RecordSheets fired into nothing and every
|
||
save completed in silence.
|
||
|
||
**shadcn's `accent` is a SUBTLE surface, not the brand.** shadcn uses
|
||
`bg-accent` for hover, focus and selected states — dropdown items, command
|
||
rows, ghost buttons. The brand is `primary`. In `tailwind.config.js`, `accent`
|
||
is therefore aliased to `--accent-subtle` and `primary` to `--accent`. Use
|
||
`bg-primary` for a solid brand fill; never `bg-accent`. Mapping them the other
|
||
way makes every hover state paint a full-strength brand block, which in dark
|
||
mode with the monochrome palette is a glaring white slab.
|
||
|
||
**Grid and flex children need `min-w-0`.** They default to
|
||
`min-width: auto`, meaning they refuse to shrink below their content — and a
|
||
`tabular-nums` figure, a `whitespace-nowrap` badge or a `truncate` title is all
|
||
it takes. The page then scrolls sideways on a phone and nothing reports an
|
||
error. This was fixed three separate times at individual call sites before
|
||
`Card` was given `min-w-0` on its base class; **any new container primitive
|
||
needs the same**. Check with:
|
||
|
||
```js
|
||
document.documentElement.scrollWidth - document.documentElement.clientWidth
|
||
```
|
||
|
||
It should be 0 on every route at 393px wide.
|
||
|
||
**pnpm needs `CI=true` in any non-interactive build.** When it decides a
|
||
modules directory is stale it asks before removing it; with no TTY it cannot
|
||
ask, so it aborts with `ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY`. This reads
|
||
like a pnpm bug and is not — it is pnpm refusing to delete files nobody
|
||
confirmed. Both the Dockerfile and CI set it. The usual trigger is a host
|
||
`node_modules` reaching the build context, which is why `.dockerignore` exists:
|
||
pnpm's tree is symlinks into a content-addressed store, so copying it into an
|
||
image yields dangling links and a modules directory pnpm considers corrupt.
|
||
|
||
**`tsx` is a production dependency, not a dev one.** The server runs TypeScript
|
||
directly — `pnpm exec tsx apps/api/src/server.ts` is the container's command —
|
||
so pruning it away breaks the image. It lives in `dependencies` deliberately;
|
||
moving it back to `devDependencies` because "it's a build tool" makes
|
||
`pnpm install --prod` produce an image that cannot start.
|
||
|
||
**Drizzle-generated migrations are not always valid SQL.** A `jsonb → integer`
|
||
cast was emitted without the `USING` clause Postgres requires. Always apply a
|
||
new migration to a real empty database before pushing — CI does this, but find
|
||
out before CI does.
|
||
|
||
**The proxy allows exactly one inline script, by hash.** `index.html` carries a
|
||
pre-paint theme script that prevents a white flash for dark-mode users. Editing
|
||
it changes its hash and the browser silently blocks it. CI asserts the hash;
|
||
if it fails, update the CSP in `deploy/Caddyfile.example`, on the server, and
|
||
the expected value in the workflow.
|
||
|
||
**`prices.onDemand` from the Prime Intellect API is the TOTAL FOR THE NODE.**
|
||
Verified: 1× A100 at 1.79, 2× A100 at 3.58. `gpuMemory` is likewise a node
|
||
total. `packages/prime/src/map.ts` now divides both by `gpuCount` at the
|
||
boundary and keeps the node totals in `raw` for reconciliation — this was a
|
||
real bug that made an 8-GPU node read eight times too expensive. Anything new
|
||
that reads an upstream price must normalise the same way.
|
||
|
||
**The SPA fallback must never answer an `/api/` path.** Without an explicit
|
||
guard, an unknown API route returns `200 text/html` — the app shell — and the
|
||
caller sees `response.ok === true` before failing on `JSON.parse` with
|
||
"Unexpected token '<'", a long way from the cause. Both the static-file
|
||
middleware and the SPA fallback in `apps/api/src/server.ts` carry the guard;
|
||
anything added after them needs it too.
|
||
|
||
**Prime Intellect has two API hosts.** `api.primeintellect.ai` is compute and
|
||
pods. Inference is `api.pinference.ai/api/v1`, OpenAI-compatible.
|
||
|
||
**Piggy's default model thinks aloud, and the harness makes it worse.** The
|
||
agent SDK defaults `thinkingLevel` to `medium`; on `nvidia/nemotron-3-nano-30b-a3b`
|
||
that produced 6,195 output tokens of reasoning and an *empty* answer. The fix is
|
||
two halves and both are needed — see [§6](#6-piggy-and-the-harness-it-runs-on).
|
||
|
||
**A route file with green tests can still be unmounted.** Every route module is
|
||
a factory returning a `Hono` app, and `createApp` has to call it. The tests
|
||
mount the factory themselves, so they pass whether or not `app.ts` ever does.
|
||
Four modules are in exactly that state right now — `read-guards.ts`,
|
||
`learn.ts`, `hubspot.ts`, `hubspot-webhook.ts` — which is why read
|
||
authorisation is unenforced and `/learn` answers 404 from a page that is in the
|
||
navigation. After adding a route file, curl the path against a running server;
|
||
the test suite cannot tell you.
|
||
|
||
**Deployment traps** live in `deploy/README.md` — chiefly that every Caddy site
|
||
block on that host needs `bind 10.0.0.2`, and that the CI runner uses
|
||
`container.network: host` so dependencies must be published on `127.0.0.1`.
|
||
|
||
---
|
||
|
||
## 6. Piggy, and the harness it runs on
|
||
|
||
Everything below was learned by running the thing. The product-level account is
|
||
in the README under *The agent surface*; this section is the engineering one,
|
||
and it exists because much of what follows either contradicts the SDK's own
|
||
documentation or is invisible in TypeScript.
|
||
|
||
### 6.1 The shape
|
||
|
||
`apps/piggy` embeds **Prime Agent** — Prime Intellect's harness,
|
||
`@earendil-works/pi-coding-agent@0.84.1`, MIT — as a Node library. Nothing is
|
||
shelled out to, and there is no second process.
|
||
|
||
```
|
||
src/agent/session.ts Builds a turn: runtime, credential, model, prompt,
|
||
tools, and the assertions that make the tool set a
|
||
fact rather than a hope
|
||
src/agent/models.json The provider document the harness reads: five models,
|
||
their prices, their context windows, their reasoning
|
||
maps. Copied verbatim into PIGGY_AGENT_DIR when the
|
||
runtime is first built (once per process)
|
||
src/agent/models.ts Validates that file and turns it into the picker's
|
||
catalogue. One source for price and size
|
||
src/agent/prompt.ts Piggy's system prompt, including the tool list the
|
||
harness stops writing (§6.4)
|
||
src/agent/tool-bridge.ts PIG's zod `AgentTool`s → harness `ToolDefinition`s
|
||
src/chat-tools.ts Read tools ─┐
|
||
src/page-tools.ts Page summaries ├─ the product; the harness swap did
|
||
src/lifecycle-tools.ts Lifecycle ─┘ not touch a line of them
|
||
src/write-tools.ts The five write tools and the approval flow
|
||
src/chat-server.ts The NDJSON server, the approval rendezvous, the ledger
|
||
src/provider.ts + worker.ts + queue.ts The queue worker, which does NOT use
|
||
the harness at all — it still speaks
|
||
OpenAI-completions directly
|
||
```
|
||
|
||
The queue worker and the chat agent are different code paths that happen to
|
||
share a process. `PIGGY_MODEL` and `PIGGY_INFERENCE_BASE` belong to the worker;
|
||
`PIGGY_AGENT_*` and `models.json` belong to the agent. Changing one does not
|
||
change the other, which has already confused one person into "fixing" the model
|
||
in the wrong place.
|
||
|
||
### 6.2 No shell, and why the flag is not enough
|
||
|
||
The session is constructed with `noTools: 'all'` **plus** an explicit `tools`
|
||
allowlist (the `createAgentSession` call in `agent/session.ts`). Neither alone
|
||
would do:
|
||
`noTools: 'all'` removes the built-ins, and the allowlist is the positive
|
||
statement of what may exist. But both are *the harness's* configuration, and the
|
||
harness composes its tool set from several sources — built-ins, extensions,
|
||
skills, custom tools — so a future release that changes the precedence between
|
||
them would widen the set without changing a line of PIG. Three gates exist for
|
||
that reason:
|
||
|
||
1. `assertPigToolBoundary` (`src/chat.ts`) — a name must start `pig_` and must
|
||
not read like a shell. PIG's own code, PIG's own rule.
|
||
2. `assertUniqueToolNames` (`agent/session.ts`) — the harness keeps its tools in
|
||
a `Map` keyed by name and *sets* each one in turn
|
||
(`dist/core/agent-session.js:1963-1968`), so a duplicate silently overwrites
|
||
the other. That is how a read tool ends up answering for a write tool of the
|
||
same name, with nothing anywhere saying so.
|
||
3. `assertExactToolSet` (`agent/session.ts`) — compares the live
|
||
`session.agent.state.tools` against exactly what was handed in and throws at
|
||
session construction if they differ. This is the one that would notice a
|
||
harness upgrade.
|
||
|
||
`test/agent-session.test.ts` pins all three, including `pig_bash` and friends.
|
||
Extensions, skills, prompt templates, themes and context-file discovery are all
|
||
disabled on the `DefaultResourceLoader`, and `PIGGY_AGENT_DIR` is deliberately
|
||
not a checkout: the harness reads context files from its cwd, and the cwd is
|
||
also appended to the live system prompt verbatim as
|
||
`Current working directory: …`.
|
||
|
||
### 6.3 Four places the SDK's own docs are wrong
|
||
|
||
Each of these compiles, starts, and fails somewhere else.
|
||
|
||
**`apiKey` in `models.json` is not an environment variable name.** Writing
|
||
`"apiKey": "PRIME_API_KEY"` sends the literal string `PRIME_API_KEY` as the
|
||
bearer token, and the endpoint answers 401. The value is a *template*:
|
||
`$PRIME_API_KEY` or `${PRIME_API_KEY}` interpolate, a leading `!` executes the
|
||
rest as a shell command, and anything else is a literal
|
||
(`dist/core/resolve-config-value.js:116-128`). PIG uses none of those forms —
|
||
it calls
|
||
`modelRuntime.setRuntimeApiKey(PIGGY_PROVIDER_ID, config.PRIME_API_KEY)`
|
||
(`agent/session.ts`), which is the only line that authenticates Piggy and keeps
|
||
the key out of the file that gets written to disk.
|
||
|
||
**There is no built-in `prime-inference` provider in 0.84.1.** The published
|
||
docs describe a build that is not on npm; `KnownProvider` in
|
||
`@earendil-works/pi-ai/dist/types.d.ts:19` lists forty providers and none of
|
||
them is Prime Intellect's inference host. PIG registers one itself from
|
||
`models.json`, and the id `prime-inference` has to match in three places — the
|
||
JSON key, `setRuntimeApiKey`, and `modelRuntime.getModel`. A typo in any of them
|
||
surfaces as a 401 or an undefined model, never as "unknown provider".
|
||
|
||
**A `ResourceLoader` you pass in is never reloaded for you.**
|
||
`createAgentSession` constructs and reloads one *only when you do not supply
|
||
one* (`dist/core/sdk.js:75-78`). Pass your own and forget `await loader.reload()`
|
||
and the session runs on the stock coding-assistant preamble — no error, no
|
||
warning, and an agent that offers to read your files.
|
||
|
||
**The stock prompt is a coding-assistant prompt and must be replaced, not
|
||
appended to.** It opens "You are an expert coding assistant operating inside pi"
|
||
and cites the SDK's own README paths (`dist/core/system-prompt.js:73`).
|
||
Appending does not help: a CRM agent told it edits code reaches for tools it
|
||
does not have and apologises for not having them. The replacement goes through
|
||
the loader's `systemPromptOverride`, which takes the literal text — the
|
||
`systemPrompt` option is a *file source*, and handing it a prompt loads nothing
|
||
and says nothing.
|
||
|
||
### 6.4 Replacing the prompt silently removes the tool list
|
||
|
||
`buildSystemPrompt` returns early on the `customPrompt` branch
|
||
(`dist/core/system-prompt.js:13-33`); the "Available tools" section is only ever
|
||
built further down, on the branch where no custom prompt was supplied
|
||
(`:40`, `:75`). So the moment the preamble is replaced — which is not optional
|
||
here — every tool becomes invisible to the model, `promptSnippet` or not.
|
||
|
||
`agent/prompt.ts` therefore renders the list itself, in `toolSection`. A 30B
|
||
model that cannot see a tool in its prompt answers from the page title instead
|
||
of calling it, and that failure is completely silent: the tool is registered,
|
||
callable, and never called. If you add a tool, give it a `promptSnippet`, and
|
||
check it appears in `session.systemPrompt`.
|
||
|
||
### 6.5 The thinking-level trap
|
||
|
||
The one that cost real money.
|
||
|
||
The harness defaults `thinkingLevel` to `medium`. On the default model that
|
||
produced **6,195 output tokens of reasoning and an empty answer**, stopping at
|
||
`finish_reason: length` — the budget was gone before a word of the reply was
|
||
written, and reasoning bills as output. `low` was worse. After the fix the same
|
||
question answered correctly in **149 output tokens**.
|
||
|
||
The fix is two halves and either alone is silent:
|
||
|
||
- `PIGGY_AGENT_THINKING` defaults to `off` (`src/config.ts`), and
|
||
- the model entry carries a `thinkingLevelMap` mapping `off` → `"none"`
|
||
(`src/agent/models.json`).
|
||
|
||
Why the second is needed: a thinking level of `off` becomes
|
||
`reasoningEffort: undefined` in the provider
|
||
(`@earendil-works/pi-ai/dist/api/openai-completions.js:473-474`), and the
|
||
request builder then emits `reasoning_effort` **only if the model has a map**:
|
||
|
||
```js
|
||
else if (options?.reasoningEffort && model.reasoning && compat.supportsReasoningEffort) {
|
||
params.reasoning_effort = model.thinkingLevelMap?.[options.reasoningEffort] ?? options.reasoningEffort;
|
||
}
|
||
else if (!options?.reasoningEffort && model.reasoning && compat.supportsReasoningEffort) {
|
||
const offValue = model.thinkingLevelMap?.off;
|
||
if (typeof offValue === "string") { params.reasoning_effort = offValue; }
|
||
}
|
||
— dist/api/openai-completions.js:657-666
|
||
```
|
||
|
||
Without the map, `off` sends **no reasoning parameter at all** and the
|
||
endpoint's own default — thinking on, verbosely — wins. This is per model. The
|
||
two nemotron entries have a map; deepseek, opus and gpt-5.6 do not, and were
|
||
left to their own defaults deliberately. **If you change `PIGGY_AGENT_MODEL` and
|
||
answers start coming back empty or truncated, this is why.**
|
||
`test/agent-thinking.test.ts` fails if the default model has no map, and
|
||
`e2e/prime-agent.test.ts` counts the tokens against the live endpoint.
|
||
|
||
### 6.6 Modes, and the one function that decides
|
||
|
||
`PiggyMode` is `read_only` | `confirm` | `auto`.
|
||
|
||
- **`read_only`** offers no write tool at all. Not offered-and-refused: absent
|
||
(`createPigWriteTools` returns `[]`). A model that can see a capability
|
||
narrates using it.
|
||
- **`confirm`** — the shipped default — turns every write into a proposal. The
|
||
tool emits an `approval_required` card, the turn stays open, the decision
|
||
arrives on a separate `POST /internal/approve`, and only then does the
|
||
mutation run.
|
||
- **`auto`** writes immediately, as the calling user, under their permissions.
|
||
|
||
Contracts, commitments, allocations and compliance require a human in **every**
|
||
mode. That rule is one function — `requiresApproval` in
|
||
`packages/core/src/piggy-protocol.ts` — and it is the single source of truth:
|
||
the write tools read it, the tests assert against it, and nothing restates it.
|
||
If you add a guarded kind, add it to `PIGGY_ALWAYS_CONFIRM_KINDS` and everything
|
||
downstream follows.
|
||
|
||
Two properties of the write path are not negotiable. Every write goes through
|
||
`executeMutation` with the caller's own `Principal`, so Piggy holds no privilege
|
||
of its own — there is no elevated principal anywhere in `write-tools.ts` and
|
||
there must never be one. And a refusal is an *answer*: a missing capability, a
|
||
declined card and a rejected input all come back as ordinary tool results whose
|
||
first line says `NOT SAVED`. Thrown into the stream they would end the turn on
|
||
the user's own permissions, which reads to them as Piggy being broken.
|
||
|
||
The rendezvous itself (`ApprovalRegistry` in `src/chat-server.ts`) is single-use
|
||
— an id is deleted the instant it settles, so a replayed decision cannot apply a
|
||
change twice — deadlined at five minutes, and turn-owned: an abandoned turn
|
||
rejects every approval it opened, because a pending promise there holds a billed
|
||
inference connection open.
|
||
|
||
### 6.7 The browser never learns which harness this is
|
||
|
||
Prime Agent emits twenty-three event types. PIG's own protocol
|
||
(`PiggyChatEvent` in `packages/core/src/piggy-protocol.ts`) has nine, and
|
||
`translateSessionEvent` in `src/chat-server.ts` maps exactly four of the
|
||
harness's — `message_update`, `tool_execution_start`, `tool_execution_end`,
|
||
`turn_end` — and drops the rest on the server. That is deliberate: a harness
|
||
upgrade is then a server change and never a client one.
|
||
|
||
The risk in a `default: return` is the upgrade that *adds* an event — a
|
||
delegated sub-agent, a permission request — which would be dropped in silence
|
||
for as long as it took somebody to notice a missing feature.
|
||
`test/chat-server.test.ts` therefore writes out both lists and asserts, at
|
||
compile time, that they are mutually assignable with `AgentSessionEvent['type']`.
|
||
Bump the SDK and `tsc` tells you what is new before anything runs.
|
||
|
||
### 6.8 Working on Piggy without spending credit
|
||
|
||
Almost all of it is free, and only one path is not.
|
||
|
||
- **The unit suite never makes a request.** `createPiggySession` resolves the
|
||
model, builds the prompt and registers the tools entirely offline with a fake
|
||
key, so the tool set, the prompt, the thinking level and the model's own
|
||
ceiling are all inspectable without inference. That is what
|
||
`test/agent-session.test.ts` and `test/agent-thinking.test.ts` do.
|
||
- **The whole chat protocol is drivable with no model at all.**
|
||
`startPiggyChatServer` takes `createSession`, `createReadTools` and
|
||
`createWriteTools` as options; the tests hand it a fake harness that emits
|
||
real `AgentSessionEvent`s. `e2e/approval-rendezvous.test.ts` does this against
|
||
a real database, which is how the approval flow is tested end to end for free.
|
||
- **`src/dev/mock-inference.ts`** (`pnpm -F @pig/piggy run dev:mock`, port 8945)
|
||
speaks the OpenAI-compatible wire protocol with steering directives —
|
||
`/mock error`, `/mock ratelimit`, `/mock cut`, `/mock badtool`. Note what it
|
||
serves: the **queue worker**, through `PIGGY_INFERENCE_BASE`. The agent reads
|
||
its base URL from `models.json`, so pointing the chat path at the mock means
|
||
editing that file.
|
||
- **`src/dev/verify-prime-agent.ts`**
|
||
(`pnpm -F @pig/piggy exec tsx src/dev/verify-prime-agent.ts [modelId]`) is the
|
||
live probe: it asks the real endpoint one question with a seeded tool and
|
||
prints the model, the tool set, whether anything shell-shaped survived, the
|
||
first 200 characters of the system prompt and the answer. It spends a few
|
||
hundred tokens. Nothing in CI runs it.
|
||
- **The database.** Anything that writes runs against a scratch database, never
|
||
the development book — an activity appearing in somebody's feed because a test
|
||
ran is exactly what a CRM must not do. `e2e/write-tools.test.ts` and
|
||
`e2e/approval-rendezvous.test.ts` take `PIGGY_WRITE_DATABASE_URL` and refuse
|
||
`pig_combined` by name.
|
||
- **The one paid test** is `e2e/prime-agent.test.ts`, gated on
|
||
`PIGGY_E2E_LIVE=1` *and* a key, because a suite that spends money whenever the
|
||
environment happens to be loaded spends money by accident. One turn is about
|
||
$0.0003.
|
||
|
||
```bash
|
||
# Unit suite: no database, no key, no network.
|
||
pnpm -F @pig/piggy run typecheck && pnpm -F @pig/piggy run test
|
||
|
||
# E2E: a scratch database of its own. `pig_combined` is refused by name.
|
||
docker exec pig-ux-db psql -U pig -d postgres -c "CREATE DATABASE pig_scratch"
|
||
DATABASE_URL=postgres://pig:pig@localhost:54330/pig_scratch pnpm -F @pig/db run migrate
|
||
DATABASE_URL=postgres://pig:pig@localhost:54330/pig_scratch \
|
||
PIGGY_WRITE_DATABASE_URL=postgres://pig:pig@localhost:54330/pig_scratch \
|
||
pnpm -F @pig/piggy run test:e2e # the live case skips, and says so
|
||
|
||
# Add the paid one deliberately, never by default.
|
||
PIGGY_E2E_LIVE=1 PRIME_API_KEY=... pnpm -F @pig/piggy run test:e2e
|
||
```
|
||
|
||
---
|
||
|
||
## 7. Conventions
|
||
|
||
**Comments explain *why*, never *what*.** The code says what it does. Comments
|
||
carry the reasoning that would otherwise be lost — why this treatment and not
|
||
the obvious one, what breaks if it changes, what was tried and rejected. Match
|
||
the density already in the file; do not add narration.
|
||
|
||
**Match the surrounding style.** British spelling in prose and comments
|
||
(`utilisation`, `normalise`). Types are explicit at module boundaries. No
|
||
default exports.
|
||
|
||
**Enums come from `@pig/core`.** Never retype a stage list or a tier into a zod
|
||
schema — import it, so removing a value stops validating rather than silently
|
||
persisting.
|
||
|
||
**Tests pin decisions, not implementations.** The valuable cases are the ones
|
||
that would pass under a plausible-but-wrong version. Look at
|
||
`packages/core/test/margin.test.ts` for the register.
|
||
|
||
**Commits explain the reasoning**, including what was tried and rejected, and
|
||
say plainly when something was found by running the code rather than reading
|
||
it. Read `git log` for the register.
|
||
|
||
**Verify by running, not by assuming.** Several of the worst defects here were
|
||
found only by fetching a URL from another machine or applying a migration to a
|
||
real database. "It should work" has been wrong repeatedly.
|
||
|
||
---
|
||
|
||
## 8. Where to start
|
||
|
||
**Every task in the original three-wave plan has shipped.**
|
||
[`docs/build-plan.md`](./docs/build-plan.md) is now an audited record of that
|
||
rather than a queue, and it carries the remaining work at the bottom. The two
|
||
interfaces everything else codes against — `packages/core/src/permissions.ts`
|
||
(the RBAC model) and `apps/api/src/lib/mutation.ts` (the write path) — are
|
||
settled; read them before adding any write.
|
||
|
||
**The highest-value work now, in order:**
|
||
|
||
1. **Mount `createReadGuardRoutes`.** The read half of the permission model is
|
||
written, tabulated and tested, and does nothing, because `app.ts` never
|
||
mounts it. Until it does, every authenticated member can read supplier cost
|
||
and margin. It is one line, and it must be registered *before* the handlers
|
||
it guards — Hono runs matched handlers in registration order.
|
||
2. **Mount `learn.ts`.** `/learn` is in the navigation and its API answers 404.
|
||
3. **Enqueue the other six agent task kinds.** The worker is complete; only
|
||
`enrich_account` and `enrich_contact` are ever written to `agent_tasks`, so
|
||
Piggy does far less than the ontology implies.
|
||
4. **Mount the HubSpot routes, or delete them.** Seven tables, OAuth, sync jobs
|
||
and webhook verification, all written, tested and unreachable.
|
||
5. **Give Piggy's writes their notification.** A stage change made through the
|
||
API raises a Slack notification; the same change made in chat does not,
|
||
because `write-tools.ts` passes no `NotificationOutbox` — it runs in the
|
||
Piggy process and the outbox is wired in the API server. The other open
|
||
Piggy items are listed under *Left to do* in the build plan.
|
||
|
||
`app.ts` is the one shared file. If your change needs a route mounted, a public
|
||
path allowlisted or a schema widened there, say so rather than racing another
|
||
agent for it.
|
||
|
||
---
|
||
|
||
## 9. What not to do
|
||
|
||
- Do not copy component files out of other people's repositories. Where a
|
||
primitive is a shadcn/ui original, take it from upstream, where it is
|
||
canonical and current. Compositions we write ourselves.
|
||
- Do not open self-registration on the identity provider. It is shared with
|
||
another application. PIG mints accounts itself, gated on an invite.
|
||
- Do not put a production SSH key on the CI runner. Deployment is manual on
|
||
purpose.
|
||
- Do not weaken the production guard that refuses to start without an identity
|
||
provider.
|
||
- Do not seed or infer email addresses for real people.
|