karti
c821b2ca07
CI / verify (push) Successful in 2m55s
The seam existed with only a Supabase implementation, so an on-prem deployment had no way to authenticate. A customer running PIG inside their own network already has Okta, Entra, Keycloak, Auth0 or Google Workspace; asking them to stand up a second identity system is a serious adoption tax and in a regulated environment usually refused outright. Setting PIG_OIDC_ISSUER is normally the whole configuration — the JWKS is discovered from the issuer's well-known document. PIG_OIDC_JWKS_URI skips discovery entirely for an air-gapped network. OIDC takes precedence over Supabase so an on-prem install can leave the hosted values in its environment file without them quietly taking over. Three decisions worth stating: Discovery is resolved lazily and the FAILURE is not cached. Doing it per request would put the customer's identity provider on the critical path of every API call; doing it eagerly at boot would mean their IdP rebooting takes the CRM down with it. So it happens on first use and retries on the next request. The audience check is optional but warned about loudly. Without it, a token the provider issued for ANY other application in the same tenant verifies here — a token minted for an unrelated internal tool would be accepted as a PIG session. It cannot be mandatory because some providers legitimately issue single-audience tokens. Email falls back through email, preferred_username and upn, because providers disagree, but a preferred_username without an "@" is ignored — PIG keys membership on the address, and a bare username must never become an account identity. Also fixed a warning that claimed "authentication is DISABLED" on a correctly configured OIDC deployment. That is worse than silence: an operator who reads it on a secure install learns to ignore the warnings. The dev bypass itself was already correct — it keys on the resolved provider rather than on Supabase. 18 new tests, most of them about what the provider must REFUSE: a foreign signing key, a foreign issuer, a token for a different application, an expired token, a token with no subject, and a discovery outage that must not become permanent. Keys are generated per test and the JWKS is served locally, so they run offline. Verified: production refuses to start with neither provider, starts with OIDC alone, enforces 401 on an unauthenticated request, and warns only about the genuinely missing admin list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
261 lines
11 KiB
Markdown
261 lines
11 KiB
Markdown
# Working on PIG
|
||
|
||
Onboarding for an agent or engineer joining this codebase cold. Read this
|
||
first, then [`docs/build-plan.md`](./docs/build-plan.md) for what to build.
|
||
|
||
Named `AGENTS.md` because that is the file coding agents look for by
|
||
convention. Everything here applies equally to humans.
|
||
|
||
---
|
||
|
||
## 1. What PIG is, in three sentences
|
||
|
||
A company that aggregates GPU capacity and resells it runs **two pipelines**,
|
||
and its business is the spread between them. Generic CRMs model one pipeline
|
||
against companies — they have no concept of inventory, no concept of a
|
||
commitment you already paid for, and so cannot answer the only question that
|
||
matters: *which contracted capacity is sold, to whom, at what margin, and what
|
||
is idle right now.*
|
||
|
||
PIG is built around that question, it is Apache 2.0, and it is designed to be
|
||
self-hosted by the customer.
|
||
|
||
**The load-bearing table is `allocations`**, which joins a `capacity_commitment`
|
||
(what we bought, at a known cost) to a `demand_deal` (what we sold, at a known
|
||
price). Margin, utilisation and idle capacity all fall out of that one join.
|
||
Everything else is plumbing that exists to keep that ledger honest.
|
||
|
||
---
|
||
|
||
## 2. Orientation
|
||
|
||
```
|
||
packages/core Ontology (stages, tiers, enums) + margin arithmetic + palette
|
||
packages/db Drizzle schema, migrations, seeds
|
||
packages/prime Typed client for the Prime Intellect compute API
|
||
apps/api Hono HTTP API, auth, capacity service
|
||
apps/web React + Vite + Tailwind + shadcn-idiom components
|
||
apps/mcp MCP server (stdio) — 9 tools
|
||
docs/ ontology.md, build-plan.md, agents.md, deploy.md, seed-data.md
|
||
```
|
||
|
||
~11,000 lines. 39 tests. Node 22+.
|
||
|
||
| | |
|
||
|---|---|
|
||
| Repo | `PIG/pig` on git.karti.ai (Gitea) |
|
||
| Live | https://primeintellectgrowth.com |
|
||
| CI | Gitea Actions, `.gitea/workflows/ci.yml`, ~2 min, must stay green |
|
||
| Deploy | `bash scripts/deploy.sh` on the host — deliberately manual |
|
||
|
||
---
|
||
|
||
## 3. Running it
|
||
|
||
```bash
|
||
npm install
|
||
|
||
# Postgres. PIG needs its own database — never point it at a shared one.
|
||
docker run -d --name pig-dev -p 5432:5432 \
|
||
-e POSTGRES_USER=pig -e POSTGRES_PASSWORD=pig -e POSTGRES_DB=pig \
|
||
postgres:16-alpine
|
||
|
||
export DATABASE_URL=postgres://pig:pig@localhost:5432/pig
|
||
npm run db:migrate
|
||
npm run db:seed # sourced, cited people — optional
|
||
npm run db:demo # a plausible demo book — optional, prefixed "DEMO — "
|
||
|
||
npm run dev:api # :8920
|
||
npm run dev:web # :5173, proxies /api to 8920
|
||
```
|
||
|
||
With no `SUPABASE_URL` set, **authentication is disabled in development** and
|
||
every request runs as the first user in the table. `loadConfig` refuses to start
|
||
in production without it, so this cannot leak.
|
||
|
||
Before pushing:
|
||
|
||
```bash
|
||
npm run typecheck && npm test
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Rules that must not be broken
|
||
|
||
These are load-bearing. Breaking one produces a subtle failure, not an error.
|
||
|
||
**Intelligence never lives in the API.** Handlers validate, authorize, call a
|
||
service, serialise. Research, enrichment, scoring and matching heuristics live
|
||
in the service layer or in the agent. The API signals the agent by *writing a
|
||
row to `agent_tasks`*, never by calling it — so the queue survives the agent
|
||
being down and no request thread ever blocks on a model.
|
||
|
||
**There are two auth providers, behind one interface.** Supabase for the
|
||
hosted deployment, OIDC for on-premises — see `apps/api/src/lib/auth-provider.ts`.
|
||
Both reduce to "verify a bearer token, return a subject and an email", because
|
||
that is all PIG needs. Never reach for a provider SDK outside that file.
|
||
|
||
**Authentication is not authorization.** A verified JWT proves someone has an
|
||
account in an identity provider that PIG *shares with another application*. It
|
||
does not prove they belong here. Access requires a row in PIG's own `users`
|
||
table. A token without one gets `403 needs_profile`, which the front end turns
|
||
into the join flow — never a login screen they have already completed.
|
||
|
||
**Cost is charged against the full commitment, not the hours that sold.**
|
||
Unsold hours are already paid for. Any other treatment reports a healthy margin
|
||
on a block that is losing money, which is precisely the failure PIG exists to
|
||
prevent. There is a test pinning this; if you "fix" it, read the test first.
|
||
|
||
**Sold and held are different.** A live hold removes capacity from everyone
|
||
else's availability — otherwise two sellers promise the same GPUs — but it is
|
||
not revenue and must never count toward utilisation.
|
||
|
||
**Money is integer cents.** Never floats. Round, never truncate: 2.43 is
|
||
2.4299999 in binary and a lost cent compounds across millions of GPU-hours.
|
||
|
||
**Agent claims carry evidence.** Enrichment writes to `facts` with a score, a
|
||
band, a source URL and a status — never directly to the record. Only `verified`
|
||
self-applies; anything weaker waits for a human. An agent allowed to write
|
||
unattributed claims will eventually write a wrong one and nobody will be able
|
||
to tell which.
|
||
|
||
**Never invent data about real people.** Seed records carry a confidence grade
|
||
and a source URL, both shown in the UI. No email address is ever inferred.
|
||
Authorship is not employment — contributors, residency participants and alumni
|
||
are recorded as what the evidence shows. Demo data is prefixed `DEMO — ` and
|
||
its customers are fictional.
|
||
|
||
---
|
||
|
||
## 5. Traps that have already bitten
|
||
|
||
Each of these cost real time. None produced an error message.
|
||
|
||
**`onConflictDoNothing()` is a no-op without a matching unique constraint.**
|
||
It has silently duplicated seed data twice. If there is no unique index to
|
||
conflict on, do an existence check instead.
|
||
|
||
**`z.coerce.boolean()` turns the string `"false"` into `true`.** Every feature
|
||
flag set to false was silently on. Use the `envBoolean` helper in
|
||
`apps/api/src/lib/config.ts`.
|
||
|
||
**Grid and flex children need `min-w-0`.** They default to
|
||
`min-width: auto`, meaning they refuse to shrink below their content — and a
|
||
`tabular-nums` figure, a `whitespace-nowrap` badge or a `truncate` title is all
|
||
it takes. The page then scrolls sideways on a phone and nothing reports an
|
||
error. This was fixed three separate times at individual call sites before
|
||
`Card` was given `min-w-0` on its base class; **any new container primitive
|
||
needs the same**. Check with:
|
||
|
||
```js
|
||
document.documentElement.scrollWidth - document.documentElement.clientWidth
|
||
```
|
||
|
||
It should be 0 on every route at 393px wide.
|
||
|
||
**Drizzle-generated migrations are not always valid SQL.** A `jsonb → integer`
|
||
cast was emitted without the `USING` clause Postgres requires. Always apply a
|
||
new migration to a real empty database before pushing — CI does this, but find
|
||
out before CI does.
|
||
|
||
**The proxy allows exactly one inline script, by hash.** `index.html` carries a
|
||
pre-paint theme script that prevents a white flash for dark-mode users. Editing
|
||
it changes its hash and the browser silently blocks it. CI asserts the hash;
|
||
if it fails, update the CSP in `deploy/Caddyfile.example`, on the server, and
|
||
the expected value in the workflow.
|
||
|
||
**`prices.onDemand` from the Prime Intellect API is the TOTAL FOR THE NODE.**
|
||
Verified: 1× A100 at 1.79, 2× A100 at 3.58. `gpuMemory` is likewise a node
|
||
total. There is an open bug for this — the mapper currently stores both as if
|
||
per-GPU, so an 8-GPU node reads eight times too expensive.
|
||
|
||
**The SPA fallback must never answer an `/api/` path.** Without an explicit
|
||
guard, an unknown API route returns `200 text/html` — the app shell — and the
|
||
caller sees `response.ok === true` before failing on `JSON.parse` with
|
||
"Unexpected token '<'", a long way from the cause. Both the static-file
|
||
middleware and the SPA fallback in `apps/api/src/server.ts` carry the guard;
|
||
anything added after them needs it too.
|
||
|
||
**Prime Intellect has two API hosts.** `api.primeintellect.ai` is compute and
|
||
pods. Inference is `api.pinference.ai/api/v1`, OpenAI-compatible.
|
||
|
||
**Piggy's default model thinks aloud.** `nvidia/nemotron-3-nano-30b-a3b` is a
|
||
hybrid reasoning model; under a tight `max_tokens` it rambles and truncates.
|
||
Pass `reasoning_effort: "none"` for tool use, routing and extraction.
|
||
|
||
**Deployment traps** live in `deploy/README.md` — chiefly that every Caddy site
|
||
block on that host needs `bind 10.0.0.2`, and that the CI runner uses
|
||
`container.network: host` so dependencies must be published on `127.0.0.1`.
|
||
|
||
---
|
||
|
||
## 6. Conventions
|
||
|
||
**Comments explain *why*, never *what*.** The code says what it does. Comments
|
||
carry the reasoning that would otherwise be lost — why this treatment and not
|
||
the obvious one, what breaks if it changes, what was tried and rejected. Match
|
||
the density already in the file; do not add narration.
|
||
|
||
**Match the surrounding style.** British spelling in prose and comments
|
||
(`utilisation`, `normalise`). Types are explicit at module boundaries. No
|
||
default exports.
|
||
|
||
**Enums come from `@pig/core`.** Never retype a stage list or a tier into a zod
|
||
schema — import it, so removing a value stops validating rather than silently
|
||
persisting.
|
||
|
||
**Tests pin decisions, not implementations.** The valuable cases are the ones
|
||
that would pass under a plausible-but-wrong version. Look at
|
||
`packages/core/test/margin.test.ts` for the register.
|
||
|
||
**Commits explain the reasoning**, including what was tried and rejected, and
|
||
say plainly when something was found by running the code rather than reading
|
||
it. Read `git log` for the register.
|
||
|
||
**Verify by running, not by assuming.** Several of the worst defects here were
|
||
found only by fetching a URL from another machine or applying a migration to a
|
||
real database. "It should work" has been wrong repeatedly.
|
||
|
||
---
|
||
|
||
## 7. Where to start
|
||
|
||
[`docs/build-plan.md`](./docs/build-plan.md) has 24 tasks in three waves with
|
||
real dependency edges.
|
||
|
||
**Do these first, alone, before anything fans out:**
|
||
|
||
- **F1** — install the shadcn primitive set
|
||
- **F3** — the RBAC permission model
|
||
- **F2** — the shared API write-path convention (needs F3 to call into)
|
||
|
||
They are small and they are the interface every other track codes against.
|
||
Starting parallel work before they settle is how it turns into merge conflict.
|
||
|
||
**Then the two that unblock a demo:**
|
||
|
||
- **A1** — allocation and commitment write paths. Today the core table can only
|
||
be populated by seed, so a visitor can look at the demo book but cannot enter
|
||
a deal of their own.
|
||
- **A2** — API keys. Nothing mints one, so the MCP server — the headline
|
||
feature — is unreachable in production.
|
||
|
||
**A4 (Piggy) is fully independent** and can start immediately alongside the
|
||
foundation. It touches no UI and no shared API conventions.
|
||
|
||
---
|
||
|
||
## 8. What not to do
|
||
|
||
- Do not copy component files from `trycompai/crm`. Most are shadcn/ui
|
||
originals — take them from upstream where they are canonical. Borrow the
|
||
compositions as ideas; the debt is credited in `NOTICE`.
|
||
- Do not open self-registration on the identity provider. It is shared with
|
||
another application. PIG mints accounts itself, gated on an invite.
|
||
- Do not put a production SSH key on the CI runner. Deployment is manual on
|
||
purpose.
|
||
- Do not weaken the production guard that refuses to start without an identity
|
||
provider.
|
||
- Do not seed or infer email addresses for real people.
|