PIG/pig
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
10d79fc35d34a1a31e801cd0a81195470a54e60f
pig/docs/build-plan.md
T
karti 13dec6b4b8
CI / verify (push) Successful in 3m45s
Details
CI / publish (push) Has been skipped
Details
Rebuild the shell, add Calendar and Learn, and govern reads 
Seven parallel agents and an adversarial verification pass. The three things
worth knowing before reading the diff:

RBAC WAS ALREADY BUILT. docs/build-plan.md marks F2 and F3 outstanding and is
stale — packages/core/src/permissions.ts and lib/mutation.ts shipped long ago.
So this does not rebuild them; it closes the gaps an audit found. The big one
is that reads were entirely ungoverned: every GET was "any authenticated
member", so a junior demand rep and a research contractor could both pull
per-block supplier cost and break-even prices from /api/capacity/margin, and
every contract's negotiated terms. For a company whose margin is the business,
that was the hole that mattered. Adds book:read / economics:read / team:read,
a readGuard middleware, and a `viewer` role below member.

THE BUTTON AND THE 403 DISAGREED — the exact thing F3 said must never happen.
Contracts.tsx never called can() at all, so its save button was always enabled
against a server requiring contract:sign; Capacity.tsx gated commitment
creation on deal:write/demand while the server wanted commitment:write/supply.

POST /api/activities was the one write bypassing executeMutation: no capability
check, and any member could mutate accounts.lastActivityAt as a side effect.
It is now a proper mutation() behind activity:write.

The shell becomes three panes — a collapsible shadcn sidebar with an account
switcher on the Piggy accent, a header with real search, and Piggy docked to
the right, page-aware and persistent across navigation. The phone keeps its
bottom tab bar, which is the thing this product already beat trycompai/crm on,
and gains the sidebar as a sheet.

Calendar is a projection over thirteen dated sources rather than a new table,
because a table would duplicate dates that already live on contracts, deals and
commitments and would drift — and one ledger answering the question is the
whole argument. It surfaces export_authorizations and compliance_artifacts,
which had indexed expires_at columns, schema comments saying they must be
alerted on, and no read endpoint or UI anywhere.

Learn carries two tracks. Concepts are members-only; the platform track can be
opened with a share code by someone with no account. The code mints a scoped
learn-only token and never a Principal — every route here resolves a principal
and then checks capabilities, so a principal-minting code would be one missing
check away from leaking the book. "Only platform-track rows may be code-visible"
is a database CHECK constraint as well as a write-path rule, and a test asserts
a valid learn token still gets 401 on /api/dashboard, /api/accounts and
/api/contracts — the same invariant scripts/deploy.sh refuses to ship without.

CD becomes tag-to-ship. CI publishes an image to the Gitea registry on a
release-* tag and cloud-2 pulls it, so no credential on the shared runner can
execute anything on production — by construction rather than by policy. Both
halves of deploy.sh's original rule survive: nothing on the runner reaches the
host, and a human still decides when it ships. deploy.sh gains a rollback and a
public-origin check, and PIG_IMAGE now reaches compose through `sudo env`,
without which sudo's env_reset silently resolved every release to pig:local.

Tests 141 -> 261.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 15:02:48 -07:00

225 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Build plan
Where PIG stands, audited against the tree rather than against the last version
of this document.
The original plan was 24 tasks in three waves with real dependency edges, so
that work could be handed to several people (or several agents) at once without
them colliding. **Every one of those tasks has shipped.** What follows is the
audit, then the work that is actually left — which is a different and shorter
list, and mostly not new code.
---
## Where PIG stands
Live at primeintellectgrowth.com. Around 45k lines of TypeScript including
tests, 261 unit tests across five packages plus a critical-path E2E suite,
green CI, 47 tables, 13 migrations.
- The ontology and margin engine, with the `allocations` join at the centre
- Both pipelines; capacity availability, matching, holds and idle alerts
- Write paths for accounts, contacts, both deal sides, commitments,
allocations, holds, contracts, calendar entries and activities
- RBAC: eleven capabilities resolved from team and role, enforced on every
write and shared with the browser — see the caveat under **Left to do**
- Auth: Supabase or any OIDC provider behind one interface; invite-gated
registration, profile creation, sign-out
- Import: CSV and .xlsx with mapping, dry-run preview and idempotent commit;
Notion and Google Sheets as OAuth sources onto the same mapping step
- Piggy: lease-based queue worker with `SKIP LOCKED` claims, renewable leases,
capped exponential backoff and `agent_runs`; plus a private read-only chat
server behind the API
- MCP server (9 tools, stdio), `pig` CLI, Prime Intellect client, demo dataset
- Slack and Buzz notification adapters behind one notifier interface
- Growth (customer lifecycle projection) and the GTM calendar
- Docker, Compose, Caddy, `deploy.sh` with rollback, and tag-to-ship CD
---
## Audit of the original plan
Verified by reading the tree on 2026-08-13. Every row was checked; none was
believed on the strength of the previous version of this file.
### Wave 0 — Foundation
| | Task | Status |
|---|---|---|
| **F1** | shadcn primitive set | **Done.** 22 primitives in `apps/web/src/components/ui`, including everything the plan listed |
| **F2** | Shared API write-path convention | **Done.** `apps/api/src/lib/mutation.ts` — ontology-derived zod schemas, one transaction per write, automatic activity logging, consistent error shape |
| **F3** | RBAC | **Done.** `packages/core/src/permissions.ts` — eleven capabilities, ranked roles, shared by API and browser. Reads were added later and are *not yet enforced*; see below |
### Wave 1
| | Task | Status |
|---|---|---|
| **A1** | Allocation + commitment write API | **Done.** `routes/capacity-writes.ts`, availability invariant enforced server-side |
| **A2** | API keys | **Done.** `routes/api-keys.ts` — mint, list, revoke; plaintext shown once |
| **A3** | Auth-provider seam | **Done.** `lib/auth-provider.ts`; OIDC is a full second implementation, not a stub |
| **A4** | Piggy | **Done.** `apps/piggy` — worker, queue, provider, tools. See the gap on task kinds below |
| **A5** | Slack adapter | **Done.** `routes/slack.ts` + `services/slack.ts`; signed-request verification, channel links, capacity slash command |
| **A6** | Buzz adapter | **Done.** `routes/buzz.ts`, same notifier interface, mounted only when `BUZZ_RELAY_URL` is set |
| **A7** | `pig` CLI | **Done.** `apps/cli` — `me`, `accounts`, `deals`, `commitments`, `allocations`, `capacity`, with `--json` |
| **A8** | Data table + ⌘K palette | **Done.** `components/DataTable.tsx`, `components/CommandPalette.tsx` |
| **A9** | `SourcedValue` + fact review | **Done.** `components/SourcedValue.tsx`, `pages/FactReview.tsx`, `routes/facts.ts` |
| **A10** | Contracts UI | **Done.** `pages/Contracts.tsx` over `contracts`, `sla_terms`, `sla_metric_targets`, `contract_obligations` |
| **A11** | Record create/edit sheets | **Done.** `components/RecordSheets.tsx` |
| **A12** | Capacity tiers | **Done.** `SECURITY_TIERS = ['government', 'secure_cloud', 'community_cloud']` with a rank comparison, so a requirement is satisfied only from at or above its tier |
| **A13** | Admin settings | **Done.** `routes/admin-settings.ts`, `components/AdminSettings.tsx` |
| **A14** | Import framework | **Done.** `routes/imports.ts`, `services/tabular-import.ts` — CSV and .xlsx, mapping, preview, idempotent commit, gated on `data:import` |
| **A15** | Notion import | **Done.** `routes/notion-import.ts` |
| **A16** | Google Sheets import | **Done.** `routes/google-sheets.ts` |
### Wave 2
| | Task | Status |
|---|---|---|
| **B1** | Allocation UI | **Done.** `components/AllocationSheet.tsx`, reachable from the matcher |
| **B2** | Piggy chat UI | **Done.** `components/PiggyChat.tsx` + `PiggyDock.tsx`, streaming with separate reasoning and tool-call events |
| **B3** | Slack/Buzz connection settings | **Done.** `components/IntegrationSettings.tsx`, `routes/integration-settings.ts` |
| **B4** | Critical-path E2E | **Done.** `apps/api/e2e/critical-path.test.ts`, run by CI against a real Postgres |
### Built since, and not in the original plan
| Feature | Where |
|---|---|
| Read-authorisation policy table and governance test | `routes/read-guards.ts`, `lib/read-guard.ts` — **written, tested, not mounted** |
| GTM calendar: thirteen event kinds across nine tables, three-month timeline | `routes/calendar.ts`, `services/calendar.ts`, `pages/Calendar.tsx`. This is what finally surfaced `export_authorizations` and `compliance_artifacts`, which had indexed `expires_at` columns and no UI at all |
| Growth / customer lifecycle projection | `services/customer-lifecycle.ts`, `pages/Growth.tsx` |
| Learn: member curriculum plus a code-gated public track | `routes/learn.ts` (**not mounted**), `pages/Learn.tsx` |
| HubSpot: OAuth, connections, sync jobs, webhooks, seven tables | `routes/hubspot.ts`, `routes/hubspot-webhook.ts` (**neither mounted**) |
| Notification outbox | `services/notification-outbox.ts` |
| Tag-to-ship CD with a host-side release poller and rollback | `.gitea/workflows/ci.yml`, `scripts/autodeploy.sh`, `deploy/pig-autodeploy.*` |
| Three-pane application shell with a docked agent | `components/Shell.tsx`, `AppHeader.tsx`, `AppSidebar.tsx` |
---
## Left to do
In rough order of value. The first four are all "wire up something that already
exists", which is a strange shape for a backlog and worth taking seriously
because that is exactly the kind of work that stays undone.
**1. Mount `createReadGuardRoutes`.** The read half of the permission model —
`book:read`, `economics:read`, `team:read` — has a policy table, middleware, and
a governance test that fails when a GET appears with no rule covering it. None
of it runs, because `app.ts` never mounts it. Until it does, any authenticated
member reads supplier cost, break-even price and every margin total regardless
of team or role. Registration order is load-bearing: Hono runs matched handlers
in the order they were registered, so the guard must be mounted before the
handlers it guards.
**2. Mount `learn.ts`.** `/learn` is in the navigation and every one of its API
paths answers 404.
**3. Enqueue the other six agent task kinds.** `AGENT_TASK_KINDS` declares
eight. Only `enrich_account` and `enrich_contact` are ever written to
`agent_tasks`, both from record creation in `routes/records.ts`. `write_brief`,
`match_capacity`, `detect_idle_capacity`, `summarise_pipeline`, `watch_renewal`
and `research_supplier` have no producer anywhere. The worker is generic and
complete; the gap is entirely on the enqueue side, and each one is a few lines
in the service that already computes the underlying answer.
**4. Mount the HubSpot routes, or delete them.** Seven `hubspot_*` tables, an
OAuth flow, sync cursors, jobs and a verified webhook endpoint, all written,
all tested, all unreachable. Whichever way this goes it should not stay in this
state — dead-but-tested code reads as shipped to anyone grepping the repo.
**5. Row-level or team-scoped reads.** Every read returns the whole book. This
is why read capabilities are platform-wide, and it is the honest reason the
permission model says so out loud. It is also the thing to build before PIG
serves a company where that is not acceptable.
**6. Piggy writes.** Interactive chat is read-only by design for now. The
queue-side agent writes only to `facts`. A write path for the chat agent needs
the same evidence discipline plus a confirmation step, and should not be added
casually.
**7. Fill in `.env.example`.** `POSTGRES_PASSWORD` and
`PIG_SETTINGS_ENCRYPTION_KEY` are both load-bearing and both missing from it.
`ANTHROPIC_API_KEY` is declared in `apps/api/src/lib/config.ts` and read by
nothing — remove it or use it.
**8. A remote MCP transport.** The server is stdio only; there is no
Streamable HTTP transport and no `/mcp` endpoint on the API, so every user runs
the server locally. The package is also unpublished, so `npx @pig/mcp` does not
work and the documented install command has to be a path into a clone.
Not started at all, and deliberately: email or calendar ingestion, forecasting,
quota and attainment, invoicing or billing reconciliation, multi-tenancy, and
any native mobile application.
---
## On borrowing from Comp AI CRM
Their repo (MIT) was cloned and inventoried early on, and three findings shaped
the plan. All three have since been acted on.
**Their component library was far deeper — 68 primitives to our 9 at the time.**
PIG now has 22, including the ones that mattered: `data-table`, `command`,
`sheet`, `drawer`, `sidebar`, `form`. The agent-chat compositions
(`message`, `reasoning`, `thinking-indicator`) were rebuilt rather than copied,
inside `PiggyChat.tsx`.
**`SourcedValue` / `Provenance` was worth adopting outright** — a dotted
underline on any agent-derived value, with a tooltip carrying the claim, the
reasons, when it was observed and the source URL. PIG already held that data in
`facts` and surfaced none of it. It does now.
**We were ahead of them on mobile, not behind**, and the ratio has held: PIG is
63 `.tsx` files with safe-area handling, a bottom tab bar, a sidebar Sheet and a
hard rule that no route may scroll sideways at 393px. Their app is effectively
desktop-only.
**Do not copy their component files.** Most are shadcn/ui originals, which are
MIT and designed to be installed from upstream — take them from source, where
they are canonical and current. Borrow the compositions as ideas, and credit in
`NOTICE` as already done.
---
## Prime Intellect API — verified facts
Confirmed against the live API, not assumed.
**Two different hosts.** `api.primeintellect.ai` is the compute/pods API
(availability, pods, billing). Inference is `api.pinference.ai/api/v1`, which
is OpenAI-compatible (`/chat/completions`, `/models`, and an Anthropic-style
`/messages`). PIG's config carries both separately — `PRIME_API_BASE` and
`PIGGY_INFERENCE_BASE`.
**`prices.onDemand` is the TOTAL FOR THE NODE, not per-GPU.** Verified:
datacrunch lists 1× A100 at 1.79 and 2× A100 at 3.58. `gpuMemory` is likewise a
node total (640 for 8× 80GB). This was a real bug — an 8-GPU node read eight
times too expensive — and is **fixed**: `packages/prime/src/map.ts` divides both
by `gpuCount` at the boundary and keeps the node totals in `raw` for
reconciliation. Anything new that reads an upstream price must do the same.
**Piggy's default model:** `nvidia/nemotron-3-nano-30b-a3b` ($0.05/$0.20 per
Mtok). It is a *hybrid reasoning* model that thinks aloud by default and will
ramble or truncate under a tight `max_tokens`. Pass **`reasoning_effort:
"none"`** for tool use, routing, extraction and classification — roughly one
second, terse output. Leave reasoning on only for genuine maths or logic, where
it arrives in a separate `reasoning_content` field while `content` stays clean.
**Billing** is pay-as-you-go against a shared balance, not a per-model
whitelist. Check with `GET /api/v1/billing/wallet`.
**The user-agent concern did not reproduce** — both a browser UA and PIG's own
returned 200. Worth remembering it was once an issue if a 403 ever appears.
## Open questions
- **Which inference host for on-prem?** Settled in shape: the model *name* is
admin-selectable at runtime, the *host* is `PIGGY_INFERENCE_BASE` in the
environment. What is untested is a customer pointing it at their own
OpenAI-compatible endpoint.
- **Who may import?** Currently team admins and platform admins
(`data:import`, minimum role `admin`, all teams). Easy to loosen, unpleasant
to tighten after the fact.
- **Which key does PIG get?** The existing Prime Intellect key is broad and
never expires. PIG's sync should hold a separate one scoped to
`Availability → Read`.
Reference in New Issue View Git Blame Copy Permalink