pig/deploy/README.md

# Deploying PIG

PIG is an API/web container, Postgres, and — when you ask for it — a private
Piggy container running a Prime Agent session over the CRM, behind any reverse
proxy that terminates TLS. Nothing here is specific to a particular host.

## 1. DNS

Point the apex and `www` at the machine. Both must resolve before the proxy can
obtain a certificate.

```
A   primeintellectgrowth.com       -> <your IP>
A   www.primeintellectgrowth.com   -> <your IP>
```

## 2. Configuration

```bash
cp .env.example .env    # then edit
```

The values that must be set for a production start:

| Variable | Why |
|---|---|
| `POSTGRES_PASSWORD` | Generate a fresh one; never reuse another service's |
| `PIG_PUBLIC_URL` | The single origin the app is served from |
| `SUPABASE_URL` / `SUPABASE_ANON_KEY` | Authentication. The app refuses to start in production without a Supabase URL, because it would otherwise serve the whole CRM unauthenticated |
| `PIG_ADMIN_EMAILS` | Who may administer. **Every address here must already have an account** — an unregistered address listed as an admin is a standing offer of admin rights to whoever claims it first |
| `PIG_SETTINGS_ENCRYPTION_KEY` | Base64-encoded 32 bytes (`openssl rand -base64 32`). Only needed for the Notion and Google OAuth secrets typed into the admin UI, which the API refuses to store without it |

Optional: `PRIME_API_KEY`, the Slack and Buzz credentials, and the whole Piggy
block — the agent is off unless you [turn it on](#turning-piggy-on).

`PRIME_API_KEY` is now one key with two jobs: the API syncs GPU availability
from `api.primeintellect.ai` with it, and Piggy calls models on
`api.pinference.ai` with it. Scope it to `Availability → Read` plus inference,
and nothing that can provision. Piggy still accepts `PIGGY_INFERENCE_API_KEY`
as an alias for the same value, so a host configured before the agent moved onto
Prime Inference keeps starting untouched.

Piggy listens on `piggy:8931` inside the Compose network. The port is exposed to
other containers but never published to the host, and Caddy must not route to
it. The CRM API authenticates the user, forwards only bounded chat context, and
uses `PIGGY_INTERNAL_TOKEN` in an Authorization header. Never put that token in
a query string, where proxies and access logs can retain it.

## 3. Start

```bash
docker compose -p pig up -d db
docker compose -p pig run --rm --no-deps app pnpm exec tsx packages/db/src/migrate.ts
docker compose -p pig up -d --build app
docker compose -p pig run --rm --no-deps app pnpm exec tsx packages/db/src/seed/index.ts   # optional
```

**Migrate from a one-off container, before the app starts — not with `exec`.**
`exec` needs a running app to attach to, and a release that queries a table its
migration has not yet created crash-loops before you can attach to it. You then
have a container restarting every few seconds and no way in. `run --rm
--no-deps` uses the same image without the app, and without starting its
dependencies twice. This is what commit d4d7095 changed and it is what
`scripts/deploy.sh` does.

That starts the CRM alone: the agent is behind a Compose profile and stays
down. To run it, see [Turning Piggy on](#turning-piggy-on) — set the switch in
`.env` rather than starting the container by hand, because a hand-started Piggy
is one no later deploy knows to upgrade.

Use `-p pig`. A compose project that shares a name with a neighbouring stack
will adopt its volumes, which is a memorable way to lose a database.

## 4. Reverse proxy

See `Caddyfile.example`. Serve the app and API from the **same** origin.

Two things that will otherwise cost you an hour:

- **If other sites on the host use `bind <address>`, yours must too.** Caddy
  groups site blocks into servers by listen address. A block without `bind`
  lands in a *separate* server on `:443`, and the more specific listener wins
  for traffic arriving on that address — which is all public traffic after NAT.
  The symptom is a valid certificate, a 200 response, an empty body, and none
  of your headers. It looks like the app is broken; it is that the request
  never reached it.

- **The CSP must carry the hash of the inline theme script** in `index.html`.
  That script sets light or dark before first paint so dark-mode users do not
  get a white flash. Editing it changes the hash and CSP will silently block
  it — the browser console prints the hash it expects.

  **That hash exists in three places, and only two of them are checked.**

  | Copy | Checked by |
  |---|---|
  | `.gitea/workflows/ci.yml` (the `expected` constant) | itself, on every run |
  | `deploy/Caddyfile.example` | nothing — it is an example |
  | **the live `Caddyfile` on the host** | **nothing at all** |

  The live one is the only copy that decides whether a browser runs the script.
  Nothing in this repository can see it, CI cannot fail on it, and the failure
  is a white flash for dark-mode users with no error anywhere. Editing that
  script means editing all three by hand and reloading Caddy.

## 5. Verify

```bash
curl -s https://primeintellectgrowth.com/api/health
# {"ok":true,"service":"pig","version":"0.1.0"}
```

Check the **public origin**, not just `127.0.0.1:8920`. The `bind` failure
above answers with a valid certificate, HTTP 200 and an empty body, which
satisfies every check that only asks whether something responded.
`scripts/deploy.sh` now asserts the body is non-empty and contains the
application's mount point for this reason.

## Turning Piggy on

Piggy is the in-app agent, and it is worth knowing what it is before you run it.

It is a **Prime Agent session** — Prime Intellect's own agent harness,
`@earendil-works/pi-coding-agent`, embedded as a library rather than shelled out
to — holding **PIG's CRM tools and nothing else**. The harness is constructed
with every built-in tool disabled (`noTools: 'all'`) and an explicit allowlist
on top, so the model has **no shell, no filesystem access and no Python**. The
live tool list is compared against that allowlist when a session starts and a
mismatch is a startup error, so a future harness release cannot quietly widen
it.

Three things follow for an operator:

- **It writes.** `PIGGY_AGENT_MODE` decides how: `read_only`, `confirm`
  (default — a change is proposed as a card and applied when a person clicks) or
  `auto`. Contracts, commitments, allocations and compliance records always
  require a click regardless. Every write runs as the calling user's own
  principal, so Piggy cannot reach a record its user could not, and the audit
  trail names the human.
- **The model is chosen per conversation**, from a five-model picker defined in
  `apps/piggy/src/agent/models.json`. `PIGGY_AGENT_MODEL` is only the default
  for a user who has not chosen.
- **Conversations are persisted** in `piggy_conversations` and `piggy_messages`
  (migration 0014), so an agent turn now depends on the schema being current.
  `scripts/deploy.sh` migrates from a one-off container before it starts either
  container, which is what keeps that true.

It is **off by default** and nothing about it is configurable from the admin UI
— every value below is read once, when the container boots.

Three keys in `.env`, and all three are needed:

```bash
PIGGY_ENABLED=true
PRIME_API_KEY=<a Prime Intellect key with inference; PIGGY_INFERENCE_API_KEY
               is still accepted as the legacy alias for the same value>
PIGGY_INTERNAL_TOKEN=<openssl rand -hex 32>
```

The fourth thing the API needs, `PIGGY_INTERNAL_URL`, is already set to
`http://piggy:8931` by `docker-compose.yml`. Set it in `.env` only for a Piggy
running outside Compose.

Then deploy as usual:

```bash
bash scripts/deploy.sh
```

**Do not start the container by hand.** The service carries
`profiles: ['piggy']`, and compose skips a profile-gated service *silently*:
without the profile, `pull`, `build` and `up` behave as though it were not in
the file, with no warning and a zero exit.
`deploy.sh` reads `PIGGY_ENABLED` from `.env` and adds `--profile piggy`
to the pull, the build, the `up` **and the rollback**, so the agent moves with
the app. A Piggy started once with `--profile piggy up -d` and then forgotten is
not covered by any of that: it keeps running the image of the day it was
started, against a schema several migrations newer, which is the failure the
comment at the top of `docker-compose.yml` is about. `deploy.sh` therefore
compares the piggy container's image ID with the release's and rolls back if
they differ.

`scripts/autodeploy.sh` compares both containers' digests for the same reason.
Without that, an old Piggy is invisible to the poller: the app matches the
newest tag, the poller says "up to date" every five minutes, and the agent runs
last month's code indefinitely.

### A missing API key crash-loops the worker

`PRIME_API_KEY` (or its alias `PIGGY_INFERENCE_API_KEY`) is required by
`apps/piggy/src/config.ts`. Without either the process exits at boot with
`Invalid Piggy configuration: PRIME_API_KEY ... is required.`, and
`restart: unless-stopped` starts it again — so the symptom is a container
restarting every few seconds, not an error anyone sees in the CRM.
`PIGGY_INTERNAL_TOKEN` shorter than 32 characters fails the same way, and so
does a `PIGGY_AGENT_MODEL` that is not one of the five ids in
`apps/piggy/src/agent/models.json` — rejected at boot on purpose, because the
alternative is a model that 404s on a user's first question.

`deploy.sh` catches all of them: it waits for the container to report healthy
and exits 3 if it does not, deliberately **without** rolling back, because the
previous image reads the same `.env` and would fail identically.

**A blank line is not an absent one, and here it is actively misleading.**
`PRIME_API_KEY` and `PIGGY_INFERENCE_API_KEY` are two spellings of one key, and
each is declared `.min(1).optional()`. Compose passes a blank `.env` line
through as the empty string, so a file that sets `PRIME_API_KEY` correctly *and*
carries a leftover empty `PIGGY_INFERENCE_API_KEY=` line crash-loops Piggy with:

```
Invalid Piggy configuration:
  PIGGY_INFERENCE_API_KEY: String must contain at least 1 character(s)
```

— a message about the key you did not use. **Comment the unused spelling out.**
Before enabling Piggy on an existing host, check for exactly this:

```bash
grep -nE '^(PRIME_API_KEY|PIGGY_INFERENCE_API_KEY)=$' /opt/pig/.env
```

Any line that prints is one to comment out.

### The thinking-level trap — read this before changing the model

The single setting most likely to make a working deployment look broken.

The harness defaults `thinkingLevel` to `medium`, which is tuned for a coding
agent. On the default model that produced **6,195 output tokens of reasoning and
an empty answer**: the turn hit its token ceiling while still thinking and came
back with `finish_reason: length`. `low` was worse. `off` is Piggy's default, and
for the nemotron models it maps to the endpoint's `reasoning_effort: none` — the
same question then answered correctly in **149 output tokens**.

The mapping is **per model** and lives in `thinkingLevelMap` in
`apps/piggy/src/agent/models.json`. The two nemotron entries have one; deepseek,
opus and gpt-5.6 do not, and for them `off` omits `reasoning_effort` entirely so
the endpoint's own default applies.

So if you change `PIGGY_AGENT_MODEL` and start getting empty answers, truncated
answers or a surprising bill, this is where to look — not at the agent, the
tools or the network. Give the new model a `thinkingLevelMap` before raising
`PIGGY_AGENT_THINKING`.

### Where the harness is allowed to look at the filesystem

`PIGGY_AGENT_DIR` is the harness's own directory. `docker-compose.yml` pins it
to `/var/lib/piggy-agent`, which the image creates owned by the unprivileged
`node` user at mode 0700. Leave it alone.

Two reasons it is not the default `~/.pig/piggy-agent`:

- **Writability.** Under `docker run` with `USER node`, `~` resolves to
  `/home/node` and works. That is incidental: a runtime that starts this image
  with a numeric user and no matching passwd entry (`runAsUser: 1000` under
  Kubernetes) leaves `HOME` unset, `os.homedir()` falls back to `/`, and the
  agent dies creating its directory — on the first turn, long after the deploy
  reported success.
- **Prompt containment.** The harness discovers extensions, skills and context
  files from its cwd, and Piggy hands it this directory as cwd. Point it at the
  checkout, or bind-mount a repository over it, and source files become
  reachable from a CRM agent's prompt. **Never bind-mount anything here.**
  `deploy.sh` reads the value back off the running container and refuses to
  report success if it sits inside `/app`.

The directory is **not persisted**, deliberately. Nothing in it is worth keeping
across a restart: Piggy rewrites `models.json` there from the image at every
boot, the credential store is in-memory by design, sessions are in-memory, and
the conversations live in Postgres. A cold start costs nothing measurable, and a
volume would only be a way for a file to outlive the image that wrote it.

### Health

The piggy container has its own healthcheck, against the chat server's
unauthenticated `GET /internal/health`:

```bash
docker compose -p pig --profile piggy ps
# NAME         STATUS
# pig-piggy-1  Up 2 minutes (healthy)

docker compose -p pig --profile piggy exec piggy \
  node -e "fetch('http://127.0.0.1:8931/internal/health').then(r=>r.text()).then(console.log)"
# {"ok":true,"service":"piggy-chat","model":"nvidia/nemotron-3-nano-30b-a3b"}
```

It needs its own because the image's `HEALTHCHECK` asks for
`127.0.0.1:8920/api/health` — the API's port, which this container does not
serve. Inherited unchanged, Piggy reported `unhealthy` for ever while working
perfectly.

There is no published port and there must not be one. The listener is reachable
only from inside the Compose network, the API authenticates the user before
forwarding anything, and the bearer token goes in a header — never a query
string, where a proxy or an access log would keep it.

### What the image carries for the agent

Two things about the production image are worth knowing before you debug a
container that will not start.

**`models.json` is a runtime file, not a compiled-in constant.**
`apps/piggy/src/agent/models.json` is read from disk at boot, validated, and
copied into the agent directory for the harness to register its provider from.
It reaches the image inside `COPY apps/piggy`, and the Dockerfile parses it
during the build so that a narrowed `COPY` or a new `.dockerignore` rule fails
there rather than at 03:00 in a crash loop.

**Production dependencies are installed with `--ignore-scripts`.** The Prime
Agent SDK drags in a large transitive tree, including `@google/genai` and
`protobufjs`; their install scripts — and esbuild's — are denied in
`pnpm-workspace.yaml` on purpose, so nothing a dependency pulls in can execute
code at install time. Piggy talks to exactly one provider over an
OpenAI-compatible API and none of that tree is on a path it executes. The
Dockerfile imports the SDK during the build to prove the scriptless install
still yields a loadable agent, and the container has been run end to end against
a live key: health, a tool call, and a correct answer.

### Tuning

The agent's own settings, all with defaults in `apps/piggy/src/config.ts`:

| Variable | Default | What it does |
|---|---|---|
| `PIGGY_AGENT_MODEL` | `nvidia/nemotron-3-nano-30b-a3b` | The default answer model. Must be one of the ids in `apps/piggy/src/agent/models.json`; anything else is refused at boot |
| `PIGGY_AGENT_MODE` | `confirm` | `read_only`, `confirm` or `auto`. Contracts, commitments, allocations and compliance always confirm regardless |
| `PIGGY_AGENT_THINKING` | `off` | See [the thinking-level trap](#the-thinking-level-trap--read-this-before-changing-the-model) before touching it |
| `PIGGY_AGENT_MAX_TOKENS` | `4096` | Output tokens per agent turn, reasoning included. Clamped down to the chosen model's own ceiling |
| `PIGGY_AGENT_DIR` | `~/.pig/piggy-agent` | Pinned to `/var/lib/piggy-agent` by `docker-compose.yml`. Do not override under Compose |

The pre-agent settings still apply to the queue worker and exist to be lowered:
`PIGGY_MAX_TOKENS` (per queued task), `PIGGY_CHAT_MAX_TOKENS` (per interactive
answer), `PIGGY_MAX_TURNS`, `PIGGY_POLL_INTERVAL_MS`, `PIGGY_LEASE_SECONDS`,
`PIGGY_REASONING_EFFORT` and the two `PIGGY_PRICE_*_CENTS_PER_MTOK` values that
make the cost recorded against each run exact. `.env.example` lists them
commented out, and that is not decoration: an empty `PIGGY_MAX_TOKENS=` line is
passed to the container as the empty string, which coerces to 0 and refuses to
start. Leave a key commented to get its default; do not leave it blank.

### The database is the only undo for an agent write

Piggy can modify CRM records, so a release that ships a broken write tool can
corrupt data no migration ever touched. `scripts/deploy.sh` dumps the database
before it migrates and before the new image starts, to `backups/` next to the
checkout:

```
backups/pig-20260814-030201.sql.gz
```

That directory is in `.gitignore`, so `autodeploy.sh`'s forced checkout at a
release tag leaves it alone. The dump is verified rather than assumed: the
script decompresses it and refuses to migrate if a database with tables in it
produced less than 4 KB of SQL, because `pg_dump | gzip` turns silence into a
plausible-looking 20-byte file and a deploy log that claims a backup was taken.

Nothing prunes these. They are the only copy — copy them off the host if the
data matters as much as the uptime does.

Changing any of them means restarting the container — `bash scripts/deploy.sh`,
or `docker compose -p pig --profile piggy up -d piggy` if the release is
otherwise unchanged.

## Learn videos

PIG hosts its own Learn videos. There is no video service to configure, no
embed host and — because a native `<video>` is not an iframe — nothing to add
to the proxy's `frame-src`. The files are covered by `default-src 'self'`.

### Where they live

| | |
|---|---|
| Host directory | `PIG_MEDIA_HOST_DIR`, normally `/opt/pig/media` |
| Inside the container | `/app/media`, bind-mounted **read-only** |
| Read by the app from | `PIG_MEDIA_DIR` (compose sets it to `/app/media`) |
| Served at | `/media/learn/<filename>` |
| From source, no container | `PIG_MEDIA_DIR=./media`, relative to the repository root |

```bash
sudo mkdir -p /opt/pig/media
sudo chown "$USER" /opt/pig/media
# then in /opt/pig/.env
#   PIG_MEDIA_HOST_DIR=/opt/pig/media
```

**Create the directory before `compose up`.** Docker creates a missing bind
source itself, as an empty directory owned by root — every video then 404s and
you cannot copy a file in without `sudo`.

The mount is read-only. PIG never writes a video: a file arrives by being
copied onto the host, so the write path is not reachable over HTTP at all.
Deploys do not touch the directory, and neither does a rollback — the videos
outlive any particular release.

### Naming: content-addressed, and why

A filename is `<slug>.<hash>.<ext>`, for example
`pig-tour.7f3a91c2.mp4`. Only `[A-Za-z0-9._-]` is accepted, with `mp4`, `webm`
or `m4v` as the extension; anything else is refused both as a stored source and
as a file read.

```bash
slug=pig-tour
hash=$(sha256sum "$slug.mp4" | cut -c1-8)
mv "$slug.mp4" "$slug.$hash.mp4"
```

The hash is load-bearing, not decoration:

**The media FILES are served unauthenticated. The LISTING is not.** Who learns
that a video exists — its title, its track, whether it is code-visible at all —
is decided by `/api/learn` and `/api/learn/public`. The bytes are handed to
anyone who can name the file. This is how every video platform works: a gated
manifest in front of segments on an open CDN. It is also what a `<video>`
element requires, since a media element re-requests byte ranges on every seek
and carries no bearer token while doing it.

*What it costs.* A URL, once shared, is a permanent public link to that video.
Someone given the share code can copy the `src` out of the page and post it,
and **rotating the Learn access code does not close it**. The remedies are to
rename the file (a new hash, therefore a new URL) or delete it. We accept that:
these are product demos meant to be shareable with the code. The material that
must never leak is the supply and demand concept tracks, and those are gated by
the listing — which is where the boundary genuinely is.

*What the hash buys.* There is no directory index and a wrong guess is a flat
404, so an unguessable name makes the exposure "whoever has the link" rather
than "the internet". That is precisely an unlisted video.

### Publishing one

```bash
cp pig-tour.7f3a91c2.mp4 /opt/pig/media/
pnpm db:demo -- --hosted     # inserts rows for the files that are present
```

The seed discovers files by slug, so the hash never has to be written into the
manifest. It is idempotent — the unique key on `(track, provider, external_id)`
does that work — and it inserts a row **only when the file is on disk**, because
a Learn card whose video 404s reads as a broken product rather than a missing
one. Rows for files that have since disappeared are reported, never deleted:
deleting them would empty the curriculum the first time someone ran the seed
with the media directory unmounted.

`pnpm db:demo -- --clear` removes the `DEMO — ` rows and leaves these alone.
`pnpm db:demo -- --clear-hosted` removes these and leaves the files on disk.

### Checking it

```bash
curl -sI https://primeintellectgrowth.com/media/learn/pig-tour.7f3a91c2.mp4
# accept-ranges: bytes          <- without this, the scrubber does nothing
curl -s -r 0-99 -o /dev/null -D - \
  https://primeintellectgrowth.com/media/learn/pig-tour.7f3a91c2.mp4
# HTTP/2 206 ... content-range: bytes 0-99/<size>
```

A 200 where a 206 is expected means something in front of PIG is buffering the
response and dropping the range — the video will play from the start and refuse
to seek.

## Upgrading

### By hand

```bash
bash scripts/deploy.sh
```

It fetches `origin/main`, starts the database, dumps it and checks the dump is
usable, builds, migrates from a one-off container, starts the app — and Piggy,
when `PIGGY_ENABLED` is on — and refuses to call the deploy done until the
health endpoint, the unauthenticated-401 gate, the piggy container's image,
agent directory and health, and the public origin all agree.

The database is started **before** the dump rather than after it, which is a
recent fix: the dump `exec`s into that container, so on a host where the stack
was down — a reboot, a `compose down`, a first-ever deploy — the backup step
failed and took the release with it.

### By tag — the normal path

Shipping is two steps and the second one is a human being:

```bash
git tag release-2026-08-13 && git push origin release-2026-08-13
```

That is the entire ship decision. What follows:

1. CI runs the full `verify` job against the tagged commit — the same job a
   push to main runs. A tag does not skip verification.
2. Only if that passes, the `publish` job builds and pushes
   `git.karti.ai/pig/pig:<tag>` and `:<short-sha>` to the Gitea registry.
3. Within five minutes `pig-autodeploy.timer` on the host notices that the
   newest `release-*` tag has a digest different from the running container,
   checks the tree out at that tag, and runs `scripts/deploy.sh` with
   `PIG_IMAGE` set — so it pulls the published image instead of rebuilding it.

**Push to main deploys nothing.** Tagging does.

The direction of travel is the point. No credential on the shared CI runner can
execute anything on this host; the host holds a pull-only registry token and
fetches. That preserves both halves of the constraint written at the top of
`scripts/deploy.sh` — no production key on the runner, and a human still
choosing when it ships.

**Trap: `sudo` throws `PIG_IMAGE` away.** The default sudoers policy sets
`env_reset`, so `PIG_IMAGE=… sudo docker compose …` hands compose an environment
without it and compose interpolates the `pig:local` fallback from
`docker-compose.yml`. The pull then dies with "pull access denied for pig" — and
if it had not died, the migrate, the `up` and the rollback would all have run
the stale local image while the log named the release tag. Every compose
invocation in `deploy.sh` therefore goes through the `dc()` wrapper, which uses
`sudo env PIG_IMAGE=… COMPOSE_PROFILES=… docker compose …`; `sudo -E` and bare
`sudo VAR=val` are both refused by that same policy. `COMPOSE_PROFILES` travels
the same way and for a sharper reason: dropped, compose does not error, it just
leaves Piggy out of whatever you asked for. Anything new that shells out to
compose must use the wrapper.

### Rollback

`scripts/deploy.sh` records the image the app container was running before it
replaces it. If the health check, the unauthenticated-401 gate, the piggy image
assertion or the public-origin marker check fails, it re-tags that image,
restarts the app — and Piggy with it, since they are one image running two
commands — reports whether the restored version is healthy, and exits non-zero. Previously
those exits left the broken release live, which was fine when a human was
watching the terminal and an outage when the poller ran at 04:00.

**The exit code says what is serving**, because that is the one thing the
on-call needs at 04:00 and `autodeploy.sh` can see nothing else:

| Exit | Meaning |
|---|---|
| 0 | deployed |
| 1 | a gate failed and the **previous** image was restored |
| 3 | a gate failed and the **release under test is still live** |

3 covers the cases where a rollback was not attempted (the fault is not
attributable to the release), where there was no previous image to restore, and
where the restore itself did not come up. `autodeploy.sh` logs a different
sentence for each, so the journal never claims a rollback that did not happen.

Two things it deliberately does **not** do:

- **It does not roll the database back.** Migrations are additive, so the
  previous image runs against the new schema. 0014 is the current example: it
  creates `piggy_conversations` and `piggy_messages` and alters nothing, so an
  image that predates it never names those tables and cannot notice they exist.
  That safety is a property of the migrations, not of this script — a migration
  that drops a column, renames one or tightens a constraint would break the
  restored image, and the dump taken before the migration is the escape hatch
  for that.
- **It does not roll back when the public origin answers with an EMPTY body.**
  Something terminated TLS and replied, so the fault is the proxy — see `bind`
  below — and the previous image would fail the same check. Exit 3.
- **It does not roll back when Piggy is enabled but does not come up.** Same
  reasoning: the previous image reads the same `.env`, so restoring it churns a
  healthy CRM without fixing the agent. Exit 3, and the log lists the three
  configuration faults that cause it — a missing or rejected `PRIME_API_KEY`, a
  short `PIGGY_INTERNAL_TOKEN`, a `PIGGY_AGENT_MODEL` outside the catalogue.
- **It does not roll back when `PIGGY_AGENT_DIR` points inside `/app`.** The
  container is healthy; the problem is that the harness's cwd would be the
  application checkout, so repository files could reach a CRM agent's prompt.
  Restoring the previous image changes nothing — it reads the same compose file
  — so the script refuses to report success and exits 3.

Those last two are the exit-3 cases where the site itself is fine.

It *does* roll back when the origin answers with a **non-empty** body that lacks
the marker. A proxy fault cannot serve a wrong-but-populated page for this
hostname; a bad release can — a changed Vite `base`, a Dockerfile step that
stopped copying `apps/web/dist`. Every earlier gate passes in that state, so
this is the only one that fires, and refusing to roll back would leave the
broken release facing the public.

If curl cannot reach the public origin **at all** (no hairpin for the public
name, egress to 443 filtered), that is not a failed deploy: it is a host that
cannot see itself from outside, and blacklisting the digest over it costs a
release. The script warns and exits 0. On a host where the public name really is
reachable from the host, set `PIG_DEPLOY_REQUIRE_PUBLIC=1` to make it fatal
(exit 3).

`scripts/autodeploy.sh` writes the failed digest to
`/var/lib/pig/failed-release` and will not retry it, so one bad tag does not
become a five-minute restart loop. Delete that file, or publish a new tag, to
try again.

To pin a specific release by hand:

```bash
git checkout --detach refs/tags/release-2026-08-12
PIG_IMAGE=git.karti.ai/pig/pig:release-2026-08-12 bash scripts/deploy.sh
```

Check the tree out at the tag as well as setting `PIG_IMAGE`. The compose file
and the migrations must come from the same commit as the image; with
`PIG_IMAGE` set, `deploy.sh` deliberately does not touch git, precisely so it
cannot drag you back to `main` behind your back.

Migrations are additive and safe to re-run; Drizzle tracks what has been
applied. `deploy.sh` takes a dump before every deploy; take one by hand before
a major upgrade anyway:

```bash
docker compose -p pig exec db pg_dump -U pig pig | gzip > pig-$(date +%F).sql.gz
```

## Installing the release poller

Only on the host that serves production, and only once.

```bash
# 1. A pull-only credential. read:package scope and NOTHING else — a token here
#    that can write packages or push to the repository undoes the reason
#    deployment is not automated from CI in the first place.
sudo install -d -m 0755 /etc/pig
printf '%s' 'gitea-token-here' | sudo tee /etc/pig/registry-token > /dev/null
sudo chmod 0600 /etc/pig/registry-token
sudo chown root:root /etc/pig/registry-token

# 2. Anything the defaults get wrong. Optional; the script assumes
#    /opt/pig, git.karti.ai and pig/pig.
sudo tee /etc/pig/autodeploy.env > /dev/null <<'EOF'
PIG_REGISTRY_USER=pig-deploy
PIG_REPO_DIR=/opt/pig
EOF
sudo chmod 0600 /etc/pig/autodeploy.env

# 3. The units.
sudo cp /opt/pig/deploy/pig-autodeploy.service /etc/systemd/system/
sudo cp /opt/pig/deploy/pig-autodeploy.timer   /etc/systemd/system/
sudo systemctl daemon-reload

# 4. Dry-run it once, in the foreground, before trusting a timer with it.
sudo systemctl start pig-autodeploy.service
sudo journalctl -u pig-autodeploy.service -n 50 --no-pager

# 5. Then arm it.
sudo systemctl enable --now pig-autodeploy.timer
systemctl list-timers pig-autodeploy.timer
```

The service is `Type=oneshot` with no `Restart=`, and the timer is not
`Persistent=true`: a missed poll is caught at the next tick rather than fired
at boot, which is when nobody is watching.

Watch a deploy:

```bash
journalctl -u pig-autodeploy.service -f
```

## On-premises: using your own identity provider

PIG authenticates against any standards-compliant OIDC provider, which is how
an install inside your own network works. Set:

```bash
PIG_OIDC_ISSUER=https://id.yourcompany.internal
PIG_OIDC_AUDIENCE=pig          # the client/app id you registered for PIG
```

That is usually the whole configuration — the JWKS is discovered from the
issuer. On an air-gapped network, set `PIG_OIDC_JWKS_URI` too and no discovery
request is made.

`PIG_OIDC_ISSUER` takes precedence over `SUPABASE_URL`, so the hosted values
can stay in the environment file without quietly taking over.

**Set the audience.** Without it, any token your provider issued for any
application in the same tenant verifies here — a token minted for an unrelated
internal tool would be accepted as a PIG session. PIG warns about this at boot
but cannot refuse, because some providers legitimately issue single-audience
tokens.

**Provisioning stays in PIG.** Authenticating proves who someone is; it does
not make them a member. They still need an invite, and their team and role live
in PIG's database. That is deliberate — your directory should not have to model
"supply lead versus demand member" for one application.

## A note on the auth project

PIG verifies JWTs but authorizes from its own `users` table. If the Supabase
project is shared with another application, its users get **nothing** here until
they are explicitly invited. That is deliberate, and it is why a valid token can
still return `403 needs_profile`.