claude
f0173440e4
Piggy was a hand-rolled OpenAI tool loop. It is now a Prime Agent session — Prime Intellect's own harness, embedded as a Node library — answering from PIG's tools and, for the first time, able to put information into the CRM rather than only read it out. The harness is a coding agent, so the first job was taking the coding agent away from it. `noTools: 'all'` plus an explicit allowlist leaves the model with PIG's ten `pig_*` tools and no bash, no filesystem, no IPython. That holds under attack: a hostile extension, a skill and a settings file planted in the agent's own directory, then `setActiveToolsByName` called with every built-in, still leaves ten tools, all ours. Both lines are load-bearing — `noTools` alone registers nothing, and the allowlist is what admits our own. Writing is gated rather than assumed. A change is proposed, not made: the tool returns a description, the transcript renders a diff card, and nothing reaches the database until someone presses Apply. Contracts, commitments, allocations and compliance always stop for a human whatever the mode. Every write runs through `executeMutation` as the calling user, so their capabilities and the audit trail apply exactly as they would to a human's. Four things about the SDK are wrong in its own documentation and cost a debugging cycle each: models.json does not resolve an env var name for `apiKey`, it sends the literal string; there is no built-in prime-inference provider in 0.84.1; a ResourceLoader you pass in is never reloaded for you; and the stock system prompt is a coding-assistant prompt that must be replaced — but replacing it also silently removes the tool list, because the harness only renders that section when it owns the prompt. AGENTS.md records all four. The expensive one was thinking level. The harness defaults to `medium`, and nemotron spent an entire 4,096-token budget reasoning and returned an empty answer. `low` was worse; `off` omits the parameter so the endpoint's default wins. An explicit `reasoning_effort: none` via `thinkingLevelMap` took a turn from 6,195 output tokens to 149. And a turn is now bounded. The harness loop is `while (true)` with no iteration cap; a runaway on a frontier model would have eaten the credit it is supposed to report on. Ceilings on model calls and tokens, enforced both through the harness hook and independently from the event stream, plus a per-user daily spend limit — and the ledger now records spend on turns that fail, which it previously discarded. Signing in lands on /piggy, which is a workspace: conversations down one side, the agent in the middle, what it did and what it cost beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1270 lines
64 KiB
YAML
1270 lines
64 KiB
YAML
# Continuous integration.
|
|
#
|
|
# Runs on every push and pull request. The job is deliberately one sequence
|
|
# rather than a fan-out: this is a small project, the whole thing takes a
|
|
# couple of minutes, and a single log is easier to read than five.
|
|
#
|
|
# What it actually proves, in order of how likely each is to catch something:
|
|
#
|
|
# 1. Every package typechecks.
|
|
# 2. The unit tests pass.
|
|
# 3. PIGGY HAS NO SHELL. Piggy runs a coding-agent harness inside a CRM, and
|
|
# the entire case for that is `noTools: 'all'` plus an explicit allowlist.
|
|
# This job boots real sessions — every mode, every context the protocol
|
|
# allows — and fails if the harness's LIVE tool set is anything other than
|
|
# the pig_ tools it was handed. It is a claim about a third-party SDK's
|
|
# tool composition, so it can be broken by a dependency bump rather than
|
|
# by a commit here, which is precisely why it is a gate and not a hope.
|
|
# 4. The default model maps the configured PIGGY_AGENT_THINKING onto an
|
|
# EXPLICIT reasoning_effort. Measured: the harness's own default of
|
|
# `medium` made nemotron spend 6,195 output tokens reasoning and return an
|
|
# empty answer, and `off` silently omits the field so the endpoint's
|
|
# default wins. Static, on models.json — no model is called.
|
|
# 5. The migration chain applies to a REAL, empty Postgres. This has already
|
|
# caught one migration that Drizzle generated but Postgres refused
|
|
# (a jsonb -> integer cast with no USING clause).
|
|
# 6. It applies TWICE with no effect: the second run leaves the schema
|
|
# byte-identical, every migration file is in the journal, and every table,
|
|
# column and index the files create is really in the database. Drizzle
|
|
# applies what the JOURNAL lists, not what the directory holds, so a
|
|
# migration shipped without its entry is never applied and migrate.ts
|
|
# still prints "Migrations applied." and exits 0.
|
|
# 7. Both seeds are idempotent — running each twice leaves EVERY row count
|
|
# the same, not merely the one table this used to check. That caught a
|
|
# seed which silently duplicated 27 contacts. `pnpm db:demo` is held to
|
|
# the same rule, on a database of its own: it had never been run by CI at
|
|
# all, which is how a non-idempotent demo seed survived four reviews.
|
|
# 8. The server boots against that database and answers.
|
|
# 9. Piggy boots against that same database, answers /internal/health, and
|
|
# the API — wired to it through the environment, not through a stub —
|
|
# reports it enabled. The relay's own tests inject a resolver, so they
|
|
# stay green whether or not the real wiring exists; only this step reads
|
|
# it. A crash on boot and an unset PIGGY_INTERNAL_URL look identical from
|
|
# the browser: the dock simply never appears.
|
|
# 10. The front end builds, and the CSP hash for the inline theme script still
|
|
# matches what the proxy is configured to allow. Editing that script
|
|
# changes its hash, and the failure mode is a silent white flash for
|
|
# dark-mode users rather than an error.
|
|
# 11. docker-compose.yml renders, and the piggy service is passed every
|
|
# environment key the worker's schema requires — and no key it does not
|
|
# read. That is the one failure nothing else here can see, because it
|
|
# lives between two files that are each individually correct.
|
|
#
|
|
# NO MODEL IS EVER CALLED, and no PRIME_API_KEY is available to this runner.
|
|
# Everything above is provable offline; a gate that cost a paid inference call
|
|
# would be switched off within a month.
|
|
#
|
|
# THE CSP HASH IS DUPLICATED IN THREE PLACES: the `expected` constant below,
|
|
# `deploy/Caddyfile.example`, and the LIVE Caddyfile on cloud-2. Only the first
|
|
# two are checked by anything. The live one is the copy that actually decides
|
|
# whether a browser runs the script, and nothing in this repository can see it,
|
|
# so changing the script means editing all three by hand — see deploy/README.md.
|
|
#
|
|
# Shipping is a two-step, and the second step is a human:
|
|
#
|
|
# push to main -> `verify` only. Nothing is published, nothing deploys.
|
|
# tag release-* -> `verify`, then `publish` pushes the image to the Gitea
|
|
# registry. The production host notices it and deploys.
|
|
#
|
|
# So the tag IS the ship decision. No credential on this runner can reach
|
|
# cloud-2; the host pulls, the runner never pushes to it.
|
|
|
|
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
# A tag push runs the same verification and then, and only then, publishes.
|
|
tags: ['release-*']
|
|
pull_request:
|
|
|
|
jobs:
|
|
verify:
|
|
runs-on: ubuntu-latest
|
|
|
|
# Postgres is started as a step rather than through `services:`, because
|
|
# this runner is configured with `container.network: host`.
|
|
#
|
|
# That single setting explains three failed attempts, and is worth writing
|
|
# down so nobody repeats them:
|
|
#
|
|
# `services:` Service containers are not resolvable by
|
|
# name from a host-networked job, giving
|
|
# "getaddrinfo EAI_AGAIN postgres".
|
|
# `--network container:$HOSTNAME` /etc/hostname reports the HOST's name,
|
|
# not a container id, so the namespace
|
|
# join finds no such container.
|
|
# default-gateway addressing The wrong idea entirely: with host
|
|
# networking the default route is the
|
|
# real router, not a docker bridge.
|
|
#
|
|
# Because the job shares the host's network namespace, a published port is
|
|
# simply on 127.0.0.1. The port is derived from the run id so concurrent
|
|
# runs cannot collide.
|
|
env:
|
|
PG_CONTAINER: pig-ci-pg-${{ github.run_id }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '22'
|
|
|
|
# Corepack ships with Node and installs the exact pnpm pinned by
|
|
# `packageManager` in package.json, so CI, the image and a laptop all run
|
|
# the same version. `--activate` puts it on PATH; the download prompt is
|
|
# disabled because a non-interactive runner cannot answer it and would
|
|
# otherwise hang until the job times out.
|
|
- name: Enable pnpm
|
|
env:
|
|
COREPACK_ENABLE_DOWNLOAD_PROMPT: '0'
|
|
run: |
|
|
corepack enable
|
|
corepack prepare --activate
|
|
pnpm --version
|
|
|
|
- name: Start Postgres
|
|
run: |
|
|
PG_PORT=$(( 45000 + (${{ github.run_id }} % 15000) ))
|
|
echo "Publishing Postgres on 127.0.0.1:${PG_PORT}"
|
|
|
|
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
|
|
docker run -d --name "$PG_CONTAINER" \
|
|
-p "127.0.0.1:${PG_PORT}:5432" \
|
|
-e POSTGRES_USER=pig -e POSTGRES_PASSWORD=pig -e POSTGRES_DB=pig \
|
|
postgres:16-alpine
|
|
|
|
# pg_isready inside the container only proves the server started.
|
|
# What matters is that THIS job can reach it through the published
|
|
# port, so the readiness check is made from here, over TCP.
|
|
for i in $(seq 1 60); do
|
|
if node -e "
|
|
const net=require('net');
|
|
const s=net.connect(${PG_PORT},'127.0.0.1');
|
|
s.on('connect',()=>{s.end();process.exit(0)});
|
|
s.on('error',()=>process.exit(1));
|
|
" 2>/dev/null; then
|
|
echo "Reachable after ${i}s"
|
|
echo "DATABASE_URL=postgres://pig:pig@127.0.0.1:${PG_PORT}/pig" >> "$GITHUB_ENV"
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
echo "Postgres never became reachable on 127.0.0.1:${PG_PORT}"
|
|
docker logs "$PG_CONTAINER" 2>&1 | tail -30
|
|
exit 1
|
|
|
|
- name: Install
|
|
# --frozen-lockfile fails rather than quietly resolving a different
|
|
# tree when the lockfile and manifests disagree. That is the whole
|
|
# point of committing a lockfile, and it is the default in CI anyway —
|
|
# stated here so it survives someone running this locally.
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Typecheck every package
|
|
run: pnpm run typecheck
|
|
|
|
- name: Unit tests
|
|
run: pnpm run test
|
|
|
|
- name: Piggy has no shell, and this is the step that proves it
|
|
# The security property of the whole re-platform, as a gate rather than a
|
|
# hope. Piggy embeds Prime Agent — a CODING agent — inside a CRM, and the
|
|
# only reason that is defensible is that the harness is constructed with
|
|
# `noTools: 'all'` and an explicit allowlist, so the model gets PIG's read
|
|
# and write tools and no shell, no filesystem and no Python.
|
|
#
|
|
# That is a claim about how a third-party SDK composes its tool sources.
|
|
# It can therefore be broken by `pnpm update` rather than by a commit to
|
|
# this repository, and it would break silently: a leaked `bash` tool
|
|
# changes nothing a user can see until the day somebody asks Piggy to read
|
|
# /etc/passwd and it does.
|
|
#
|
|
# Checked twice over, because the two halves fail differently. The unit
|
|
# test is the readable statement of the property and lives next to the
|
|
# code; this step additionally refuses to accept a green result from a
|
|
# suite where that test was renamed away, deleted or skipped. The live
|
|
# boot below then rebuilds the REAL production tool set — the same
|
|
# functions chat-server.ts calls — and reads the harness's own tool list
|
|
# back out, which is the only thing that can catch a leak the test file
|
|
# does not think to name.
|
|
run: |
|
|
set -euo pipefail
|
|
WORK=$(mktemp -d)
|
|
|
|
# ---------------------------------------------------------------------
|
|
# 1. The test that pins the property really ran, and really passed.
|
|
# ---------------------------------------------------------------------
|
|
TAP="$WORK/agent-session.tap"
|
|
if ! pnpm -F @pig/piggy exec node --test --test-reporter=tap --import tsx test/agent-session.test.ts | tee "$TAP"; then
|
|
echo 'apps/piggy/test/agent-session.test.ts failed. Read the assertion above before anything else:'
|
|
echo 'it is the test that holds the agent to PIG tools only.'
|
|
exit 1
|
|
fi
|
|
|
|
# Named, because a security gate that would go green if somebody deleted the
|
|
# test is not a gate. If a test below is legitimately renamed, rename it here
|
|
# in the same commit.
|
|
SAFETY_TESTS='the session exposes exactly the tools it was handed, and nothing else
|
|
a tool outside the PIG boundary never reaches the harness'
|
|
while IFS= read -r NAME; do
|
|
LINE=$(grep -F -- " - ${NAME}" "$TAP" | head -1 || true)
|
|
case "$LINE" in
|
|
'ok '*) echo " passed: ${NAME}" ;;
|
|
'not ok '*) echo "PIGGY'S SANDBOX TEST FAILED: ${NAME}"; exit 1 ;;
|
|
*)
|
|
echo "The tool-boundary test '${NAME}' did not run."
|
|
echo 'It lives in apps/piggy/test/agent-session.test.ts. If it was renamed, rename it here too;'
|
|
echo 'if it was deleted, put it back — it is the readable form of the sandbox property.'
|
|
exit 1
|
|
;;
|
|
esac
|
|
done <<TESTS
|
|
${SAFETY_TESTS}
|
|
TESTS
|
|
|
|
# A skipped or todo test reports `ok` in TAP. Left unchecked, `test.skip` on
|
|
# the sandbox test is a green build.
|
|
for COUNTER in fail skipped todo; do
|
|
VALUE=$(grep -E "^# ${COUNTER} [0-9]+$" "$TAP" | tail -1 | awk '{print $3}')
|
|
test "${VALUE:-missing}" = '0' || {
|
|
echo "node --test reported '# ${COUNTER} ${VALUE:-missing}' for agent-session.test.ts."
|
|
echo 'A skipped or failing sandbox test is not a pass.'
|
|
exit 1
|
|
}
|
|
done
|
|
PASSED=$(grep -E '^# pass [0-9]+$' "$TAP" | tail -1 | awk '{print $3}')
|
|
test "${PASSED:-0}" -ge 2 || { echo "Only ${PASSED:-0} test(s) ran in agent-session.test.ts."; exit 1; }
|
|
echo "agent-session.test.ts: ${PASSED} tests, none failed, none skipped."
|
|
|
|
# ---------------------------------------------------------------------
|
|
# 2. And the harness a real conversation would get agrees.
|
|
# ---------------------------------------------------------------------
|
|
cat > "$WORK/piggy-tool-boundary.mts" <<'BOUNDARYEOF'
|
|
/**
|
|
* The security property of the harness swap, asserted against a LIVE session.
|
|
*
|
|
* Piggy runs a coding agent inside a CRM. The entire case for that is that the
|
|
* harness is started with `noTools: 'all'` and an explicit allowlist, so the
|
|
* model has PIG's tools and no shell, no filesystem and no code execution. That
|
|
* is a claim about a third-party SDK's tool composition, which means it can be
|
|
* broken by a dependency bump rather than by a commit to this repository — so
|
|
* it is checked here, on every run, against the tool set the harness actually
|
|
* ended up with.
|
|
*
|
|
* The production tool set is REBUILT here rather than stubbed: the same
|
|
* `createInteractivePigTools` + `createPigWriteTools` the chat server assembles,
|
|
* for every context the protocol allows and every mode, because the tool set is
|
|
* a function of both. No model is called and no key is needed; sessions are
|
|
* constructed and disposed.
|
|
*/
|
|
import { resolve } from 'node:path';
|
|
import { pathToFileURL } from 'node:url';
|
|
|
|
/**
|
|
* Tool names the harness gives a coding agent. A leak is a leak whatever it is
|
|
* called — the exact-set comparison below is what catches an unknown one — but
|
|
* these are named so the failure message says "a shell tool is live" rather
|
|
* than "unexpected tool", and so the check keeps meaning something if the
|
|
* comparison is ever loosened.
|
|
*
|
|
* Matched as WHOLE names, not substrings. `read`, `write` and `edit` are
|
|
* ordinary English: a substring test would reject a perfectly good
|
|
* `pig_read_contract` while catching nothing a whole-name test misses, because
|
|
* every PIG tool is `pig_`-prefixed and no built-in is.
|
|
*/
|
|
const HARNESS_BUILTIN_NAMES = new Set([
|
|
'bash',
|
|
'shell',
|
|
'read',
|
|
'write',
|
|
'edit',
|
|
'multi_edit',
|
|
'ls',
|
|
'glob',
|
|
'grep',
|
|
'find',
|
|
'python',
|
|
'ipython',
|
|
'notebook_edit',
|
|
'fetch',
|
|
'web_fetch',
|
|
'web_search',
|
|
'task',
|
|
'todo_write',
|
|
]);
|
|
|
|
/** Substrings that make a tool dangerous whatever else it is called. */
|
|
const DANGEROUS_SUBSTRING = /bash|shell|ipython|python|filesystem|subprocess|file_read|file_write|\bexec\b/i;
|
|
|
|
function assertNameIsSafe(name: string, where: string): string[] {
|
|
const problems: string[] = [];
|
|
if (!/^pig_[a-z0-9_]+$/.test(name)) {
|
|
problems.push(`${where}: '${name}' is not a pig_ tool.`);
|
|
}
|
|
if (HARNESS_BUILTIN_NAMES.has(name)) {
|
|
problems.push(`${where}: '${name}' is a harness built-in — shell, filesystem or code execution.`);
|
|
}
|
|
if (DANGEROUS_SUBSTRING.test(name)) {
|
|
problems.push(`${where}: '${name}' names a shell, an interpreter or the filesystem.`);
|
|
}
|
|
return problems;
|
|
}
|
|
|
|
const load = async (path: string): Promise<Record<string, unknown>> =>
|
|
(await import(pathToFileURL(resolve(path)).href)) as Record<string, unknown>;
|
|
|
|
/**
|
|
* Only the shape this check reads. The harness's own types are not imported:
|
|
* this file is written to a temp directory outside the workspace, so a bare
|
|
* specifier here would not resolve.
|
|
*/
|
|
interface HarnessTool {
|
|
name: string;
|
|
}
|
|
|
|
interface PigModules {
|
|
createDatabase: (options: { url: string; max: number }) => unknown;
|
|
createInteractivePigTools: (db: unknown, context: unknown) => unknown[];
|
|
toPrimeTools: (tools: readonly unknown[]) => HarnessTool[];
|
|
createPigWriteTools: (deps: Record<string, unknown>) => HarnessTool[];
|
|
createPiggySession: (options: Record<string, unknown>) => Promise<{
|
|
session: { agent: { state: { tools: readonly HarnessTool[] } } };
|
|
dispose: () => void;
|
|
}>;
|
|
pageRoutes: readonly string[];
|
|
recordTypes: readonly string[];
|
|
modes: readonly string[];
|
|
}
|
|
|
|
async function loadModules(): Promise<PigModules> {
|
|
const db = await load('packages/db/src/index.ts');
|
|
const core = await load('packages/core/src/index.ts');
|
|
const chatTools = await load('apps/piggy/src/chat-tools.ts');
|
|
const bridge = await load('apps/piggy/src/agent/tool-bridge.ts');
|
|
const writeTools = await load('apps/piggy/src/write-tools.ts');
|
|
const session = await load('apps/piggy/src/agent/session.ts');
|
|
return {
|
|
createDatabase: db.createDatabase as PigModules['createDatabase'],
|
|
createInteractivePigTools: chatTools.createInteractivePigTools as PigModules['createInteractivePigTools'],
|
|
toPrimeTools: bridge.toPrimeTools as PigModules['toPrimeTools'],
|
|
createPigWriteTools: writeTools.createPigWriteTools as PigModules['createPigWriteTools'],
|
|
createPiggySession: session.createPiggySession as PigModules['createPiggySession'],
|
|
pageRoutes: core.PIGGY_PAGE_ROUTES as readonly string[],
|
|
recordTypes: core.PIGGY_RECORD_TYPES as readonly string[],
|
|
modes: core.PIGGY_MODES as readonly string[],
|
|
};
|
|
}
|
|
|
|
const databaseUrl = process.env.DATABASE_URL;
|
|
if (!databaseUrl) {
|
|
console.error('DATABASE_URL is not set; the tools are built against a real database handle.');
|
|
process.exit(1);
|
|
}
|
|
|
|
const pig = await loadModules();
|
|
const database = pig.createDatabase({ url: databaseUrl, max: 1 });
|
|
|
|
/**
|
|
* The person Piggy is acting as. Never elevated: the write tools bind to this
|
|
* principal, and a synthetic admin here would test a privilege level no real
|
|
* conversation has.
|
|
*/
|
|
const principal = {
|
|
userId: '00000000-0000-0000-0000-000000000001',
|
|
email: 'ci@pig.invalid',
|
|
name: 'CI',
|
|
isPlatformAdmin: false,
|
|
teams: [],
|
|
via: 'development',
|
|
scopes: [],
|
|
};
|
|
|
|
/** Never called: no tool is executed here, only registered. */
|
|
const propose = async (): Promise<never> => {
|
|
throw new Error('A CI boundary check proposed a change, which means it executed a tool.');
|
|
};
|
|
|
|
const contexts: { label: string; value: unknown }[] = [
|
|
{ label: 'no context (dashboard)', value: undefined },
|
|
...pig.pageRoutes.map((route) => ({ label: `page ${route}`, value: { type: 'page', route } })),
|
|
...pig.recordTypes.map((type) => ({
|
|
label: `record ${type}`,
|
|
value: { type, id: '00000000-0000-0000-0000-000000000002' },
|
|
})),
|
|
];
|
|
|
|
const problems: string[] = [];
|
|
let checked = 0;
|
|
let widest = 0;
|
|
|
|
for (const mode of pig.modes) {
|
|
for (const context of contexts) {
|
|
const where = `mode=${mode} ${context.label}`;
|
|
// Exactly what chat-server.ts's buildToolSet assembles for this turn.
|
|
const tools = [...pig.toPrimeTools(pig.createInteractivePigTools(database, context.value))];
|
|
if (mode !== 'read_only') {
|
|
tools.push(...pig.createPigWriteTools({ db: database, principal, mode, propose }));
|
|
}
|
|
for (const tool of tools) problems.push(...assertNameIsSafe(tool.name, `${where} handed in`));
|
|
|
|
// A session that refuses to start is a finding, not a crash: session.ts
|
|
// makes the same assertion at construction, and its message is the one
|
|
// worth printing next to the others rather than as a stack trace.
|
|
let piggy: Awaited<ReturnType<PigModules['createPiggySession']>>;
|
|
try {
|
|
piggy = await pig.createPiggySession({ mode, tools, context: context.value });
|
|
} catch (error) {
|
|
problems.push(`${where}: the session refused to start — ${error instanceof Error ? error.message : String(error)}`);
|
|
checked += 1;
|
|
continue;
|
|
}
|
|
try {
|
|
const live = piggy.session.agent.state.tools.map((tool) => tool.name).sort();
|
|
const wanted = tools.map((tool) => tool.name).sort();
|
|
for (const name of live) problems.push(...assertNameIsSafe(name, `${where} LIVE`));
|
|
|
|
// The complete property: anything the harness composed in from an
|
|
// extension, a skill or a built-in shows up here as an extra name,
|
|
// whatever it is called.
|
|
const unexpected = live.filter((name) => !wanted.includes(name));
|
|
const missing = wanted.filter((name) => !live.includes(name));
|
|
if (unexpected.length > 0) {
|
|
problems.push(`${where}: the harness added tools nobody handed it: ${unexpected.join(', ')}`);
|
|
}
|
|
if (missing.length > 0) {
|
|
problems.push(`${where}: PIG tools never reached the model: ${missing.join(', ')}`);
|
|
}
|
|
widest = Math.max(widest, live.length);
|
|
} finally {
|
|
piggy.dispose();
|
|
}
|
|
checked += 1;
|
|
}
|
|
}
|
|
|
|
if (problems.length > 0) {
|
|
console.error('PIGGY IS NOT SANDBOXED. Every one of these is a live agent tool outside PIG:');
|
|
for (const problem of problems) console.error(` ${problem}`);
|
|
console.error('');
|
|
console.error('Do not ship this. `noTools: all` plus the allowlist in apps/piggy/src/agent/session.ts');
|
|
console.error('is the only thing standing between a CRM chat box and a shell on the container.');
|
|
process.exit(1);
|
|
}
|
|
|
|
console.log(
|
|
`${checked} live sessions checked (every mode x every context); the widest tool set was ${widest} tools, all of them pig_.`,
|
|
);
|
|
process.exit(0);
|
|
BOUNDARYEOF
|
|
# The key is a placeholder and stays one: no model is called here, and this
|
|
# runner holds no PRIME_API_KEY. The schema demands the field, so it is given
|
|
# a value that would fail loudly if anything ever did reach the endpoint.
|
|
#
|
|
# PIGGY_AGENT_DIR is pointed at a temp directory rather than $HOME: the
|
|
# harness treats it as cwd, and this act_runner is shared with every other
|
|
# repository on cloud-1.
|
|
PIGGY_AGENT_DIR="$WORK/agent" \
|
|
PIGGY_INTERNAL_TOKEN=piggy-ci-internal-token-0123456789 \
|
|
PRIME_API_KEY=ci-placeholder-no-model-is-called \
|
|
pnpm exec tsx "$WORK/piggy-tool-boundary.mts"
|
|
|
|
- name: The default model still asks for an explicit reasoning effort
|
|
# A regression gate for the most expensive bug this project has had, and
|
|
# the cheapest to reintroduce: one edit to models.json brings it back.
|
|
# Static — it reads models.json and the config schema and calls nothing.
|
|
run: |
|
|
set -euo pipefail
|
|
WORK=$(mktemp -d)
|
|
cat > "$WORK/piggy-thinking-map.mts" <<'THINKINGEOF'
|
|
/**
|
|
* The reasoning trap, made unrepeatable.
|
|
*
|
|
* MEASURED, on the live endpoint: the harness defaults `thinkingLevel` to
|
|
* `medium`, and on nemotron that produced 6,195 output tokens of reasoning and
|
|
* an EMPTY answer — the turn hit its ceiling while still thinking, and came
|
|
* back with finish_reason `length`. `low` was worse. `off` is not a fix on its
|
|
* own either: it OMITS `reasoning_effort` from the request, so whatever the
|
|
* endpoint defaults to wins, silently. What fixed it was a `thinkingLevelMap`
|
|
* on the model entry mapping `off` to an explicit `"none"` — 149 output tokens
|
|
* and a correct answer.
|
|
*
|
|
* So the shipped default model must map the configured thinking level onto an
|
|
* explicit effort, and this asserts exactly that. It is static: it reads
|
|
* models.json and the config schema, and never calls a model. A gate for this
|
|
* that cost a paid inference call would be turned off within a month.
|
|
*/
|
|
import { readFileSync } from 'node:fs';
|
|
import { resolve } from 'node:path';
|
|
import { pathToFileURL } from 'node:url';
|
|
|
|
interface ModelEntry {
|
|
id: string;
|
|
thinkingLevelMap?: Record<string, unknown>;
|
|
}
|
|
|
|
const load = async (path: string): Promise<Record<string, unknown>> =>
|
|
(await import(pathToFileURL(resolve(path)).href)) as Record<string, unknown>;
|
|
|
|
const MODELS_JSON = 'apps/piggy/src/agent/models.json';
|
|
const CONFIG_TS = 'apps/piggy/src/config.ts';
|
|
|
|
/**
|
|
* The raw file, not the parsed catalogue. `models.ts` validates with a zod
|
|
* object that does not mention `thinkingLevelMap`, so the parsed value drops
|
|
* the very field under test — while the harness reads this file verbatim.
|
|
*/
|
|
const document = JSON.parse(readFileSync(MODELS_JSON, 'utf8')) as {
|
|
providers?: Record<string, { models?: ModelEntry[] }>;
|
|
};
|
|
const models = Object.values(document.providers ?? {}).flatMap((provider) => provider.models ?? []);
|
|
if (models.length === 0) {
|
|
console.error(`${MODELS_JSON} declares no models.`);
|
|
process.exit(1);
|
|
}
|
|
|
|
/**
|
|
* The levels PIGGY_AGENT_THINKING may be set to, read from the schema rather
|
|
* than copied: a level added there with no mapping here is the same hole by
|
|
* another name.
|
|
*/
|
|
const configSource = readFileSync(CONFIG_TS, 'utf8');
|
|
const enumMatch = /PIGGY_AGENT_THINKING:[\s\S]{0,400}?\.enum\(\[([^\]]+)\]\)/.exec(configSource);
|
|
if (!enumMatch?.[1]) {
|
|
console.error(`Could not find the PIGGY_AGENT_THINKING enum in ${CONFIG_TS}.`);
|
|
console.error('This check derives the levels from the schema; if the schema moved, teach it where.');
|
|
process.exit(1);
|
|
}
|
|
const levels = [...enumMatch[1].matchAll(/'([a-z]+)'/g)].map(([, level]) => level as string);
|
|
if (levels.length === 0) {
|
|
console.error(`The PIGGY_AGENT_THINKING enum in ${CONFIG_TS} parsed to nothing.`);
|
|
process.exit(1);
|
|
}
|
|
|
|
/**
|
|
* What a deployment will actually run with. Read through the config schema, so
|
|
* this tracks the defaults and honours an override set in this environment,
|
|
* rather than restating either.
|
|
*/
|
|
const config = (await load(CONFIG_TS)) as {
|
|
loadPiggyConfig: (env: NodeJS.ProcessEnv) => { PIGGY_AGENT_MODEL: string; PIGGY_AGENT_THINKING: string };
|
|
};
|
|
const resolved = config.loadPiggyConfig({
|
|
...process.env,
|
|
// Values only, so the schema will parse; nothing here reaches a database or
|
|
// an endpoint.
|
|
DATABASE_URL: process.env.DATABASE_URL ?? 'postgres://ci/probe',
|
|
PIGGY_INTERNAL_TOKEN: 'ci-probe-token-of-at-least-32-characters',
|
|
PRIME_API_KEY: 'ci-probe-key-never-sent-anywhere',
|
|
});
|
|
const defaultModelId = resolved.PIGGY_AGENT_MODEL;
|
|
const thinking = resolved.PIGGY_AGENT_THINKING;
|
|
console.log(`default model: ${defaultModelId}; PIGGY_AGENT_THINKING: ${thinking}`);
|
|
|
|
const problems: string[] = [];
|
|
const defaultModel = models.find((model) => model.id === defaultModelId);
|
|
if (!defaultModel) {
|
|
problems.push(`${defaultModelId} is the configured default but has no entry in ${MODELS_JSON}.`);
|
|
} else if (!defaultModel.thinkingLevelMap) {
|
|
problems.push(
|
|
`${defaultModelId} has no thinkingLevelMap, so thinkingLevel '${thinking}' is sent to the endpoint as no reasoning_effort at all.`,
|
|
);
|
|
} else {
|
|
const map = defaultModel.thinkingLevelMap;
|
|
const mapped = map[thinking];
|
|
if (typeof mapped !== 'string' || mapped.trim() === '') {
|
|
problems.push(
|
|
`${defaultModelId}'s thinkingLevelMap does not map '${thinking}' onto an explicit reasoning_effort.`,
|
|
);
|
|
}
|
|
// Every level, not merely the configured one: PIGGY_AGENT_THINKING is an
|
|
// environment variable, so an operator can select any of them without
|
|
// touching this repository.
|
|
const uncovered = levels.filter((level) => typeof map[level] !== 'string' || `${map[level]}`.trim() === '');
|
|
if (uncovered.length > 0) {
|
|
problems.push(
|
|
`${defaultModelId}'s thinkingLevelMap leaves ${uncovered.join(', ')} unmapped; setting PIGGY_AGENT_THINKING to one of those omits reasoning_effort again.`,
|
|
);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* A partial map on any other model is the same trap waiting for whoever
|
|
* switches model in the picker.
|
|
*/
|
|
for (const model of models) {
|
|
if (!model.thinkingLevelMap) continue;
|
|
const uncovered = levels.filter(
|
|
(level) => typeof model.thinkingLevelMap?.[level] !== 'string' || `${model.thinkingLevelMap[level]}`.trim() === '',
|
|
);
|
|
if (uncovered.length > 0 && model.id !== defaultModelId) {
|
|
problems.push(`${model.id} has a thinkingLevelMap that does not cover: ${uncovered.join(', ')}.`);
|
|
}
|
|
}
|
|
|
|
if (problems.length > 0) {
|
|
console.error('The reasoning-effort regression is back, or is one edit away:');
|
|
for (const problem of problems) console.error(` ${problem}`);
|
|
console.error('');
|
|
console.error('Measured consequence: 6,195 output tokens of reasoning and an EMPTY answer');
|
|
console.error(`(finish_reason: length). Add the mapping to ${MODELS_JSON}; 'off' must map to 'none'.`);
|
|
process.exit(1);
|
|
}
|
|
|
|
console.log(
|
|
`${defaultModelId} maps every thinking level (${levels.join(', ')}) onto an explicit reasoning_effort; '${thinking}' -> '${String(defaultModel?.thinkingLevelMap?.[thinking])}'.`,
|
|
);
|
|
process.exit(0);
|
|
THINKINGEOF
|
|
pnpm exec tsx "$WORK/piggy-thinking-map.mts"
|
|
|
|
- name: Migrations apply to a real Postgres
|
|
run: pnpm exec tsx packages/db/src/migrate.ts
|
|
|
|
- name: Migrations are re-runnable, journalled, and really applied
|
|
# Three claims, and the second is the one that has no other witness.
|
|
#
|
|
# `migrate.ts` applies what meta/_journal.json LISTS. A migration file
|
|
# added without its journal entry is never applied, and migrate.ts still
|
|
# prints "Migrations applied." and exits 0 — so CI stays green and the
|
|
# missing table turns up in production as a 500 from whichever route
|
|
# reads it. Migration 0014 added two tables and a column that nothing
|
|
# else in this job touches, which is exactly the shape of that failure.
|
|
run: |
|
|
set -euo pipefail
|
|
WORK=$(mktemp -d)
|
|
|
|
# Every column, index and constraint in the public schema, sorted. A second
|
|
# migration run must be a no-op, and "it exited 0" is not that claim.
|
|
SCHEMA_OBJECTS="select 'column ' || table_name || '.' || column_name || ' ' || data_type || ' null=' || is_nullable || ' default=' || coalesce(column_default, '-') from information_schema.columns where table_schema = 'public' union all select 'index ' || indexdef from pg_indexes where schemaname = 'public' union all select 'constraint ' || conrelid::regclass::text || ' ' || conname || ' ' || pg_get_constraintdef(oid) from pg_constraint where connamespace = 'public'::regnamespace order by 1"
|
|
|
|
snapshot() { docker exec "$PG_CONTAINER" psql -U pig -d pig -tAc "$SCHEMA_OBJECTS"; }
|
|
|
|
snapshot > "$WORK/schema-before.txt"
|
|
pnpm exec tsx packages/db/src/migrate.ts
|
|
snapshot > "$WORK/schema-after.txt"
|
|
if ! diff -u "$WORK/schema-before.txt" "$WORK/schema-after.txt"; then
|
|
echo 'A SECOND MIGRATION RUN CHANGED THE SCHEMA.'
|
|
echo 'Drizzle applies each migration once and records it, so this means a file was edited'
|
|
echo 'after it shipped, or a journal entry was rewritten. Either way, every database that'
|
|
echo 'already ran the old version of that migration will never receive the difference.'
|
|
exit 1
|
|
fi
|
|
echo "a second migration run changed nothing ($(wc -l < "$WORK/schema-before.txt") schema objects)"
|
|
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig -tAc \
|
|
"select 'table ' || table_name from information_schema.tables where table_schema = 'public' and table_type = 'BASE TABLE' union all select 'index ' || indexname from pg_indexes where schemaname = 'public' union all select 'column ' || table_name || '.' || column_name from information_schema.columns where table_schema = 'public'" \
|
|
> "$WORK/objects.txt"
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig -tAc \
|
|
'select count(*) from drizzle.__drizzle_migrations' > "$WORK/applied.txt"
|
|
cat > "$WORK/migration-objects.mjs" <<'MIGRATIONEOF'
|
|
/**
|
|
* Every object the migration files claim to create must actually be in the
|
|
* database, and every migration file must be in the journal.
|
|
*
|
|
* The failure this exists for: Drizzle applies what the JOURNAL lists, not what
|
|
* the directory contains. A migration added without its `meta/_journal.json`
|
|
* entry is never applied, and `migrate.ts` still prints "Migrations applied."
|
|
* and exits 0 — so CI stays green and the table only turns up missing in
|
|
* production, as a 500 from whichever route reads it. 0014 introduced two such
|
|
* tables and a column on agent_runs; nothing else in this job reads any of them.
|
|
*/
|
|
import { readFileSync, readdirSync } from 'node:fs';
|
|
import { join } from 'node:path';
|
|
|
|
const [objectsPath, appliedPath] = process.argv.slice(2);
|
|
if (!objectsPath || !appliedPath) {
|
|
console.error('Usage: migration-objects.mjs <objects.txt> <applied-count.txt>');
|
|
process.exit(1);
|
|
}
|
|
|
|
const MIGRATIONS = 'packages/db/migrations';
|
|
|
|
const files = readdirSync(MIGRATIONS)
|
|
.filter((name) => name.endsWith('.sql'))
|
|
.sort();
|
|
const journal = JSON.parse(readFileSync(join(MIGRATIONS, 'meta', '_journal.json'), 'utf8'));
|
|
const journalTags = new Set(journal.entries.map((entry) => entry.tag));
|
|
|
|
const problems = [];
|
|
|
|
for (const file of files) {
|
|
const tag = file.replace(/\.sql$/, '');
|
|
if (!journalTags.has(tag)) {
|
|
problems.push(
|
|
`${file} is not in meta/_journal.json, so drizzle never applies it and migrate.ts still exits 0.`,
|
|
);
|
|
}
|
|
}
|
|
for (const tag of journalTags) {
|
|
if (!files.includes(`${tag}.sql`)) {
|
|
problems.push(`meta/_journal.json lists ${tag}, which has no .sql file; migrate would fail on a fresh database.`);
|
|
}
|
|
}
|
|
|
|
const applied = Number.parseInt(readFileSync(appliedPath, 'utf8').trim(), 10);
|
|
if (!Number.isInteger(applied)) {
|
|
problems.push('Could not read the applied-migration count out of drizzle.__drizzle_migrations.');
|
|
} else if (applied !== journal.entries.length) {
|
|
problems.push(
|
|
`The database has ${applied} migrations applied but the journal lists ${journal.entries.length}.`,
|
|
);
|
|
}
|
|
|
|
/** Objects the database actually has, as `kind name` lines. */
|
|
const present = new Set(
|
|
readFileSync(objectsPath, 'utf8')
|
|
.split('\n')
|
|
.map((line) => line.trim())
|
|
.filter(Boolean),
|
|
);
|
|
|
|
/**
|
|
* What each file says it creates. Only the additive statements are read: a
|
|
* DROP in a later migration would make an earlier CREATE legitimately absent,
|
|
* so anything a later file drops is removed from the expectation below rather
|
|
* than reported.
|
|
*/
|
|
const expected = new Map();
|
|
const dropped = new Set();
|
|
for (const file of files) {
|
|
const sql = readFileSync(join(MIGRATIONS, file), 'utf8');
|
|
for (const [, name] of sql.matchAll(/CREATE TABLE (?:IF NOT EXISTS )?"([^"]+)"/gi)) {
|
|
expected.set(`table ${name}`, file);
|
|
}
|
|
for (const [, name] of sql.matchAll(/CREATE (?:UNIQUE )?INDEX (?:IF NOT EXISTS )?"([^"]+)"/gi)) {
|
|
expected.set(`index ${name}`, file);
|
|
}
|
|
for (const [, table, column] of sql.matchAll(
|
|
/ALTER TABLE "([^"]+)" ADD COLUMN (?:IF NOT EXISTS )?"([^"]+)"/gi,
|
|
)) {
|
|
expected.set(`column ${table}.${column}`, file);
|
|
}
|
|
for (const [, name] of sql.matchAll(/DROP TABLE (?:IF EXISTS )?"([^"]+)"/gi)) {
|
|
dropped.add(`table ${name}`);
|
|
}
|
|
for (const [, name] of sql.matchAll(/DROP INDEX (?:IF EXISTS )?"([^"]+)"/gi)) {
|
|
dropped.add(`index ${name}`);
|
|
}
|
|
for (const [, table, column] of sql.matchAll(/ALTER TABLE "([^"]+)" DROP COLUMN (?:IF EXISTS )?"([^"]+)"/gi)) {
|
|
dropped.add(`column ${table}.${column}`);
|
|
}
|
|
}
|
|
|
|
for (const [object, file] of expected) {
|
|
if (dropped.has(object)) continue;
|
|
if (!present.has(object)) {
|
|
problems.push(`${file} creates ${object}, and the migrated database does not have it.`);
|
|
}
|
|
}
|
|
|
|
if (problems.length > 0) {
|
|
console.error('The migration chain and the database it produced disagree:');
|
|
for (const problem of problems) console.error(` ${problem}`);
|
|
process.exit(1);
|
|
}
|
|
|
|
console.log(
|
|
`${files.length} migrations, all journalled and all applied; ${expected.size} tables, columns and indexes verified present.`,
|
|
);
|
|
MIGRATIONEOF
|
|
node "$WORK/migration-objects.mjs" "$WORK/objects.txt" "$WORK/applied.txt"
|
|
|
|
- name: Seed is idempotent
|
|
# A seed that duplicates on a second run corrupts any database it is
|
|
# pointed at twice, and nobody notices until the counts look odd.
|
|
#
|
|
# EVERY table, not just contacts. The single-table version of this check
|
|
# would have passed a seed that duplicated anything else in the book.
|
|
run: |
|
|
set -euo pipefail
|
|
WORK=$(mktemp -d)
|
|
ROW_COUNTS="select table_name || ' ' || (xpath('/row/c/text()', query_to_xml(format('select count(*) as c from public.%I', table_name), false, true, '')))[1]::text from information_schema.tables where table_schema = 'public' and table_type = 'BASE TABLE' order by table_name"
|
|
counts() { docker exec "$PG_CONTAINER" psql -U pig -d pig -tAc "$ROW_COUNTS"; }
|
|
|
|
pnpm exec tsx packages/db/src/seed/index.ts > /dev/null
|
|
counts > "$WORK/before.txt"
|
|
pnpm exec tsx packages/db/src/seed/index.ts > /dev/null
|
|
counts > "$WORK/after.txt"
|
|
if ! diff -u "$WORK/before.txt" "$WORK/after.txt"; then
|
|
echo 'THE SEED IS NOT IDEMPOTENT. Every table whose count moved is listed above.'
|
|
exit 1
|
|
fi
|
|
echo "the seed left all $(wc -l < "$WORK/before.txt") tables unchanged on a second run"
|
|
|
|
- name: The demo seed is idempotent too
|
|
# `pnpm db:demo` had never been run by CI at all. The gate above covered
|
|
# the BASE seed only, which is how a non-idempotent demo seed once
|
|
# survived four rounds of review — and deploy/README.md tells operators
|
|
# to re-run this command, so duplication there corrupts the demo book on
|
|
# the second deploy rather than in a test.
|
|
#
|
|
# On a database of its own, deliberately. The demo book is a large,
|
|
# opinionated dataset; laying it over the database the rest of this job
|
|
# uses would move the row counts the E2E step asserts on, and that
|
|
# failure would read as a bug in the code under test.
|
|
run: |
|
|
set -euo pipefail
|
|
WORK=$(mktemp -d)
|
|
ROW_COUNTS="select table_name || ' ' || (xpath('/row/c/text()', query_to_xml(format('select count(*) as c from public.%I', table_name), false, true, '')))[1]::text from information_schema.tables where table_schema = 'public' and table_type = 'BASE TABLE' order by table_name"
|
|
|
|
# CREATE DATABASE is issued from `pig` rather than from `postgres`:
|
|
# `pig` is the database this job's own POSTGRES_DB created, so it is
|
|
# the one connection that is certain to exist whatever the image does.
|
|
DEMO_DB=pig_demo_idempotency
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig -c "drop database if exists ${DEMO_DB}" >/dev/null
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig -c "create database ${DEMO_DB}" >/dev/null
|
|
counts() { docker exec "$PG_CONTAINER" psql -U pig -d "$DEMO_DB" -tAc "$ROW_COUNTS"; }
|
|
|
|
# Exported for this step's shell only — each step gets its own, so the rest
|
|
# of the job keeps pointing at the database it was given.
|
|
export DATABASE_URL="${DATABASE_URL%/*}/${DEMO_DB}"
|
|
pnpm run db:migrate > /dev/null
|
|
pnpm run db:seed > /dev/null
|
|
|
|
# The base seed again, this time on the database the demo book is about to be
|
|
# laid over: "idempotent on an empty database" is not the property that
|
|
# matters to an operator re-running a seed.
|
|
counts > "$WORK/base-1.txt"
|
|
pnpm run db:seed > /dev/null
|
|
counts > "$WORK/base-2.txt"
|
|
if ! diff -u "$WORK/base-1.txt" "$WORK/base-2.txt"; then
|
|
echo 'THE BASE SEED IS NOT IDEMPOTENT on a demo database. Every table whose count moved is above.'
|
|
exit 1
|
|
fi
|
|
|
|
pnpm run db:demo > /dev/null
|
|
counts > "$WORK/demo-1.txt"
|
|
pnpm run db:demo > /dev/null
|
|
counts > "$WORK/demo-2.txt"
|
|
if ! diff -u "$WORK/demo-1.txt" "$WORK/demo-2.txt"; then
|
|
echo 'THE DEMO SEED IS NOT IDEMPOTENT. Every table whose count moved is above.'
|
|
echo 'pnpm db:demo is documented as safe to re-run, so this corrupts the demo book on the'
|
|
echo 'second deploy — and every figure on the marketing screenshots with it.'
|
|
exit 1
|
|
fi
|
|
echo "seed and demo both left all $(wc -l < "$WORK/demo-1.txt") tables unchanged on a second run"
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig -c "drop database ${DEMO_DB}" >/dev/null
|
|
|
|
- name: Critical path E2E against Postgres and Hono
|
|
run: pnpm run test:e2e
|
|
|
|
- name: Server boots and answers
|
|
run: |
|
|
NODE_ENV=development PIG_PORT=8930 pnpm exec tsx apps/api/src/server.ts &
|
|
for i in $(seq 1 30); do
|
|
curl -sf http://127.0.0.1:8930/api/health && break
|
|
sleep 1
|
|
done
|
|
curl -sf http://127.0.0.1:8930/api/health | grep -q '"ok":true'
|
|
|
|
- name: Piggy boots, and the API reports it enabled
|
|
# apps/api's own comment admits the gap this closes: its tests inject a
|
|
# resolver, so they pass whether or not the process is really wired to a
|
|
# Piggy. Here the relay is given nothing but environment variables and
|
|
# has to reach a Piggy that actually booted.
|
|
#
|
|
# It runs after the seed on purpose: with no identity provider every
|
|
# request is the development user, and that user is a seeded row.
|
|
run: |
|
|
LOGS=$(mktemp -d)
|
|
# Derived from the run id for the same reason Postgres's port is: this
|
|
# job shares the host's network namespace, so a fixed port belongs to
|
|
# the whole machine and two concurrent runs would fight over it.
|
|
PIGGY_PORT=$(( 30000 + (${{ github.run_id }} % 5000) ))
|
|
API_PORT=$(( 36000 + (${{ github.run_id }} % 5000) ))
|
|
# Worthless, and long enough for the schema's 32-character minimum.
|
|
INTERNAL_TOKEN='piggy-ci-internal-token-0123456789'
|
|
|
|
PIGGY_PID=''
|
|
API_PID=''
|
|
# There are two processes between the job's pid and the server that
|
|
# holds the port — pnpm launches tsx, tsx launches node — so the whole
|
|
# descendant tree has to go. Verified by watching a plain `kill` leave
|
|
# a Piggy behind, still holding its Postgres connections.
|
|
#
|
|
# SIGKILL, not the polite signal: nothing here needs a clean shutdown,
|
|
# and a server still listening when the next step runs is worse than
|
|
# an abrupt one.
|
|
stop() {
|
|
for pid in "$@"; do
|
|
[ -n "$pid" ] || continue
|
|
for child in $(pgrep -P "$pid" 2>/dev/null); do stop "$child"; done
|
|
kill -9 "$pid" 2>/dev/null || true
|
|
done
|
|
}
|
|
trap 'stop "$PIGGY_PID" "$API_PID"' EXIT
|
|
|
|
# Nothing here calls a model: the task queue is empty and the status
|
|
# route never reaches one. The inference base points at the discard
|
|
# port so that a future version which DID call out would fail loudly
|
|
# rather than quietly billing somebody's real endpoint.
|
|
PIGGY_INFERENCE_API_KEY=ci-stub-key \
|
|
PIGGY_INFERENCE_BASE=http://127.0.0.1:9/v1 \
|
|
PIGGY_INTERNAL_TOKEN="$INTERNAL_TOKEN" \
|
|
PIGGY_CHAT_HOST=127.0.0.1 \
|
|
PIGGY_CHAT_PORT="$PIGGY_PORT" \
|
|
pnpm exec tsx apps/piggy/src/main.ts > "$LOGS/piggy.log" 2>&1 &
|
|
PIGGY_PID=$!
|
|
|
|
for i in $(seq 1 30); do
|
|
curl -sf "http://127.0.0.1:${PIGGY_PORT}/internal/health" >/dev/null && break
|
|
sleep 1
|
|
done
|
|
HEALTH=$(curl -sf "http://127.0.0.1:${PIGGY_PORT}/internal/health" || true)
|
|
echo "GET /internal/health -> ${HEALTH:-<no response>}"
|
|
case "$HEALTH" in
|
|
*'"ok":true'*) ;;
|
|
*)
|
|
echo 'Piggy never answered. Its configuration schema rejects an incomplete environment on start, so the reason is usually the last line here:'
|
|
tail -30 "$LOGS/piggy.log"
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
NODE_ENV=development PIG_PORT="$API_PORT" \
|
|
PIGGY_ENABLED=true \
|
|
PIGGY_INTERNAL_URL="http://127.0.0.1:${PIGGY_PORT}" \
|
|
PIGGY_INTERNAL_TOKEN="$INTERNAL_TOKEN" \
|
|
pnpm exec tsx apps/api/src/server.ts > "$LOGS/api.log" 2>&1 &
|
|
API_PID=$!
|
|
|
|
for i in $(seq 1 30); do
|
|
curl -sf "http://127.0.0.1:${API_PORT}/api/health" >/dev/null && break
|
|
sleep 1
|
|
done
|
|
|
|
# The stored admin toggle is the inner gate, and an earlier step in
|
|
# this job has already created the settings row with Piggy off — the
|
|
# insert is ON CONFLICT DO NOTHING, so booting with PIGGY_ENABLED=true
|
|
# cannot correct it. Flip it here: what is under test is the wiring,
|
|
# not the switch.
|
|
docker exec "$PG_CONTAINER" psql -U pig -d pig \
|
|
-c 'update platform_settings set piggy_enabled = true' >/dev/null
|
|
|
|
STATUS=$(curl -sf "http://127.0.0.1:${API_PORT}/api/piggy/status" || true)
|
|
echo "GET /api/piggy/status -> ${STATUS:-<no response>}"
|
|
case "$STATUS" in
|
|
*'"enabled":true'*) ;;
|
|
*)
|
|
echo 'The API does not consider Piggy available, which is what the browser sees as a dock that never appears. PIGGY_ENABLED, PIGGY_INTERNAL_URL and PIGGY_INTERNAL_TOKEN are all read where the routes are composed; one of them is no longer reaching them.'
|
|
tail -30 "$LOGS/api.log"
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
- name: Front end builds
|
|
run: pnpm -F @pig/web run build
|
|
|
|
- name: Inline theme script still matches the deployed CSP hash
|
|
# The proxy allows exactly one inline script by hash. If the script
|
|
# changes and the CSP is not updated, dark-mode users get a white flash
|
|
# on every load and nothing anywhere reports an error.
|
|
run: |
|
|
node -e "
|
|
const fs=require('fs'), crypto=require('crypto');
|
|
const html=fs.readFileSync('apps/web/dist/index.html','utf8');
|
|
const m=html.match(/<script>([\s\S]*?)<\/script>/);
|
|
if(!m){ console.error('No inline script found in index.html'); process.exit(1); }
|
|
const hash='sha256-'+crypto.createHash('sha256').update(m[1]).digest('base64');
|
|
const expected='sha256-1tTDwCq+TCEyPDSZeYqW5HbmP+unUg8hrgRiZBiH/IU=';
|
|
if(hash!==expected){
|
|
console.error('Inline script hash changed.');
|
|
console.error(' now: '+hash);
|
|
console.error(' expected: '+expected);
|
|
console.error('Update the CSP in deploy/Caddyfile.example AND on the server,');
|
|
console.error('then update the expected hash in this workflow.');
|
|
process.exit(1);
|
|
}
|
|
console.log('CSP hash unchanged: '+hash);
|
|
"
|
|
|
|
- name: Compose file renders, and Piggy is passed every key it requires
|
|
# `docker compose config` is the only thing that reads docker-compose.yml
|
|
# in this repository. Without it, a typo in that file is discovered by
|
|
# the production host, at deploy time, as a container that restarts for
|
|
# ever with a message only `docker logs` shows.
|
|
run: |
|
|
WORK=$(mktemp -d)
|
|
|
|
# A dummy environment file rather than a real .env: these values are
|
|
# never used, they exist only because compose refuses to render while
|
|
# a `${VAR:?}` is unset. Passing --env-file also means a stray .env on
|
|
# the runner cannot supply a key and hide its absence from the check.
|
|
cat > "$WORK/dummy.env" <<'ENVEOF'
|
|
POSTGRES_PASSWORD=ci-dummy
|
|
PIG_PUBLIC_URL=http://localhost:8920
|
|
SUPABASE_URL=http://localhost:54321
|
|
SUPABASE_ANON_KEY=ci-dummy
|
|
ENVEOF
|
|
|
|
# --profile piggy, because a profiled service is otherwise omitted
|
|
# from the rendered output entirely — and it is the service under test.
|
|
docker compose --env-file "$WORK/dummy.env" --profile piggy config -q || {
|
|
echo 'If that complained about a missing variable, add it to the dummy environment above: a `${VAR:?}` in docker-compose.yml needs a value here, not the right value.'
|
|
exit 1
|
|
}
|
|
docker compose --env-file "$WORK/dummy.env" --profile piggy config --format json \
|
|
> "$WORK/compose.json"
|
|
|
|
# .mts, not .ts: this file lives outside the workspace, so tsx has no
|
|
# package.json to tell it the module system and would treat a .ts file
|
|
# as CommonJS, where top-level await is a syntax error.
|
|
cat > "$WORK/piggy-env-keys.mts" <<'CHECKEOF'
|
|
/**
|
|
* Every key the Piggy configuration schema requires must be handed to the piggy
|
|
* service by docker-compose.yml. One that is missing is not a failure anywhere
|
|
* else in this repository: both files are individually valid, and the gap only
|
|
* appears as a container exiting on boot with "Invalid Piggy configuration".
|
|
*
|
|
* Both sides are read at run time — the required keys by asking the schema
|
|
* itself, the provided keys from the compose file as Compose renders it. A list
|
|
* copied into this workflow would be right today and wrong by the next key.
|
|
*
|
|
* HOW THE REQUIRED KEYS ARE FOUND, and why not the obvious way. Parsing one
|
|
* empty-environment failure finds only what the base schema rejects, and misses
|
|
* everything a zod `.transform()` enforces — because the transform does not run
|
|
* until the base parse succeeds. PRIME_API_KEY is exactly that case: it is
|
|
* `.optional()` in the schema and required by the transform that resolves it
|
|
* against its legacy alias, so an empty environment never mentions it. The
|
|
* agent re-platform made that key mandatory for every deployment, and the gate
|
|
* that was supposed to notice a missing one could not see it at all.
|
|
*
|
|
* So the environment is filled in ROUNDS: parse, take the keys it named, give
|
|
* each a value, parse again. Each round can uncover requirements the previous
|
|
* round unblocked, and the loop ends when the configuration finally parses.
|
|
*/
|
|
import { readFileSync } from 'node:fs';
|
|
import { resolve } from 'node:path';
|
|
import { pathToFileURL } from 'node:url';
|
|
|
|
interface RenderedCompose {
|
|
services?: Record<string, { environment?: Record<string, string | null> }>;
|
|
}
|
|
|
|
const composeJsonPath = process.argv[2];
|
|
if (!composeJsonPath) {
|
|
console.error('Usage: piggy-env-keys.mts <rendered-compose.json>');
|
|
process.exit(1);
|
|
}
|
|
|
|
const CONFIG_TS = 'apps/piggy/src/config.ts';
|
|
|
|
const configModule = (await import(pathToFileURL(resolve(CONFIG_TS)).href)) as {
|
|
loadPiggyConfig: (env: NodeJS.ProcessEnv) => unknown;
|
|
};
|
|
|
|
/**
|
|
* One value for every key. Long, so it clears any minimum-length rule — the
|
|
* internal token demands 32 characters — and plain ASCII, so it is a valid
|
|
* string, number-free enums aside. If a future key needs something this cannot
|
|
* satisfy (a URL, an enum, an integer), the loop below stops making progress
|
|
* and says so by name rather than silently deciding the key is optional.
|
|
*/
|
|
const PROBE_VALUE = 'ci-probe-value-long-enough-for-a-32-character-minimum';
|
|
|
|
/** The keys a parse failure named, or undefined when it parsed. */
|
|
function keysRejectedBy(env: NodeJS.ProcessEnv): string[] | undefined {
|
|
try {
|
|
configModule.loadPiggyConfig(env);
|
|
return undefined;
|
|
} catch (error) {
|
|
const message = error instanceof Error ? error.message : String(error);
|
|
// loadPiggyConfig reports one indented "KEY: message" line per failure.
|
|
return [...message.matchAll(/^\s+([A-Z][A-Z0-9_]*):/gm)].flatMap(([, key]) => (key ? [key] : []));
|
|
}
|
|
}
|
|
|
|
function keysWithNoDefault(): string[] {
|
|
const env: NodeJS.ProcessEnv = {};
|
|
const required = new Set<string>();
|
|
// Bounded: every round must add at least one key, and the schema has a few
|
|
// dozen. An unbounded loop here would hang the runner rather than fail it.
|
|
for (let round = 0; round < 50; round += 1) {
|
|
const rejected = keysRejectedBy(env);
|
|
if (rejected === undefined) {
|
|
return [...required];
|
|
}
|
|
if (rejected.length === 0) {
|
|
throw new Error(
|
|
`${CONFIG_TS} rejected an environment without naming a key, so this check cannot tell which keys are required. Its error message format has changed.`,
|
|
);
|
|
}
|
|
const fresh = rejected.filter((key) => env[key] === undefined);
|
|
if (fresh.length === 0) {
|
|
throw new Error(
|
|
`The probe value does not satisfy ${rejected.join(', ')} — a URL, an enum or a number, most likely. Give this check a usable value for those keys; leaving it here would report them as optional.`,
|
|
);
|
|
}
|
|
for (const key of fresh) {
|
|
required.add(key);
|
|
env[key] = PROBE_VALUE;
|
|
}
|
|
}
|
|
throw new Error('The Piggy configuration never parsed, after 50 rounds of filling in the keys it asked for.');
|
|
}
|
|
|
|
/** Every key the schema names, required or not, read from its source. */
|
|
function keysTheSchemaKnows(): Set<string> {
|
|
const source = readFileSync(CONFIG_TS, 'utf8');
|
|
const keys = [...source.matchAll(/^ {2}([A-Z][A-Z0-9_]*):/gm)].flatMap(([, key]) => (key ? [key] : []));
|
|
if (keys.length === 0) {
|
|
throw new Error(`No configuration keys could be read out of ${CONFIG_TS}.`);
|
|
}
|
|
return new Set(keys);
|
|
}
|
|
|
|
const rendered = JSON.parse(readFileSync(composeJsonPath, 'utf8')) as RenderedCompose;
|
|
const piggy = rendered.services?.piggy;
|
|
if (!piggy) {
|
|
console.error('The rendered compose file has no `piggy` service.');
|
|
process.exit(1);
|
|
}
|
|
|
|
const provided = new Set(Object.keys(piggy.environment ?? {}));
|
|
|
|
/**
|
|
* A discovery that cannot finish must stop the build rather than return a short
|
|
* list. Every failure mode here ends with "and so this check now believes fewer
|
|
* keys are required than really are", which is the one outcome worse than no
|
|
* check at all.
|
|
*/
|
|
let required: string[];
|
|
try {
|
|
required = keysWithNoDefault();
|
|
} catch (error) {
|
|
console.error('This check can no longer work out which Piggy keys are required:');
|
|
console.error(` ${error instanceof Error ? error.message : String(error)}`);
|
|
process.exit(1);
|
|
}
|
|
console.log(`piggy requires ${required.length} key(s) with no default: ${required.join(', ')}`);
|
|
|
|
let failed = false;
|
|
|
|
const missing = required.filter((key) => !provided.has(key));
|
|
if (missing.length > 0) {
|
|
console.error(`docker-compose.yml never passes: ${missing.join(', ')}`);
|
|
console.error('The piggy container would exit on boot and restart for ever.');
|
|
console.error("Add each key to the piggy service's environment: block, and to .env.example.");
|
|
failed = true;
|
|
}
|
|
|
|
/**
|
|
* The key the whole re-platform runs on, asserted by name.
|
|
*
|
|
* Not a substitute for the discovery above — it is a canary FOR it. If a
|
|
* refactor moves the "at least one of PRIME_API_KEY / PIGGY_INFERENCE_API_KEY"
|
|
* rule somewhere the probe cannot see, the required list quietly shrinks and
|
|
* every check here still passes. This is the line that would not.
|
|
*/
|
|
if (!required.includes('PRIME_API_KEY')) {
|
|
console.error('PRIME_API_KEY is no longer reported as required by the Piggy configuration schema.');
|
|
console.error('Either it stopped being mandatory — it has not — or this check can no longer see');
|
|
console.error('which keys are, and a deployment missing its model credential would now pass CI.');
|
|
failed = true;
|
|
}
|
|
|
|
/**
|
|
* A key compose passes that the schema never reads is almost always a typo, and
|
|
* it is a silent one: unknown keys are ignored, so the container boots happily
|
|
* on the default the operator was trying to override. PIGGY_/PRIME_ only —
|
|
* anything else in that block belongs to the image or to Node.
|
|
*/
|
|
const known = keysTheSchemaKnows();
|
|
const unread = [...provided].filter((key) => /^(PIGGY|PRIME)_/.test(key) && !known.has(key));
|
|
if (unread.length > 0) {
|
|
console.error(`The piggy service is passed keys ${CONFIG_TS} never reads: ${unread.join(', ')}`);
|
|
console.error('A misspelt key is accepted in silence and the coded default applies instead.');
|
|
failed = true;
|
|
}
|
|
|
|
if (failed) process.exit(1);
|
|
|
|
console.log(
|
|
`Every required Piggy key is present in the piggy service, and all ${provided.size} keys it is passed are read by ${CONFIG_TS}.`,
|
|
);
|
|
CHECKEOF
|
|
|
|
pnpm exec tsx "$WORK/piggy-env-keys.mts" "$WORK/compose.json"
|
|
|
|
- name: Docker image builds
|
|
run: docker build -t pig:ci .
|
|
|
|
- name: Stop Postgres
|
|
if: always()
|
|
run: docker rm -f "$PG_CONTAINER" 2>/dev/null || true
|
|
|
|
# Publish the image that production will run.
|
|
#
|
|
# Only on a `release-*` tag. A push to main proves the commit is sound and
|
|
# stops there; tagging is the deliberate, human act that says "ship this".
|
|
# The production host polls the registry for the newest release tag and
|
|
# deploys it (scripts/autodeploy.sh) — which is how this gets automated
|
|
# WITHOUT the thing scripts/deploy.sh refuses to do. Nothing here holds a
|
|
# credential for cloud-2, and nothing here can execute anything on cloud-2.
|
|
#
|
|
# `gitea.ref` and `github.ref` are the same object in Gitea Actions; the
|
|
# gitea-prefixed spelling is used for the ref test because that is the one
|
|
# documented for tag conditions, and github.* elsewhere to match the job
|
|
# above.
|
|
publish:
|
|
needs: verify
|
|
if: startsWith(gitea.ref, 'refs/tags/release-')
|
|
runs-on: ubuntu-latest
|
|
|
|
env:
|
|
REGISTRY: git.karti.ai
|
|
# Gitea namespaces packages under the lowercased owner, so PIG/pig is
|
|
# published as pig/pig.
|
|
IMAGE: git.karti.ai/pig/pig
|
|
# THE POINT OF THIS VARIABLE: the shared act_runner on cloud-1 runs with
|
|
# `container.network: host`, and its docker config is visible to jobs
|
|
# from every other repository on that host. A plain `docker login` would
|
|
# leave a credential in ~/.docker/config.json that any of them could
|
|
# read. Pointing DOCKER_CONFIG at a per-run directory keeps the token out
|
|
# of the shared file entirely; the logout step below is the second belt.
|
|
DOCKER_CONFIG: /tmp/pig-docker-${{ github.run_id }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Log in to the Gitea registry
|
|
# The per-run Actions token, not a long-lived secret: it is minted for
|
|
# this run and dies with it. --password-stdin because an argument is
|
|
# visible in the runner's process list to anything else on that host.
|
|
run: |
|
|
mkdir -p "$DOCKER_CONFIG"
|
|
printf '%s' '${{ secrets.GITHUB_TOKEN }}' \
|
|
| docker login "$REGISTRY" -u '${{ github.actor }}' --password-stdin
|
|
|
|
- name: Build and push
|
|
# Both cloud-1 and cloud-2 are aarch64, so this is a native build and
|
|
# needs no --platform. The layer cache from the `verify` job's
|
|
# `docker build` is warm on this same daemon, so the rebuild is cheap.
|
|
#
|
|
# Two tags, always pushed together: the tag is what a human asked for,
|
|
# the short sha is what is unambiguous a year later when tags have been
|
|
# moved or deleted.
|
|
run: |
|
|
TAG="${GITHUB_REF#refs/tags/}"
|
|
SHORT_SHA=$(printf '%s' "${{ github.sha }}" | cut -c1-7)
|
|
echo "Publishing $IMAGE:$TAG and $IMAGE:$SHORT_SHA"
|
|
|
|
docker build -t "$IMAGE:$TAG" -t "$IMAGE:$SHORT_SHA" .
|
|
docker push "$IMAGE:$TAG"
|
|
docker push "$IMAGE:$SHORT_SHA"
|
|
|
|
# Print the digest: it is what the host poller compares against, and
|
|
# the only identifier that cannot be reassigned.
|
|
docker image inspect "$IMAGE:$TAG" \
|
|
--format '{{range .RepoDigests}}{{println .}}{{end}}'
|
|
|
|
- name: Log out
|
|
if: always()
|
|
# Runs even when the build failed, because a failed job that left a
|
|
# credential behind is exactly the leak this is guarding against.
|
|
run: |
|
|
docker logout "$REGISTRY" || true
|
|
rm -rf "$DOCKER_CONFIG"
|