Put Piggy on Prime Agent, and let it write to the book
Piggy was a hand-rolled OpenAI tool loop. It is now a Prime Agent session — Prime Intellect's own harness, embedded as a Node library — answering from PIG's tools and, for the first time, able to put information into the CRM rather than only read it out. The harness is a coding agent, so the first job was taking the coding agent away from it. `noTools: 'all'` plus an explicit allowlist leaves the model with PIG's ten `pig_*` tools and no bash, no filesystem, no IPython. That holds under attack: a hostile extension, a skill and a settings file planted in the agent's own directory, then `setActiveToolsByName` called with every built-in, still leaves ten tools, all ours. Both lines are load-bearing — `noTools` alone registers nothing, and the allowlist is what admits our own. Writing is gated rather than assumed. A change is proposed, not made: the tool returns a description, the transcript renders a diff card, and nothing reaches the database until someone presses Apply. Contracts, commitments, allocations and compliance always stop for a human whatever the mode. Every write runs through `executeMutation` as the calling user, so their capabilities and the audit trail apply exactly as they would to a human's. Four things about the SDK are wrong in its own documentation and cost a debugging cycle each: models.json does not resolve an env var name for `apiKey`, it sends the literal string; there is no built-in prime-inference provider in 0.84.1; a ResourceLoader you pass in is never reloaded for you; and the stock system prompt is a coding-assistant prompt that must be replaced — but replacing it also silently removes the tool list, because the harness only renders that section when it owns the prompt. AGENTS.md records all four. The expensive one was thinking level. The harness defaults to `medium`, and nemotron spent an entire 4,096-token budget reasoning and returned an empty answer. `low` was worse; `off` omits the parameter so the endpoint's default wins. An explicit `reasoning_effort: none` via `thinkingLevelMap` took a turn from 6,195 output tokens to 149. And a turn is now bounded. The harness loop is `while (true)` with no iteration cap; a runaway on a frontier model would have eaten the credit it is supposed to report on. Ceilings on model calls and tokens, enforced both through the harness hook and independently from the event stream, plus a per-user daily spend limit — and the ledger now records spend on turns that fail, which it previously discarded. Signing in lands on /piggy, which is a workspace: conversations down one side, the agent in the middle, what it did and what it cost beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,338 @@
|
||||
/**
|
||||
* The approval rendezvous, end to end, against a real database.
|
||||
*
|
||||
* `test/chat-server.test.ts` proves the choreography — card raised, decision
|
||||
* posted, single-use, deadlined, cancelled on abandonment — with a write tool
|
||||
* that only pretends to write. `test/write-tools.test.ts` proves the write tools
|
||||
* never open a transaction for a change nobody agreed to. Neither can prove the
|
||||
* sentence the whole feature rests on, which is what a user reads on the card:
|
||||
*
|
||||
* "Decline this and nothing changes."
|
||||
*
|
||||
* That is a claim about Postgres, made across two HTTP requests and a promise
|
||||
* parked in the middle of a turn. So this suite wires the real chat server to the
|
||||
* real `createPigWriteTools` against a real database, declines a real proposal
|
||||
* over `/internal/approve`, and then goes and looks at the rows. The applied case
|
||||
* runs the identical call to the same endpoint so that "untouched" means
|
||||
* something: the same request, answered the other way, does move the deal.
|
||||
*
|
||||
* No inference is involved and no key is needed — the harness is a fake that
|
||||
* drives the tool the way Prime Agent drives it, signal and all. What is real is
|
||||
* everything PIG owns.
|
||||
*
|
||||
* docker exec pig-ux-db psql -U pig -d postgres -c "CREATE DATABASE pig_c3_scratch"
|
||||
* DATABASE_URL=postgres://pig:pig@localhost:54330/pig_c3_scratch pnpm -F @pig/db run migrate
|
||||
* PIGGY_WRITE_DATABASE_URL=postgres://pig:pig@localhost:54330/pig_c3_scratch \
|
||||
* pnpm -F @pig/piggy run test:e2e
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import test, { after, before } from 'node:test';
|
||||
import type { AgentSession, AgentSessionEvent, ToolDefinition } from '@earendil-works/pi-coding-agent';
|
||||
import type { PiggyChatEvent } from '@pig/core';
|
||||
import {
|
||||
accounts,
|
||||
activities,
|
||||
agentRuns,
|
||||
createDatabase,
|
||||
demandDeals,
|
||||
users,
|
||||
type Database,
|
||||
} from '@pig/db';
|
||||
import { and, eq } from 'drizzle-orm';
|
||||
import { startPiggyChatServer, type PiggySessionFactory } from '../src/chat-server';
|
||||
|
||||
const databaseUrl = process.env.PIGGY_WRITE_DATABASE_URL;
|
||||
|
||||
if (!databaseUrl) {
|
||||
test.skip('the approval rendezvous E2E needs PIGGY_WRITE_DATABASE_URL pointing at a scratch database');
|
||||
}
|
||||
if (databaseUrl?.includes('pig_combined')) {
|
||||
throw new Error('The approval rendezvous E2E must never run against the development book.');
|
||||
}
|
||||
|
||||
const TOKEN = 'test-internal-token-for-piggy-0000000';
|
||||
|
||||
const db: Database = createDatabase({ url: databaseUrl ?? 'postgres://unused', max: 2 });
|
||||
const marker = `PIGGY-C3-${randomUUID()}`;
|
||||
const fixture = { userId: '', accountId: '', dealId: '' };
|
||||
let base = '';
|
||||
|
||||
function principal(): Record<string, unknown> {
|
||||
return {
|
||||
userId: fixture.userId,
|
||||
email: `${marker}@example.test`,
|
||||
name: 'Dana Okonjo',
|
||||
isPlatformAdmin: false,
|
||||
teams: [{ team: 'demand', role: 'member' }],
|
||||
via: 'jwt',
|
||||
scopes: ['read', 'write'],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The harness, reduced to what it does around a tool call.
|
||||
*
|
||||
* It hands the tool the abort signal — which is what lets a tool parked on an
|
||||
* approval discover that the reader has gone — and turns its result into the two
|
||||
* events the chat server translates.
|
||||
*/
|
||||
function fakeSessions(toolName: string, params: Record<string, unknown>): PiggySessionFactory {
|
||||
return async (options) => {
|
||||
const listeners = new Set<(event: AgentSessionEvent) => void>();
|
||||
const aborted = new AbortController();
|
||||
const session = {
|
||||
subscribe(listener: (event: AgentSessionEvent) => void) {
|
||||
listeners.add(listener);
|
||||
return () => listeners.delete(listener);
|
||||
},
|
||||
async prompt() {
|
||||
const emit = (event: AgentSessionEvent): void => {
|
||||
for (const listener of [...listeners]) listener(event);
|
||||
};
|
||||
const tool = options.tools.find((candidate) => candidate.name === toolName);
|
||||
assert.ok(tool, `${toolName} was not handed to the session`);
|
||||
emit({ type: 'tool_execution_start', toolCallId: 'call_1', toolName, args: params } as
|
||||
unknown as AgentSessionEvent);
|
||||
const result = await tool.execute(
|
||||
'call_1',
|
||||
params,
|
||||
aborted.signal,
|
||||
undefined,
|
||||
undefined as never,
|
||||
);
|
||||
emit({
|
||||
type: 'tool_execution_end',
|
||||
toolCallId: 'call_1',
|
||||
toolName,
|
||||
result,
|
||||
isError: false,
|
||||
} as unknown as AgentSessionEvent);
|
||||
emit({
|
||||
type: 'turn_end',
|
||||
message: { role: 'assistant', usage: { input: 120, output: 30 }, stopReason: 'stop' },
|
||||
toolResults: [],
|
||||
} as unknown as AgentSessionEvent);
|
||||
},
|
||||
async abort() {},
|
||||
dispose() {},
|
||||
} as unknown as AgentSession;
|
||||
|
||||
return {
|
||||
session,
|
||||
modelId: options.modelId ?? 'nvidia/nemotron-3-nano-30b-a3b',
|
||||
systemPrompt: 'You are Piggy.',
|
||||
dispose: () => aborted.abort(),
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
interface StreamReader {
|
||||
frames: PiggyChatEvent[];
|
||||
rest(): Promise<PiggyChatEvent[]>;
|
||||
}
|
||||
|
||||
/** Reads up to the approval card, then hands back a reader for the remainder. */
|
||||
async function readUntilApproval(response: Response): Promise<StreamReader> {
|
||||
const body = response.body;
|
||||
assert.ok(body, 'the turn should have streamed a body');
|
||||
const reader = body.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = '';
|
||||
|
||||
const drain = (chunk: Uint8Array | undefined, into: PiggyChatEvent[]): void => {
|
||||
buffer += decoder.decode(chunk, { stream: true });
|
||||
const lines = buffer.split('\n');
|
||||
buffer = lines.pop() ?? '';
|
||||
for (const line of lines) if (line) into.push(JSON.parse(line) as PiggyChatEvent);
|
||||
};
|
||||
|
||||
const frames: PiggyChatEvent[] = [];
|
||||
while (!frames.some((frame) => frame.type === 'approval_required')) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
drain(value, frames);
|
||||
}
|
||||
return {
|
||||
frames,
|
||||
rest: async () => {
|
||||
const tail: PiggyChatEvent[] = [];
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
drain(value, tail);
|
||||
}
|
||||
return tail;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** One turn, up to the card. The decision is posted while it is still open. */
|
||||
async function proposeStageChange(stage: string): Promise<StreamReader> {
|
||||
const response = await fetch(`${base}/internal/chat`, {
|
||||
method: 'POST',
|
||||
headers: { authorization: `Bearer ${TOKEN}`, 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
principal: principal(),
|
||||
message: `Move the deal to ${stage}.`,
|
||||
mode: 'confirm',
|
||||
conversationId: `conv-${stage}`,
|
||||
}),
|
||||
});
|
||||
assert.equal(response.status, 200);
|
||||
return readUntilApproval(response);
|
||||
}
|
||||
|
||||
async function decide(
|
||||
conversationId: string,
|
||||
changeId: string,
|
||||
decision: 'apply' | 'reject',
|
||||
): Promise<number> {
|
||||
const response = await fetch(`${base}/internal/approve`, {
|
||||
method: 'POST',
|
||||
headers: { authorization: `Bearer ${TOKEN}`, 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ conversationId, changeId, decision }),
|
||||
});
|
||||
return response.status;
|
||||
}
|
||||
|
||||
function askedChangeId(reader: StreamReader): string {
|
||||
const asked = reader.frames.find((frame) => frame.type === 'approval_required');
|
||||
assert.ok(asked && asked.type === 'approval_required', 'no approval card was raised');
|
||||
// The card a person reads must name the record and the movement, or approving
|
||||
// it is a click on a uuid.
|
||||
assert.match(asked.change.summary, /Northwind/);
|
||||
return asked.change.id;
|
||||
}
|
||||
|
||||
let server: ReturnType<typeof startPiggyChatServer> | undefined;
|
||||
|
||||
before(async () => {
|
||||
if (!databaseUrl) return;
|
||||
const [user] = await db
|
||||
.insert(users)
|
||||
.values({ email: `${marker}@example.test`, name: 'Dana Okonjo', authSubject: randomUUID() })
|
||||
.returning({ id: users.id });
|
||||
assert.ok(user);
|
||||
fixture.userId = user.id;
|
||||
|
||||
const [account] = await db
|
||||
.insert(accounts)
|
||||
.values({ name: `${marker} Northwind Robotics`, side: 'demand' })
|
||||
.returning({ id: accounts.id });
|
||||
assert.ok(account);
|
||||
fixture.accountId = account.id;
|
||||
|
||||
const [deal] = await db
|
||||
.insert(demandDeals)
|
||||
.values({ accountId: account.id, name: `${marker} Northwind H200`, stage: 'proposal' })
|
||||
.returning({ id: demandDeals.id });
|
||||
assert.ok(deal);
|
||||
fixture.dealId = deal.id;
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
server?.close();
|
||||
if (!databaseUrl) return;
|
||||
// The run rows only null their user out on delete, so they are cleared by
|
||||
// hand; everything else cascades from the account.
|
||||
if (fixture.userId) await db.delete(agentRuns).where(eq(agentRuns.principalUserId, fixture.userId));
|
||||
if (fixture.accountId) await db.delete(accounts).where(eq(accounts.id, fixture.accountId));
|
||||
if (fixture.userId) await db.delete(users).where(eq(users.id, fixture.userId));
|
||||
await db.$client.end({ timeout: 5 });
|
||||
});
|
||||
|
||||
function start(stage: string): void {
|
||||
server?.close();
|
||||
server = startPiggyChatServer(db, {
|
||||
port: 0,
|
||||
internalToken: TOKEN,
|
||||
// The real write tools, against the real database, as the real caller.
|
||||
createReadTools: () => [] as ToolDefinition[],
|
||||
createSession: fakeSessions('pig_update_deal_stage', {
|
||||
dealType: 'demand',
|
||||
dealId: fixture.dealId,
|
||||
stage,
|
||||
reason: 'Legal cleared the MSA this morning.',
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function listen(): Promise<void> {
|
||||
assert.ok(server);
|
||||
await new Promise((resolve) => server?.once('listening', resolve));
|
||||
base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||
}
|
||||
|
||||
test('a declined proposal leaves the book exactly as it was', { skip: !databaseUrl }, async () => {
|
||||
start('procurement');
|
||||
await listen();
|
||||
|
||||
const [before] = await db.select().from(demandDeals).where(eq(demandDeals.id, fixture.dealId));
|
||||
const auditBefore = await db
|
||||
.select()
|
||||
.from(activities)
|
||||
.where(eq(activities.demandDealId, fixture.dealId));
|
||||
|
||||
const reader = await proposeStageChange('procurement');
|
||||
const changeId = askedChangeId(reader);
|
||||
// Still nothing written: the turn is parked on a promise, mid-tool-call.
|
||||
const [during] = await db.select().from(demandDeals).where(eq(demandDeals.id, fixture.dealId));
|
||||
assert.equal(during?.stage, before?.stage, 'the deal moved while the card was still on screen');
|
||||
|
||||
assert.equal(await decide('conv-procurement', changeId, 'reject'), 202);
|
||||
const tail = await reader.rest();
|
||||
|
||||
const [after] = await db.select().from(demandDeals).where(eq(demandDeals.id, fixture.dealId));
|
||||
assert.equal(after?.stage, 'proposal', 'a declined change moved the deal anyway');
|
||||
assert.equal(after?.updatedAt?.getTime(), before?.updatedAt?.getTime(), 'the row was touched');
|
||||
const auditAfter = await db
|
||||
.select()
|
||||
.from(activities)
|
||||
.where(eq(activities.demandDealId, fixture.dealId));
|
||||
assert.equal(auditAfter.length, auditBefore.length, 'a declined change wrote an audit row');
|
||||
|
||||
// And the model is told the truth, in the tool result it will summarise from.
|
||||
const result = tail.find((frame) => frame.type === 'tool_result');
|
||||
assert.ok(result && result.type === 'tool_result');
|
||||
assert.equal(result.ok, true, 'a decline is an answer, not a tool failure');
|
||||
assert.deepEqual(result.result, {
|
||||
tool: 'pig_update_deal_stage',
|
||||
kind: 'deal',
|
||||
status: 'declined',
|
||||
reason: 'declined by the user',
|
||||
});
|
||||
const settled = tail.find((frame) => frame.type === 'approval_resolved');
|
||||
assert.equal(settled?.type === 'approval_resolved' ? settled.decision : null, 'reject');
|
||||
});
|
||||
|
||||
test('the same call, approved, does move the deal', { skip: !databaseUrl }, async () => {
|
||||
start('deployment');
|
||||
await listen();
|
||||
|
||||
const reader = await proposeStageChange('deployment');
|
||||
const changeId = askedChangeId(reader);
|
||||
assert.equal(await decide('conv-deployment', changeId, 'apply'), 202);
|
||||
const tail = await reader.rest();
|
||||
|
||||
const [after] = await db.select().from(demandDeals).where(eq(demandDeals.id, fixture.dealId));
|
||||
assert.equal(after?.stage, 'deployment');
|
||||
const [audit] = await db
|
||||
.select()
|
||||
.from(activities)
|
||||
.where(and(eq(activities.demandDealId, fixture.dealId), eq(activities.type, 'stage_change')));
|
||||
assert.ok(audit, 'the applied write left the audit row the mutation convention writes');
|
||||
assert.equal(audit.actorUserId, fixture.userId, 'written as the caller, never as Piggy itself');
|
||||
assert.equal(audit.meta?.actorAgent, 'piggy');
|
||||
|
||||
const result = tail.find((frame) => frame.type === 'tool_result');
|
||||
assert.equal(
|
||||
result?.type === 'tool_result' && (result.result as { status?: string }).status,
|
||||
'applied',
|
||||
);
|
||||
// Answering again cannot apply it twice: the id was consumed when it settled.
|
||||
assert.equal(await decide('conv-deployment', changeId, 'apply'), 404);
|
||||
const [unchanged] = await db.select().from(demandDeals).where(eq(demandDeals.id, fixture.dealId));
|
||||
assert.equal(unchanged?.stage, 'deployment');
|
||||
});
|
||||
Reference in New Issue
Block a user