Put Piggy on Prime Agent, and let it write to the book
Piggy was a hand-rolled OpenAI tool loop. It is now a Prime Agent session — Prime Intellect's own harness, embedded as a Node library — answering from PIG's tools and, for the first time, able to put information into the CRM rather than only read it out. The harness is a coding agent, so the first job was taking the coding agent away from it. `noTools: 'all'` plus an explicit allowlist leaves the model with PIG's ten `pig_*` tools and no bash, no filesystem, no IPython. That holds under attack: a hostile extension, a skill and a settings file planted in the agent's own directory, then `setActiveToolsByName` called with every built-in, still leaves ten tools, all ours. Both lines are load-bearing — `noTools` alone registers nothing, and the allowlist is what admits our own. Writing is gated rather than assumed. A change is proposed, not made: the tool returns a description, the transcript renders a diff card, and nothing reaches the database until someone presses Apply. Contracts, commitments, allocations and compliance always stop for a human whatever the mode. Every write runs through `executeMutation` as the calling user, so their capabilities and the audit trail apply exactly as they would to a human's. Four things about the SDK are wrong in its own documentation and cost a debugging cycle each: models.json does not resolve an env var name for `apiKey`, it sends the literal string; there is no built-in prime-inference provider in 0.84.1; a ResourceLoader you pass in is never reloaded for you; and the stock system prompt is a coding-assistant prompt that must be replaced — but replacing it also silently removes the tool list, because the harness only renders that section when it owns the prompt. AGENTS.md records all four. The expensive one was thinking level. The harness defaults to `medium`, and nemotron spent an entire 4,096-token budget reasoning and returned an empty answer. `low` was worse; `off` omits the parameter so the endpoint's default wins. An explicit `reasoning_effort: none` via `thinkingLevelMap` took a turn from 6,195 output tokens to 149. And a turn is now bounded. The harness loop is `while (true)` with no iteration cap; a runaway on a frontier model would have eaten the credit it is supposed to report on. Ceilings on model calls and tokens, enforced both through the harness hook and independently from the event stream, plus a per-user daily spend limit — and the ledger now records spend on turns that fail, which it previously discarded. Signing in lands on /piggy, which is a workspace: conversations down one side, the agent in the middle, what it did and what it cost beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,168 @@
|
||||
/**
|
||||
* The bridge from PIG's zod-declared tools to Prime Agent's typebox ones.
|
||||
*
|
||||
* Two of these cases exist because the defect they pin is invisible to tsc and
|
||||
* survived a release each.
|
||||
*
|
||||
* The optional-parameter round trip is the first. `zodToJsonSchema(..., {
|
||||
* target: 'openAi' })` emits an optional field as required-and-nullable and
|
||||
* drops a `.describe()` attached to the optional wrapper, so a parameter that
|
||||
* reads as thoroughly documented in the source reaches the model with no
|
||||
* sentence at all and a demand that it be sent. Nothing about that typechecks.
|
||||
*
|
||||
* The snippet case is the second. A custom tool without `promptSnippet` is
|
||||
* registered, callable, and absent from the system prompt's tool list — so the
|
||||
* model never learns it exists, and the only symptom is Piggy declining to look
|
||||
* something up it is perfectly able to look up.
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import type { ExtensionContext } from '@earendil-works/pi-coding-agent';
|
||||
import type { Database } from '@pig/db';
|
||||
import { z } from 'zod';
|
||||
import { toPrimeTools } from '../src/agent/tool-bridge';
|
||||
import { createInteractivePigTools } from '../src/chat-tools';
|
||||
import { defineTool, type AgentTool } from '../src/provider';
|
||||
|
||||
/** The harness hands `execute` a context these tools never read. */
|
||||
const ctx = {} as ExtensionContext;
|
||||
|
||||
interface ParameterSchema {
|
||||
type: string;
|
||||
required?: string[];
|
||||
properties?: Record<string, { description?: string; type?: unknown }>;
|
||||
additionalProperties?: boolean;
|
||||
$schema?: string;
|
||||
}
|
||||
|
||||
function schemaOf(tool: { parameters: unknown }): ParameterSchema {
|
||||
return tool.parameters as ParameterSchema;
|
||||
}
|
||||
|
||||
function onlyTool(tool: AgentTool) {
|
||||
const [bridged] = toPrimeTools([tool]);
|
||||
assert.ok(bridged, 'the bridge returned no tool');
|
||||
return bridged;
|
||||
}
|
||||
|
||||
test('an optional parameter survives the bridge as optional, with its description', () => {
|
||||
const bridged = onlyTool(
|
||||
defineTool({
|
||||
name: 'pig_probe',
|
||||
description: 'Probe the bridge. Never registered on a real session.',
|
||||
inputSchema: z
|
||||
.object({
|
||||
needed: z.string().describe('The one required parameter.'),
|
||||
// Both spellings the existing tools use. `.nullish()` is what
|
||||
// `chat-tools.ts` and `page-tools.ts` write, to survive a model that
|
||||
// sends an explicit null; `.optional()` is the plain case.
|
||||
describedBeforeWrapper: z.number().int().describe('Horizon in days.').nullish(),
|
||||
describedAfterWrapper: z.string().optional().describe('A trailing note.'),
|
||||
})
|
||||
.strict(),
|
||||
execute: async () => ({}),
|
||||
}),
|
||||
);
|
||||
|
||||
const schema = schemaOf(bridged);
|
||||
assert.deepEqual(schema.required, ['needed'], 'only the required parameter is required');
|
||||
assert.equal(
|
||||
schema.properties?.describedBeforeWrapper?.description,
|
||||
'Horizon in days.',
|
||||
'a description applied before the optional wrapper reaches the model',
|
||||
);
|
||||
assert.equal(
|
||||
schema.properties?.describedAfterWrapper?.description,
|
||||
'A trailing note.',
|
||||
'a description applied after the optional wrapper reaches the model too',
|
||||
);
|
||||
assert.equal(schema.additionalProperties, false, 'a strict zod object stays closed');
|
||||
// Meta about the document rather than about the parameters; the provider has
|
||||
// no use for it and it is paid for on every message.
|
||||
assert.equal(schema.$schema, undefined);
|
||||
});
|
||||
|
||||
test('every bridged tool carries a promptSnippet, or it is invisible to the model', () => {
|
||||
const bridged = toPrimeTools(createInteractivePigTools({} as Database, undefined));
|
||||
assert.ok(bridged.length > 0);
|
||||
for (const tool of bridged) {
|
||||
assert.ok(tool.promptSnippet, `${tool.name} has no promptSnippet`);
|
||||
assert.ok(!tool.promptSnippet.includes('\n'), `${tool.name} snippet is not one line`);
|
||||
assert.ok(tool.label, `${tool.name} has no label`);
|
||||
assert.ok(
|
||||
tool.promptSnippet.length < tool.description.length,
|
||||
`${tool.name} snippet should be terser than its description`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('the boundary assertion is a second gate behind noTools', () => {
|
||||
const outsiders = ['bash_run', 'pig_bash', 'run_shell', 'read_file'];
|
||||
for (const name of outsiders) {
|
||||
assert.throws(
|
||||
() =>
|
||||
toPrimeTools([
|
||||
defineTool({
|
||||
name,
|
||||
description: 'Should never reach the harness.',
|
||||
inputSchema: z.object({}).strict(),
|
||||
execute: async () => ({}),
|
||||
}),
|
||||
]),
|
||||
/outside the PIG tool boundary/,
|
||||
`${name} was allowed through`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('a bridged tool returns the payload it returns today, byte for byte', async () => {
|
||||
const payload = { headline: 'Two commitments are idle.', idleHours: 1_200, cheapest: null };
|
||||
const bridged = onlyTool(
|
||||
defineTool({
|
||||
name: 'pig_probe_payload',
|
||||
description: 'Return a fixed payload.',
|
||||
inputSchema: z.object({ withinDays: z.number().int().nullish() }).strict(),
|
||||
execute: async () => payload,
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await bridged.execute('call-1', { withinDays: null }, undefined, undefined, ctx);
|
||||
const [content] = result.content;
|
||||
assert.equal(content?.type, 'text');
|
||||
assert.equal(
|
||||
content?.type === 'text' ? content.text : '',
|
||||
JSON.stringify(payload),
|
||||
'the model sees the tool payload unchanged',
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.details,
|
||||
{ tool: 'pig_probe_payload', result: payload },
|
||||
'the structured payload rides on details for the chat server',
|
||||
);
|
||||
});
|
||||
|
||||
test('the zod schema, not the typebox one, is what actually guards execute', async () => {
|
||||
let executed = 0;
|
||||
const bridged = onlyTool(
|
||||
defineTool({
|
||||
name: 'pig_probe_gate',
|
||||
description: 'Count executions.',
|
||||
inputSchema: z.object({ query: z.string().min(2).max(8) }).strict(),
|
||||
execute: async () => {
|
||||
executed += 1;
|
||||
return {};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
// The harness forwards tool arguments untouched — it never checks them
|
||||
// against `parameters` — so anything the zod parse does not stop reaches a
|
||||
// query. Each of these is something a model has actually sent.
|
||||
for (const bad of [{ query: 'x' }, { query: 'x'.repeat(50) }, { query: 'ok', extra: 1 }, {}]) {
|
||||
await assert.rejects(() => bridged.execute('call', bad, undefined, undefined, ctx));
|
||||
}
|
||||
assert.equal(executed, 0, 'no invalid call reached the tool body');
|
||||
|
||||
await bridged.execute('call', { query: 'Halcyon' }, undefined, undefined, ctx);
|
||||
assert.equal(executed, 1);
|
||||
});
|
||||
Reference in New Issue
Block a user