claude
f0173440e4
Piggy was a hand-rolled OpenAI tool loop. It is now a Prime Agent session — Prime Intellect's own harness, embedded as a Node library — answering from PIG's tools and, for the first time, able to put information into the CRM rather than only read it out. The harness is a coding agent, so the first job was taking the coding agent away from it. `noTools: 'all'` plus an explicit allowlist leaves the model with PIG's ten `pig_*` tools and no bash, no filesystem, no IPython. That holds under attack: a hostile extension, a skill and a settings file planted in the agent's own directory, then `setActiveToolsByName` called with every built-in, still leaves ten tools, all ours. Both lines are load-bearing — `noTools` alone registers nothing, and the allowlist is what admits our own. Writing is gated rather than assumed. A change is proposed, not made: the tool returns a description, the transcript renders a diff card, and nothing reaches the database until someone presses Apply. Contracts, commitments, allocations and compliance always stop for a human whatever the mode. Every write runs through `executeMutation` as the calling user, so their capabilities and the audit trail apply exactly as they would to a human's. Four things about the SDK are wrong in its own documentation and cost a debugging cycle each: models.json does not resolve an env var name for `apiKey`, it sends the literal string; there is no built-in prime-inference provider in 0.84.1; a ResourceLoader you pass in is never reloaded for you; and the stock system prompt is a coding-assistant prompt that must be replaced — but replacing it also silently removes the tool list, because the harness only renders that section when it owns the prompt. AGENTS.md records all four. The expensive one was thinking level. The harness defaults to `medium`, and nemotron spent an entire 4,096-token budget reasoning and returned an empty answer. `low` was worse; `off` omits the parameter so the endpoint's default wins. An explicit `reasoning_effort: none` via `thinkingLevelMap` took a turn from 6,195 output tokens to 149. And a turn is now bounded. The harness loop is `while (true)` with no iteration cap; a runaway on a frontier model would have eaten the credit it is supposed to report on. Ceilings on model calls and tokens, enforced both through the harness hook and independently from the event stream, plus a per-user daily spend limit — and the ledger now records spend on turns that fail, which it previously discarded. Signing in lands on /piggy, which is a workspace: conversations down one side, the agent in the middle, what it did and what it cost beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
237 lines
10 KiB
Bash
Executable File
237 lines
10 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Poll the registry for a new release and deploy it. Runs on the production
|
|
# host, on a timer. Installation: see deploy/README.md.
|
|
#
|
|
# This is the half of "push to deploy" that does not require handing the CI
|
|
# runner a key to this machine. CI publishes an image when a human tags
|
|
# `release-*`; this pulls. The only credential involved lives HERE, is
|
|
# read-only, and can do nothing but fetch a manifest.
|
|
#
|
|
# What it does, in order:
|
|
#
|
|
# 1. Ask the registry for the newest release-* tag.
|
|
# 2. Compare that tag's manifest digest with the digest of the image the
|
|
# running app container was started from — and the piggy container's too,
|
|
# when Piggy is enabled, because it is the same image and a Piggy left
|
|
# behind runs old code against a migrated schema. All equal -> exit 0,
|
|
# silently. This is the normal case and it happens every five minutes.
|
|
# 3. Otherwise check the working tree out at that tag — the compose file and
|
|
# the migrations must come from the same commit as the image — and hand
|
|
# over to scripts/deploy.sh with PIG_IMAGE set.
|
|
#
|
|
# A failed deploy is recorded and NOT retried. A poller that reattempts the
|
|
# same broken release every five minutes turns one bad tag into a restart loop;
|
|
# deleting the record (or tagging a new release) is the human's signal to try
|
|
# again.
|
|
|
|
set -euo pipefail
|
|
|
|
# git checkout rewrites this file while bash is reading it — bash reads a
|
|
# script incrementally, by byte offset, so a checkout mid-run resumes at an
|
|
# offset into different content. Re-exec from a private copy before touching
|
|
# git. This has a long history of producing genuinely baffling syntax errors.
|
|
if [ "${PIG_AUTODEPLOY_REEXEC:-}" != '1' ]; then
|
|
_copy=$(mktemp -t pig-autodeploy.XXXXXX)
|
|
cat "$0" > "$_copy"
|
|
PIG_AUTODEPLOY_REEXEC=1 exec bash "$_copy" "$@"
|
|
fi
|
|
trap 'rm -f "$0"' EXIT
|
|
|
|
REPO_DIR="${PIG_REPO_DIR:-/opt/pig}"
|
|
REGISTRY="${PIG_REGISTRY:-git.karti.ai}"
|
|
# Gitea namespaces packages under the lowercased owner: PIG/pig -> pig/pig.
|
|
IMAGE_REPO="${PIG_IMAGE_REPO:-pig/pig}"
|
|
TOKEN_FILE="${PIG_REGISTRY_TOKEN_FILE:-/etc/pig/registry-token}"
|
|
REGISTRY_USER="${PIG_REGISTRY_USER:-pig-deploy}"
|
|
STATE_DIR="${PIG_STATE_DIR:-/var/lib/pig}"
|
|
TAG_PREFIX="${PIG_RELEASE_TAG_PREFIX:-release-}"
|
|
|
|
IMAGE_NAME="$REGISTRY/$IMAGE_REPO"
|
|
FAILED_MARKER="$STATE_DIR/failed-release"
|
|
|
|
log() { printf '%s autodeploy: %s\n' "$(date -Is)" "$*"; }
|
|
die() { log "ERROR: $*"; exit 1; }
|
|
|
|
# Only one deploy at a time. The systemd timer will not overlap a run with
|
|
# itself, but a human running this by hand during a scheduled run would.
|
|
LOCK="${PIG_LOCK_FILE:-/var/lock/pig-autodeploy.lock}"
|
|
exec 9>"$LOCK"
|
|
if ! flock -n 9; then
|
|
log "another run holds $LOCK; nothing to do"
|
|
exit 0
|
|
fi
|
|
|
|
[ -r "$TOKEN_FILE" ] || die "no readable registry token at $TOKEN_FILE"
|
|
# The token is the only secret here and it is pull-only, but a world-readable
|
|
# credential on a production host is still worth refusing.
|
|
PERMS=$(stat -c '%a' "$TOKEN_FILE")
|
|
case "$PERMS" in
|
|
600|400) ;;
|
|
*) die "$TOKEN_FILE is mode $PERMS; must be 0600" ;;
|
|
esac
|
|
REGISTRY_TOKEN=$(tr -d '\r\n' < "$TOKEN_FILE")
|
|
[ -n "$REGISTRY_TOKEN" ] || die "$TOKEN_FILE is empty"
|
|
|
|
cd "$REPO_DIR" || die "no checkout at $REPO_DIR"
|
|
|
|
# ---------------------------------------------------------------- registry
|
|
#
|
|
# Gitea's registry speaks the standard v2 token dance: an unauthenticated
|
|
# request gets a 401 pointing at /v2/token. Exchange the pull-only credential
|
|
# for a bearer token scoped to this one repository and fall back to basic auth
|
|
# if the exchange gives nothing, because Gitea accepts both and which one you
|
|
# get has changed across versions.
|
|
AUTH_ARGS=()
|
|
BEARER=$(curl -sS --max-time 20 -u "$REGISTRY_USER:$REGISTRY_TOKEN" \
|
|
"https://$REGISTRY/v2/token?service=$REGISTRY&scope=repository:$IMAGE_REPO:pull" \
|
|
2>/dev/null | sed -n 's/.*"token":"\([^"]*\)".*/\1/p' | head -n1 || true)
|
|
if [ -n "$BEARER" ]; then
|
|
AUTH_ARGS=(-H "Authorization: Bearer $BEARER")
|
|
else
|
|
AUTH_ARGS=(-u "$REGISTRY_USER:$REGISTRY_TOKEN")
|
|
fi
|
|
|
|
MANIFEST_ACCEPT=(
|
|
-H 'Accept: application/vnd.oci.image.index.v1+json'
|
|
-H 'Accept: application/vnd.oci.image.manifest.v1+json'
|
|
-H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json'
|
|
-H 'Accept: application/vnd.docker.distribution.manifest.v2+json'
|
|
)
|
|
|
|
TAGS_JSON=$(curl -sS --max-time 30 "${AUTH_ARGS[@]}" \
|
|
"https://$REGISTRY/v2/$IMAGE_REPO/tags/list?n=1000") \
|
|
|| die "could not reach https://$REGISTRY/v2/$IMAGE_REPO/tags/list"
|
|
|
|
# sort -V rather than sort: release-10 must come after release-9. This assumes
|
|
# release tags sort sensibly — date-stamped or semver. A tag scheme that does
|
|
# not is a tag scheme this poller will deploy in the wrong order.
|
|
LATEST_TAG=$(printf '%s' "$TAGS_JSON" | tr ',' '\n' \
|
|
| sed -n "s/.*\"\(${TAG_PREFIX}[^\"]*\)\".*/\1/p" | sort -V | tail -n1)
|
|
[ -n "$LATEST_TAG" ] || { log "no ${TAG_PREFIX}* tags published yet"; exit 0; }
|
|
|
|
REMOTE_DIGEST=$(curl -sSI --max-time 30 "${AUTH_ARGS[@]}" "${MANIFEST_ACCEPT[@]}" \
|
|
"https://$REGISTRY/v2/$IMAGE_REPO/manifests/$LATEST_TAG" \
|
|
| tr -d '\r' | sed -n 's/^[Dd]ocker-[Cc]ontent-[Dd]igest:[[:space:]]*//p' | tail -n1)
|
|
[ -n "$REMOTE_DIGEST" ] || die "no digest for $IMAGE_NAME:$LATEST_TAG"
|
|
|
|
# ---------------------------------------------------------------- compare
|
|
#
|
|
# Compare digests, not tags. A tag can be moved; a digest is the content. This
|
|
# also means a re-pushed tag redeploys, which is what you want the one time it
|
|
# matters.
|
|
|
|
# Read one key out of the deployed .env WITHOUT sourcing it: sourcing an
|
|
# environment file executes it, and this one holds every secret the deployment
|
|
# has. The same helper as scripts/deploy.sh, for the same reason.
|
|
env_value() {
|
|
[ -f .env ] || return 0
|
|
sed -n "s/^[[:space:]]*$1=//p" .env | tail -n1 | sed -e 's/^"\(.*\)"$/\1/' -e "s/^'\(.*\)'\$/\1/"
|
|
}
|
|
|
|
# The words the API accepts, from `envBoolean` in apps/api/src/lib/config.ts.
|
|
is_true() {
|
|
# Whitespace and a trailing inline comment are both dropped by compose before
|
|
# a container sees the value; read it the same way deploy.sh does.
|
|
local value=${1%%[[:space:]]#*}
|
|
case "$(printf '%s' "$value" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')" in
|
|
1 | true | yes | on) return 0 ;;
|
|
*) return 1 ;;
|
|
esac
|
|
}
|
|
|
|
# The piggy service is profile-gated, and compose skips a profile-gated service
|
|
# silently — with no profile, `ps -q piggy` prints nothing at all unless that
|
|
# container happens to be running already. Without this, a Piggy left on an
|
|
# older image is invisible here: the app matches the newest tag, this reports
|
|
# "up to date" every five minutes, and the agent runs last month's code.
|
|
COMPOSE_PROFILES=''
|
|
if is_true "$(env_value PIGGY_ENABLED)"; then
|
|
COMPOSE_PROFILES='piggy'
|
|
fi
|
|
export COMPOSE_PROFILES
|
|
|
|
# The registry digest of the image a running container was started from, or the
|
|
# empty string when there is no such container.
|
|
running_digest() {
|
|
local cid image
|
|
cid=$(docker compose -p pig ps -q "$1" 2>/dev/null | head -n1 || true)
|
|
[ -n "$cid" ] || return 0
|
|
image=$(docker inspect -f '{{.Image}}' "$cid" 2>/dev/null || true)
|
|
[ -n "$image" ] || return 0
|
|
docker image inspect "$image" --format '{{range .RepoDigests}}{{println .}}{{end}}' 2>/dev/null \
|
|
| sed -n "s|^$IMAGE_NAME@||p" | head -n1 || true
|
|
}
|
|
|
|
RUNNING_DIGEST=$(running_digest app)
|
|
PIGGY_DIGEST=''
|
|
[ -z "$COMPOSE_PROFILES" ] || PIGGY_DIGEST=$(running_digest piggy)
|
|
|
|
# Both halves of the release, or neither. app and piggy are the same image
|
|
# running two commands, so a piggy behind the app is a deploy that only half
|
|
# happened — and it is the half that writes to the database.
|
|
if [ "$RUNNING_DIGEST" = "$REMOTE_DIGEST" ] \
|
|
&& { [ -z "$COMPOSE_PROFILES" ] || [ "$PIGGY_DIGEST" = "$REMOTE_DIGEST" ]; }; then
|
|
log "up to date at $LATEST_TAG ($REMOTE_DIGEST)"
|
|
exit 0
|
|
fi
|
|
|
|
if [ -f "$FAILED_MARKER" ] && [ "$(cat "$FAILED_MARKER")" = "$REMOTE_DIGEST" ]; then
|
|
log "$LATEST_TAG ($REMOTE_DIGEST) already failed to deploy; not retrying."
|
|
log "Remove $FAILED_MARKER to try again, or publish a new release tag."
|
|
exit 0
|
|
fi
|
|
|
|
log "deploying $LATEST_TAG"
|
|
log " running: ${RUNNING_DIGEST:-<none>}"
|
|
[ -z "$COMPOSE_PROFILES" ] || log " piggy: ${PIGGY_DIGEST:-<none>}"
|
|
log " wanted: $REMOTE_DIGEST"
|
|
|
|
# ---------------------------------------------------------------- deploy
|
|
#
|
|
# Log in as root, which is the identity `sudo docker` in deploy.sh runs as, so
|
|
# the pull there finds this credential. Deliberately not logged out afterwards:
|
|
# a human re-running deploy.sh by hand needs the same pull to work, and the
|
|
# token is read-only.
|
|
printf '%s' "$REGISTRY_TOKEN" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin > /dev/null \
|
|
|| die "docker login to $REGISTRY failed"
|
|
|
|
git fetch --tags --force --prune origin
|
|
# Detached, at the exact tag. The image, the compose file and the migration set
|
|
# must all come from one commit; a tree left on main while running a tagged
|
|
# image is how a migration gets applied that the image knows nothing about.
|
|
git checkout --detach --force "refs/tags/$LATEST_TAG" \
|
|
|| die "cannot check out refs/tags/$LATEST_TAG"
|
|
|
|
mkdir -p "$STATE_DIR"
|
|
if PIG_IMAGE="$IMAGE_NAME:$LATEST_TAG" bash scripts/deploy.sh; then
|
|
rm -f "$FAILED_MARKER"
|
|
printf '%s %s\n' "$LATEST_TAG" "$REMOTE_DIGEST" > "$STATE_DIR/current-release"
|
|
log "deployed $LATEST_TAG"
|
|
else
|
|
STATUS=$?
|
|
printf '%s' "$REMOTE_DIGEST" > "$FAILED_MARKER"
|
|
log "deploy of $LATEST_TAG FAILED (exit $STATUS)."
|
|
# What is serving right now depends on WHICH gate failed, and this line is
|
|
# what the on-call reads at 04:00. It used to claim a rollback unconditionally
|
|
# — including in the one case where the broken release is still up, which is
|
|
# precisely the case that needs hands. deploy.sh exits 3 for that.
|
|
if [ "$STATUS" -eq 3 ]; then
|
|
log "THE FAILING RELEASE IS STILL LIVE — deploy.sh did not roll back (it"
|
|
log "either judged the fault external to the release, had no previous image,"
|
|
log "or the restored image did not come up). Check the site NOW."
|
|
log "Two exit-3 cases leave the CRM serving normally, both about Piggy:"
|
|
log " - it is enabled but did not come up, usually a missing or rejected"
|
|
log " PRIME_API_KEY (PIGGY_INFERENCE_API_KEY is the accepted alias);"
|
|
log " - PIGGY_AGENT_DIR points inside /app, which would let the agent read"
|
|
log " the checkout — refused rather than served."
|
|
log "Both are .env or compose faults, not bad images: a new tag will not fix"
|
|
log "them, and the failed-release marker is correctly telling you so. The log"
|
|
log "above says which it was."
|
|
else
|
|
log "deploy.sh rolls the app back on a failed health, auth or public-marker"
|
|
log "gate, so the previous release should still be serving — verify that first."
|
|
fi
|
|
exit "$STATUS"
|
|
fi
|