Deploy

Migration status: COMPLETE (2026-08-01). karti/lumbridge-bench is the canonical and only repository — karti/bench.karti.ai is archived in gitea, fully contained in this repo's main.

The standalone site is retired. Bench is a Lumbridge product, not its own destination, so bench.karti.ai now serves a permanent redirect to https://lumbridgecorp.com/bench. The bench-karti.service backend is stopped and disabled on web-host.

What that leaves on the box:

Caddy vhost for bench.karti.ai is now redir https://lumbridgecorp.com/bench permanent (a pre-change Caddyfile backup is at /etc/caddy/Caddyfile.bak-bench-<timestamp>)
DNS bench.karti.ai A 170.9.14.61 and the Let's Encrypt cert stay — the redirect needs both
checkout at /home/ubuntu/workspace/bench.karti.ai remains, now on origin = karti/lumbridge-bench; the bench-karti unit is stopped and disabled, not removed

To bring a Bench UI back up, deploy it as part of Lumbridge rather than reviving the subdomain. Use deploy/lumbridge-bench.service and a fresh checkout; do not re-enable bench-karti.

The eval runner (kbench) is unaffected by any of this — it is a CLI, not the site.

Gotchas hit during the first deploy

Caddy binds 10.0.0.2, not loopback. Testing a vhost from the box with curl --resolve host:443:127.0.0.1 returns 000 and looks like an outage. Use --resolve host:443:10.0.0.2.
Order matters: DNS before the vhost reload. Caddy will attempt ACME the moment the vhost loads; if the A record does not exist yet the challenge fails with NXDOMAIN and the retry sits in backoff for minutes. Adding DNS first, or reloading Caddy again after adding it, gets the cert immediately.
The oci CLI is not on PATH on build-host — it lives at ~/bin/oci. And oci dns zone list needs --compartment-id, but oci dns zone get --zone-name-or-id karti.ai does not, which is the easier way in.

Pre-existing, unrelated

Caddy's log shows recurring ACME failures for www.office.karti.ai and www.og.karti.ai — NXDOMAIN, no www A records exist. Predates this deploy and does not affect the apex domains, but it means those two sites retry ACME forever. Either add the www records or drop them from their vhosts.


repo	`karti/lumbridge-bench` on gitea, private
host	web-host, port 8909
service	`bench-karti` compatibility service (systemd)
checkout	`/home/ubuntu/workspace/bench.karti.ai` compatibility path
domain	`bench.karti.ai` → web-host, DNS via oci on build-host (profile `cloud2-sanjose`)

One-time setup on web-host

cd /home/ubuntu/workspace
git clone https://gitea.example.internal:8444/karti/lumbridge-bench.git bench.karti.ai
cd bench.karti.ai/site
cp .env.example .env.local   # loopback Lumbridge control plane
npm install && npm run build

sudo cp ../deploy/bench-karti.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now bench-karti
curl -s localhost:8909/api/health

Then add the caddy/nginx vhost for bench.karti.ai → localhost:8909 and the DNS A record, matching the other *.karti.ai sites.

CI secret

The workflow needs one repo secret, CLOUD2_SSH_KEY: the private half of a deploy key whose public half is in ubuntu@web-host's authorized_keys. Generate a dedicated key rather than reusing another site's.

ssh-keygen -t ed25519 -f bench-karti-ci-deploy -C "bench-karti-ci-deploy" -N ""
# public half -> web-host authorized_keys; private half -> gitea repo secret

Suggestion inbox

The public form forwards an inert record to the Lumbridge control plane at LUMBRIDGE_CONTROL_PLANE_URL; on web-host this is the private loopback service. It stores only the Hugging Face URL, a review reason, and optional notes. Contact data is not collected. The old Supabase migrations remain as history but are not used by the live suggestion path. Public Bench cards continue to come from reviewed results/*.json.

Deploying results

Score cards are committed JSON, so publishing a new measurement is a commit:

kbench run brain --tasks ifeval --limit 100
git add results/ && git commit -m "measure: qwen3.6-35b-a3b @ your-node"
git push

CI rebuilds and the board picks it up. There is no database sync step because there is no database behind the board.

What is NOT wired yet

Automatic submission execution. Intentionally not part of the product. The public form only records a suggestion for owner review; see DECISIONS.md#d10.
Auth. Schema and invite code exist; the signup/login UI does not.
Suggestion workflow UI. The owner console now lists the review inbox. Accept/reject controls and all run scheduling remain manual.

4.7 KiB Raw Blame History