Lumbridge Bench

2026-08-04 00:44:07 -07:00
commit 006feee0f7
65 changed files with 13516 additions and 0 deletions
@@ -0,0 +1,106 @@
+# Deploy
+
+**Migration status: COMPLETE (2026-08-01).** `karti/lumbridge-bench` is the canonical and only
+repository — `karti/bench.karti.ai` is archived in gitea, fully contained in this repo's `main`.
+
+**The standalone site is retired.** Bench is a Lumbridge product, not its own destination, so
+`bench.karti.ai` now serves a permanent redirect to `https://lumbridgecorp.com/bench`. The
+`bench-karti.service` backend is stopped and disabled on web-host.
+
+What that leaves on the box:
+
+- Caddy vhost for `bench.karti.ai` is now `redir https://lumbridgecorp.com/bench permanent`
+  (a pre-change Caddyfile backup is at `/etc/caddy/Caddyfile.bak-bench-<timestamp>`)
+- DNS `bench.karti.ai A 170.9.14.61` and the Let's Encrypt cert stay — the redirect needs both
+- checkout at `/home/ubuntu/workspace/bench.karti.ai` remains, now on `origin` =
+  `karti/lumbridge-bench`; the `bench-karti` unit is stopped and disabled, not removed
+
+**To bring a Bench UI back up**, deploy it as part of Lumbridge rather than reviving the subdomain.
+Use `deploy/lumbridge-bench.service` and a fresh checkout; do not re-enable `bench-karti`.
+
+The eval runner (`kbench`) is unaffected by any of this — it is a CLI, not the site.
+
+## Gotchas hit during the first deploy
+
+- **Caddy binds `10.0.0.2`, not loopback.** Testing a vhost from the box with
+  `curl --resolve host:443:127.0.0.1` returns `000` and looks like an outage.
+  Use `--resolve host:443:10.0.0.2`.
+- **Order matters: DNS before the vhost reload.** Caddy will attempt ACME the
+  moment the vhost loads; if the A record does not exist yet the challenge
+  fails with NXDOMAIN and the retry sits in backoff for minutes. Adding DNS
+  first, or reloading Caddy again after adding it, gets the cert immediately.
+- **The `oci` CLI is not on PATH on build-host** — it lives at `~/bin/oci`. And
+  `oci dns zone list` needs `--compartment-id`, but `oci dns zone get
+  --zone-name-or-id karti.ai` does not, which is the easier way in.
+
+## Pre-existing, unrelated
+
+Caddy's log shows recurring ACME failures for `www.office.karti.ai` and
+`www.og.karti.ai` — NXDOMAIN, no `www` A records exist. Predates this deploy
+and does not affect the apex domains, but it means those two sites retry ACME
+forever. Either add the `www` records or drop them from their vhosts.
+
+| | |
+|---|---|
+| repo | `karti/lumbridge-bench` on gitea, **private** |
+| host | web-host, port **8909** |
+| service | `bench-karti` compatibility service (systemd) |
+| checkout | `/home/ubuntu/workspace/bench.karti.ai` compatibility path |
+| domain | `bench.karti.ai` → web-host, DNS via oci on build-host (profile `cloud2-sanjose`) |
+
+## One-time setup on web-host
+
+```bash
+cd /home/ubuntu/workspace
+git clone https://gitea.example.internal:8444/karti/lumbridge-bench.git bench.karti.ai
+cd bench.karti.ai/site
+cp .env.example .env.local   # loopback Lumbridge control plane
+npm install && npm run build
+
+sudo cp ../deploy/bench-karti.service /etc/systemd/system/
+sudo systemctl daemon-reload
+sudo systemctl enable --now bench-karti
+curl -s localhost:8909/api/health
+```
+
+Then add the caddy/nginx vhost for `bench.karti.ai` → `localhost:8909` and the
+DNS A record, matching the other `*.karti.ai` sites.
+
+## CI secret
+
+The workflow needs one repo secret, `CLOUD2_SSH_KEY`: the private half of a
+deploy key whose public half is in `ubuntu@web-host`'s `authorized_keys`.
+Generate a dedicated key rather than reusing another site's.
+
+```bash
+ssh-keygen -t ed25519 -f bench-karti-ci-deploy -C "bench-karti-ci-deploy" -N ""
+# public half -> web-host authorized_keys; private half -> gitea repo secret
+```
+
+## Suggestion inbox
+
+The public form forwards an inert record to the Lumbridge control plane at
+`LUMBRIDGE_CONTROL_PLANE_URL`; on web-host this is the private loopback service.
+It stores only the Hugging Face URL, a review reason, and optional notes. Contact
+data is not collected. The old Supabase migrations remain as history but are not
+used by the live suggestion path. Public Bench cards continue to come from
+reviewed `results/*.json`.
+
+## Deploying results
+
+Score cards are committed JSON, so publishing a new measurement is a commit:
+
+```bash
+kbench run brain --tasks ifeval --limit 100
+git add results/ && git commit -m "measure: qwen3.6-35b-a3b @ your-node"
+git push
+```
+
+CI rebuilds and the board picks it up. There is no database sync step because
+there is no database behind the board.
+
+## What is NOT wired yet
+
+- **Automatic submission execution.** Intentionally not part of the product. The public form only records a suggestion for owner review; see `DECISIONS.md#d10`.
+- **Auth.** Schema and invite code exist; the signup/login UI does not.
+- **Suggestion workflow UI.** The owner console now lists the review inbox. Accept/reject controls and all run scheduling remain manual.