# Lumbridge Compute
**The AI compute layer for Lumbridge clusters. Safe models, scenes, and evals on your own hardware.**
*Run many models on one box — safely — on NVIDIA DGX Spark (GB10), DGX Station (GB300), and RTX.*
---
Lumbridge Compute is the tool you install the minute you open your DGX Spark. Boxes like the
DGX Spark (GB10), DGX Station (GB300), and RTX workstations share **one memory pool**
between CPU and GPU with little swap. On that hardware, over-committing memory doesn't
fail gracefully — the whole machine thrashes and wedges (SSH and ping included) before
the OOM killer acts. Lumbridge Compute makes that impossible, and turns the box into something you
can load with different workloads on a schedule.
> **exo runs one model across many boxes. Lumbridge Compute runs many models on one box — safely.**
> They're orthogonal; you can run Lumbridge Compute on each node of an exo cluster.
## Three layers
| Layer | Analogous to | Job |
|---|---|---|
| **Governor** | kernel / memory cgroup + OOM policy | Nothing starts unless it fits a hard budget; a watchdog kills the newest model *before* the box wedges. |
| **Scenes** | systemd targets | Named, shareable, activatable bundles of models (`studio`, `darkroom`). |
| **lumbridge-compute** (the shell) | the CLI you install | Onboard a fresh box, manage models, activate/schedule scenes. |
## The Governor (why Lumbridge Compute exists)
Unified memory means no separate VRAM pool to bounce off. vLLM, diffusers, and friends
will happily reserve past 100% of the shared pool, and the box wedges. The Governor
prevents this with two mechanisms:
1. **Admission control** — a model starts only if its declared footprint fits the budget
given what's already running, plus a safety margin.
2. **Watchdog** — a 1 Hz thread on `MemAvailable`; if it dips below a critical floor, it
kills the most-recently-started model before thrash. A backstop for a wrong estimate.
## Scenes
A **scene** is a named set of models brought up together — the unit you activate, publish,
and schedule. A single unified-memory box can't hold every model at once, so scenes let
you **time-multiplex** it: run a live voice assistant by day, then switch to an overnight
image farm at 3am. One box, the utilization of several.
Scenes reference **model ids**, never weight paths — so requantizing or upgrading a model
never breaks a published scene. See [`docs/scene-spec.md`](docs/scene-spec.md), and
[`docs/positioning.md`](docs/positioning.md) for how Lumbridge Compute relates to Ollama / llama-swap / exo.
## Evals
Lumbridge Compute ships a native, model-server-agnostic evaluation runner. Versioned YAML suites
measure streamed TTFT, client-observed prefill throughput, decode throughput, and
deterministic capability assertions against any OpenAI-compatible endpoint.
```bash
lumbridge-compute eval ls
lumbridge-compute eval run smoke
lumbridge-compute eval run performance --repeat 5
lumbridge-compute eval run finance-core
```
Every run writes a portable JSON artifact for regression tracking and future eval
registries. See [`docs/evals.md`](docs/evals.md).
## Install
```bash
# from source (single static binary, no runtime deps)
cargo install --path .
```
## Usage
```bash
lumbridge-compute status # Governor: memory, budget, running set, headroom
lumbridge-compute model ls # registry: footprints + live state + which port
lumbridge-compute scene ls # scenes with total footprint
lumbridge-compute scene show darkroom # models + footprints + admission verdict
lumbridge-compute scene adopt voice-qwen # one-time identity capture for a legacy live node
lumbridge-compute scene resume # desired Scene, then last-known-good fallback
lumbridge-compute gateway # stable streaming endpoint -> local model server
lumbridge-compute agent # resident resume + gateway + memory supervisor
# planned:
lumbridge-compute model pull # footprint-aware; warns if no scene can hold it
lumbridge-compute scene schedule darkroom 03:00 04:00 # time-multiplex the box
```
By default Lumbridge Compute reads `registry/` and `scenes/` from the current directory (override with
`--root `, `$LUMBRIDGE_COMPUTE_ROOT`, or the compatibility variable `$KUDA_ROOT`).
## Status
The Governor, identity-bound process ownership, transactional Scene switching,
persistent desired state, last-known-good recovery, resident agent, streaming gateway,
eval runner, and watchdog work today. Next: the model artifact manager (`pull`), fleet API,
and scheduler. See [node operations](docs/operations.md) and the stable
[scene spec](docs/scene-spec.md).
## Compatibility
Lumbridge Compute grew out of an earlier prototype named Kuda. The rename is complete: the
executable is `lumbridge-compute`, manifests declare `apiVersion: lumbridge/v1`, and the root
is `$LUMBRIDGE_COMPUTE_ROOT`.
Two compatibility surfaces are kept deliberately, so a node that predates the rename keeps
running without a flag day:
- `$KUDA_ROOT` is still read if `$LUMBRIDGE_COMPUTE_ROOT` is unset.
- `.kuda/` remains the on-disk state directory name, and `apiVersion: kuda/v1` is still
accepted on eval suites.
Both are inert aliases — nothing new should use them.
Lumbridge Compute runs on NVIDIA hardware but is independent and is not affiliated with or endorsed by NVIDIA.
## License
Apache-2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).