compute/docs/scene-spec.md

Lumbridge Compute Scene Spec (lumbridge/v1)

This is the public contract. Everything else — the Governor, the CLI, the scheduler — can be refactored freely. This format cannot, once people publish scenes against it. So it is deliberately small.

The core idea: scenes reference ids, not weights

A scene is a manifest listing model ids. A model registry resolves each id to actual weights + a launch command. The registry churns as models evolve (new quants, new backends, bigger context); the scene stays stable.

  scene (stable, shareable)          registry (local, evolves)
  ─────────────────────────          ──────────────────────────────
  models: [ears, brain, voice]  ──▶  brain → ~/models/Qwen3.6-35B-A3B-NVFP4-Fast
                                            → vllm serve, gmu 0.55, flashinfer...

This is the same decoupling the original harness used: it kept the id brain and a stable served_name while the underlying weights swapped dense-27B → 35B-A3B MoE, and downstream agents never noticed. The spec formalizes that as the mechanism that lets scenes "update over time as models evolve" without breaking anyone.

Why this also solves scene-sharing security

A published scene contains only ids and parameters — never shell commands. The launch commands live in your local, vetted registry. So activating a downloaded scene can only ever start models your own registry already trusts. If a scene references an id you don't have, Lumbridge Compute asks you to add it to your registry, showing the launch command for review — an explicit opt-in, not silent remote code execution. Declarative-by-construction; there is no field in which a scene can smuggle a command.

---

Scene manifest

scenes/studio.scene.yaml

apiVersion: lumbridge/v1
kind: Scene
metadata:
  name: studio
  version: 3                      # bump on any change; a published scene is reproducible
  description: "Live voice assistant — ears, brain, mouth, and music."
  author: karti
  tags: [assistant, voice, always-on]

models:                           # stable ids; the registry resolves each
  - ears           # ASR
  - brain          # MoE LLM
  - voice          # TTS
  - music          # ACE-Step

budget_gb: 100                    # optional; overrides the Governor's global budget
activation:
  order: footprint-asc            # small models first so the big load spike lands last
  wait_healthy: true              # block until each model's health check passes

scenes/darkroom.scene.yaml

apiVersion: lumbridge/v1
kind: Scene
metadata:
  name: darkroom
  version: 1
  description: "Overnight image farm — drops the brain to make room for FLUX.2-dev."
  tags: [image, overnight, unattended]

models:
  - ears
  - voice
  - music
  - image          # FLUX.2-dev — only fits because `brain` is not in this scene

activation:
  order: footprint-asc
  wait_healthy: true

Fields

Field	Required	Meaning
apiVersion	✔	lumbridge/v1.
kind	✔	Scene.
metadata.name	✔	Unique scene name; the CLI handle.
metadata.version	✔	Integer, bumped on any change. Reproducibility.
metadata.description	✔	One line, shown in lumbridge-compute scene ls.
metadata.tags	–	For the (future) registry search.
models	✔	Ordered list of model ids resolved via the registry.
budget_gb	–	Per-scene budget override; defaults to the global Governor budget.
activation.order	–	footprint-asc (default) | listed.
activation.wait_healthy	–	Default true. Block until health checks pass.

A scene never contains: weight paths, shell commands, or GPU flags. Those live in the registry. This is load-bearing for both stability and security.

---

Model registry

registry/models.yaml — local to each box, evolves freely. Ids are the stable contract; everything under serve can change.

apiVersion: lumbridge/v1
kind: Registry
version: 1

models:
  brain:
    name: "Qwen3.6 35B-A3B MoE (NVFP4)"
    footprint_gb: 66              # worst-case unified memory once serving (weights + KV + encoder)
    channel: stable               # stable | latest — how aggressively to track new weights
    health: "http://localhost:8001/v1/models"
    serve:
      kind: vllm
      port: 8001
      weights: "~/models/Qwen3.6-35B-A3B-NVFP4-Fast"
      served_name:                # stable aliases so downstream clients survive a weight swap
        - brain
        - "local-moe"
        - "unsloth/Qwen3.6-35B-A3B-NVFP4-Fast"
      args:
        max-model-len: 65536
        kv-cache-dtype: fp8
        gpu-memory-utilization: 0.55
        enforce-eager: true
        moe-backend: flashinfer_b12x      # Unsloth DGX Spark recipe; critical for speed
        limit-mm-per-prompt: '{"image": 0, "video": 0}'
      env:
        CUTE_DSL_ARCH: sm_121a

  image:
    name: "FLUX.2-dev (FP8)"
    footprint_gb: 32
    channel: stable
    health: "http://localhost:8007/health"
    serve:
      kind: diffusers            # not vllm — a separate runtime (ComfyUI/diffusers)
      port: 8007
      weights: "~/models/FLUX.2-dev"
      args: { dtype: fp8 }

  # ears / voice / music elaborated the same way (ASR, Chatterbox, ACE-Step).

channel: how scenes track evolving weights

• stable — pin the exact weights path. Reproducible; you update deliberately.
• latest — the registry may resolve to a newer quant of the same model family (e.g. a fresh NVFP4 build) on activation. Bleeding edge; use for your own box, not for scenes you publish for others.

The scene picks the id; the registry's channel decides how much the weights are allowed to drift underneath it. That's the whole "scenes evolve as models evolve" story, made explicit and controllable.

---

Governor interaction

On activate, Lumbridge Compute computes the diff between the running set and the target scene's models, then:

1. Stops running models not in the scene (frees their footprint first).
2. Starts the scene's models in activation.order, each passing admission control against budget_gb before launch.
3. Waits for health if wait_healthy.

The watchdog runs throughout, unchanged — the safety net if any footprint_gb is wrong. A scene can never talk the Governor into over-committing; admission control is not bypassable by a scene.