This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: darkroom
|
||||
version: 1
|
||||
description: "Overnight image farm — drops the brain to make room for FLUX.2-klein-9B."
|
||||
author: karti
|
||||
tags: [image, overnight, unattended]
|
||||
|
||||
# Drops `brain` so `image` (66 GB, on-hardware measured) fits. ears+voice+music+
|
||||
# image commits ~99 GB — tight against the 8 GB admission margin, so this scene
|
||||
# needs the full 108 GB ceiling (matching studio/voice-laguna), not the old 100.
|
||||
models:
|
||||
- ears
|
||||
- voice
|
||||
- music
|
||||
- image
|
||||
|
||||
budget_gb: 108
|
||||
activation:
|
||||
order: footprint-asc
|
||||
wait_healthy: true
|
||||
|
||||
# Scheduled scenes (planned): hand the box to darkroom overnight, back at 08:00.
|
||||
# schedule:
|
||||
# - activate: "03:00"
|
||||
# - handoff: "04:00" -> studio
|
||||
@@ -0,0 +1,45 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: deepseek
|
||||
version: 1
|
||||
description: "DeepSeek V4 Flash 0731 (3-bit EXL3) plus ears — a whole-box brain swap; voice, embed and music do not fit beside it."
|
||||
author: karti
|
||||
tags: [assistant, brain, experimental, whole-box, deepseek]
|
||||
|
||||
# PARKED 2026-08-06 — registered but never activated on the reference node. The intent is to
|
||||
# run this on the SECOND Spark full-time, where it gets the whole box, which is the
|
||||
# only shape that suits it. On the reference node it is strictly a downgrade: measured the same
|
||||
# day, `brain` does 62.5-64.4 tok/s against this model's published 34.30 min /
|
||||
# 38.12 median, and admitting it costs voice, embed and music. Do not activate here
|
||||
# without a deliberate reason; see the disk note in the registry entry first.
|
||||
#
|
||||
# This is a darkroom-shaped Scene: it is defined by what it displaces. `brain-ds4`
|
||||
# loads 95.39 GiB of EXL3 weights and is declared at 101 GB, so on a 121.6 GB box
|
||||
# it is the tenant and everything else is a lodger.
|
||||
#
|
||||
# Why only `ears` — this is admission control arithmetic, not taste. SAFETY_MARGIN_GB
|
||||
# is 8.0 and admission requires `committed + add + 8 <= budget`:
|
||||
#
|
||||
# brain-ds4 0 + 101 + 8 = 109 <= 118 admitted (available after the stop pass about one hundred fifteen)
|
||||
# ears 101 + 5 + 8 = 114 <= 118 admitted (available ~20, needs 13)
|
||||
# voice 106 + 8 + 8 = 122 > 118 REFUSED (available ~15, needs 16)
|
||||
#
|
||||
# So voice cannot be talked into this Scene by raising budget_gb either — the
|
||||
# physical check fails a hair before the declared one. embed (2) would squeak in
|
||||
# ahead of voice; it is left out deliberately so the /voice stack fails loudly at
|
||||
# activation rather than half-working. music (~24) is not close.
|
||||
#
|
||||
# ORDER MATTERS, same reason as the primary Scene but more so: brain-ds4 must come up
|
||||
# first and alone. Its first start also downloads ~99.5 GiB and coalesces the
|
||||
# rank-sliced EXL3 archive to TP1, so budget on the order of an hour before the
|
||||
# health check goes green, and `wait_healthy` will hold ears behind it the whole
|
||||
# time. Subsequent starts skip straight to the weight load.
|
||||
models:
|
||||
- brain-ds4
|
||||
- ears
|
||||
|
||||
budget_gb: 118
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,35 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: music
|
||||
version: 2
|
||||
description: "Music studio — MiniMax Music 3, with ears, voice, and the Lightning brain."
|
||||
author: karti
|
||||
tags: [music, creative, assistant]
|
||||
|
||||
# v2, 2026-08-13: ACE-Step XL 4B is gone; MiniMax Music 3 takes the `music`
|
||||
# slot on :8010. Same port, same request fields, so :8010 callers are unchanged.
|
||||
#
|
||||
# brain-nemotron joins the scene, which is what forces `order: listed` below.
|
||||
# The old v1 used footprint-asc, and that is actively unsafe with a vLLM MoE in
|
||||
# the set: it would start the 38GB brain LAST, into a box already holding
|
||||
# ears+voice+music. The brain-nemotron entry is explicit that whatever holds
|
||||
# :8001 comes up first and alone, because its KV-profiling spike is not bounded
|
||||
# by gpu-memory-utilization. Declaring that footprint from an estimate once
|
||||
# already ran 11GB over and let the kernel OOM-kill `ears` on the next
|
||||
# transition. So: brain first, on the empty machine, then the small stuff.
|
||||
#
|
||||
# ears 5 + voice 8 + music ~24 + brain-nemotron 38 = ~75 GB of 121.
|
||||
# NOTE: the music figure is an ESTIMATE pending a boot-and-read-RSS measurement,
|
||||
# the same discipline brain-nemotron's comment demands. Do not trust it until
|
||||
# registry/models.yaml carries a measured number.
|
||||
models:
|
||||
- brain-nemotron
|
||||
- ears
|
||||
- voice
|
||||
- music
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,29 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: nemotron-seqs16
|
||||
version: 1
|
||||
description: "stock, with the brain's max-num-seqs raised from 4 to 16. Measurement only."
|
||||
author: karti
|
||||
tags: [brain, experiment, publish]
|
||||
|
||||
# Created 2026-08-19 for /publish episode 5. This is `stock` with exactly one
|
||||
# substitution: brain-nemotron-128k -> brain-nemotron-128k-seqs16. Same weights,
|
||||
# same 128k window, same util 0.32, same speculative config, same everything
|
||||
# else in the scene. See the registry entry for why the cap is suspect.
|
||||
#
|
||||
# ⚠️ THIS IS NOT A DEFAULT. `stock` is the default scene. Reactivate it with
|
||||
# lumbridge-compute scene activate stock
|
||||
# when the sweep is done. Leaving this scene active leaves an experimental,
|
||||
# un-remeasured footprint carrying the downstream agents.
|
||||
models:
|
||||
- brain-nemotron-128k-seqs16
|
||||
- ears
|
||||
- voice-vox
|
||||
- eye
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: footprint-asc
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,36 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: nemotron
|
||||
version: 1
|
||||
description: "The assistant stack with NVIDIA Nemotron 3.5 Lightning 30B-A3B in place of the Qwen MoE brain — the A/B candidate for replacing `brain`."
|
||||
author: karti
|
||||
tags: [assistant, voice, "downstream", candidate, nemotron]
|
||||
|
||||
# Deliberately the assistant stack with ONE substitution. `brain-nemotron` takes :8001
|
||||
# in place of `brain`, so the two are mutually exclusive by construction — and
|
||||
# that is also what the hardware wants: 68GB is already spoken for with brain
|
||||
# up, and two MoE engines would each spike during KV profiling.
|
||||
#
|
||||
# Keeping ears/voice/embed/ocr identical is the point. The only variable
|
||||
# between the primary scene and this scene is which brain answers :8001, so an eval run
|
||||
# against one is comparable to an eval run against the other.
|
||||
#
|
||||
# ORDER MATTERS, same rule as the primary scene: whatever holds :8001 comes up
|
||||
# first and alone. brain's KV-profiling spike is not bounded by
|
||||
# gpu-memory-utilization and this engine is unproven on the box, so it gets
|
||||
# the empty machine.
|
||||
models:
|
||||
- brain-nemotron
|
||||
- ears
|
||||
- voice
|
||||
- embed
|
||||
- ocr
|
||||
# The eye, added 2026-08-12. Cosmos3-Edge is BF16-only and cannot be
|
||||
# quantized, so it costs a real 12GB that the other slots do not.
|
||||
- eye
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,25 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-dflash-mem
|
||||
version: 1
|
||||
description: "MEASUREMENT ONLY — DFlash2 with a 75 GB static pool, voice-vox dropped to pay for it."
|
||||
author: karti
|
||||
tags: [experimental, benchmark, dflash]
|
||||
|
||||
# NOT A PRODUCTION SCENE. It exists to answer one question: does DFlash2 beat
|
||||
# EAGLE when it is given something closer to the memory MiaAI-Lab's profile
|
||||
# assumes (0.90 / ~110 GB)? At 0.46 it did not (22.4 vs EAGLE 22.8 tok/s,
|
||||
# acceptance 0.17-0.25 vs 0.46-0.68).
|
||||
#
|
||||
# voice-vox (20 GB) is dropped purely to afford the bigger pool. Do not leave
|
||||
# this scene active — go back to `qwen38` when the measurement is done.
|
||||
models:
|
||||
- brain-qwen38-dflash-mem
|
||||
- ears
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,30 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-dflash-solo
|
||||
version: 2
|
||||
description: "Production: Qwen3.8-27B vision only, with DFlash2 and durable usage telemetry."
|
||||
author: karti
|
||||
tags: [brain, vision, sglang, dflash, llm-only, production]
|
||||
|
||||
# Spark is the dedicated 27B inference box. ASR and TTS run on metal, and
|
||||
# separate small-model canaries are intentionally absent from this Scene.
|
||||
#
|
||||
# Keep the measured 0.46 static pool. A 0.90 pool did not improve C1-C4
|
||||
# throughput and left too little page cache. Decoder choice, not unused KV
|
||||
# capacity, is the speed lever for this workload.
|
||||
#
|
||||
# Live A/B on this Spark, 2026-08-31, identical 384-token long-generation probe:
|
||||
# aggregate tok/s C1 C2 C3 C4
|
||||
# DSpark 0.90 28.39 50.46 69.59 88.21
|
||||
# DFlash2 0.46 31.54 57.09 81.43 85.96 (median of 3)
|
||||
# DFlash2 wins the common C1-C3 regime by 11-17%. C4 adds only 5.6% over C3,
|
||||
# so the future job scheduler should normally retain one latency-reserve lane
|
||||
# and borrow C4 only while queue and TTFT remain healthy.
|
||||
models:
|
||||
- brain-qwen38-dflash
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,76 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-dflash
|
||||
version: 1
|
||||
description: "qwen38 with the brain on DFlash2 speculative decoding instead of EAGLE."
|
||||
author: karti
|
||||
tags: [brain, vision, voice, long-context, sglang, dflash, experimental]
|
||||
|
||||
# ⛔ BLOCKED 2026-08-21 — DO NOT ACTIVATE ON THE CURRENT IMAGE.
|
||||
#
|
||||
# Our pinned image (lmsysorg/sglang@sha256:3c0abdf4…) ADVERTISES DFLASH as a
|
||||
# builtin speculative algorithm — `sglang serve --help` lists
|
||||
# "Builtins: EAGLE, EAGLE3, NEXTN, STANDALONE, NGRAM, DFLASH, DSPARK"
|
||||
# and accepts every flag (--speculative-draft-model-path,
|
||||
# --mamba-radix-cache-strategy). It boots, loads all 4 shards, then dies at the
|
||||
# draft-worker step with:
|
||||
#
|
||||
# File ".../speculative/dflash_worker_v2.py", line 195, in __init__
|
||||
# File ".../speculative/draft_worker_common.py", line 83, in build_draft_tp_worker
|
||||
# ValueError: Cannot find model module. 'DFlash2DraftModel' is not a
|
||||
# registered model in the Transformers library ... and 'AutoModel' is not
|
||||
# present in the model config's 'auto_map'
|
||||
#
|
||||
# i.e. the ALGORITHM is builtin but the DRAFT MODEL ARCHITECTURE is not
|
||||
# registered. That is precisely why MiaAI-Lab's recipe builds a DERIVED image
|
||||
# from its `patch/` directory for DFlash2 while DSpark runs on the stock image.
|
||||
# The flag surface being present is NOT evidence the path works — it boots for
|
||||
# ~3 minutes before failing, and with `--rm` the logs vanish with the container.
|
||||
#
|
||||
# TO UNBLOCK: build MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark's patched image
|
||||
# (./start-dflash.sh builds it), then repin the digest in the registry entry.
|
||||
#
|
||||
# WORTH IT? MiaAI-Lab's measured table for this box class:
|
||||
# probe EAGLE/MTP DSpark DFlash2
|
||||
# code 34.5 51.5 50.9
|
||||
# long essay 24.1 18.3 25.4
|
||||
# short chat 21.0 23.2 66.6
|
||||
# Our EAGLE baseline measured HERE on 2026-08-21: decode 22.8 tok/s mean
|
||||
# (TTFT 223ms), tool-calling 15/15, with live squawk traffic on the box —
|
||||
# consistent with the 21.0 short-chat column. DSpark is NOT worth the swap for
|
||||
# us (23.2 vs 22.8 short chat; its win is coding, which is Claude's job here,
|
||||
# not spark's). DFlash2's 66.6 short-chat number is the only prize worth the
|
||||
# image build — and it is the number most in need of independent confirmation,
|
||||
# since MiaAI-Lab itself footnotes it as "once token-counted correctly".
|
||||
#
|
||||
# The drafter is already downloaded: /srv/models/Qwen3.8-27B-DFlash2
|
||||
# (z-lab/Qwen3.8-27B-DFlash2@50307d4, 3.6 GB).
|
||||
#
|
||||
# A/B TWIN OF `qwen38`. Identical model set and identical brain flags except the
|
||||
# speculative algorithm (EAGLE -> DFLASH) and the drafter mount, so a benchmark
|
||||
# delta is attributable to the algorithm alone.
|
||||
#
|
||||
# Same ordering rule as `qwen38`: brain FIRST. --mem-fraction-static is a
|
||||
# fraction of TOTAL memory and is reserved as a static pool at boot.
|
||||
#
|
||||
# Same activation race too: do not `docker stop` then immediately activate —
|
||||
# wait for `free -g` to show the memory back first.
|
||||
#
|
||||
# Compare with:
|
||||
# lumbridge-compute eval run performance --model brain-qwen38-dflash
|
||||
# python3 /tmp/toolbench.py http://127.0.0.1:8001/v1 brain-qwen38-dflash 3
|
||||
# Baseline to beat (EAGLE, measured 2026-08-21, with live squawk traffic on the
|
||||
# box): decode 22.8 tok/s mean, TTFT 223ms, tool-calling 15/15.
|
||||
#
|
||||
# Revert with `lumbridge-compute scene activate qwen38`.
|
||||
models:
|
||||
- brain-qwen38-dflash
|
||||
- ears
|
||||
- voice-vox
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,25 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-dspark-solo
|
||||
version: 1
|
||||
description: "BENCHMARK ONLY — DSpark at Mia's 0.90 pool, brain alone, nothing else resident."
|
||||
author: karti
|
||||
tags: [benchmark, dspark, solo]
|
||||
|
||||
# ⚠️ NOT A PRODUCTION SCENE. No ears, no voice, no voice-vox — the desk loses
|
||||
# ASR and both narrators while this is active. Answers one question: does this
|
||||
# box reach MiaAI-Lab's 51.5 tok/s code figure when the brain owns all of it?
|
||||
#
|
||||
# budget_gb is raised to 118 deliberately: a 0.90 pool is ~110 GB and cannot be
|
||||
# admitted under the standard 100 GB budget with the 8 GB safety margin.
|
||||
#
|
||||
# Return to production immediately after measuring:
|
||||
# lumbridge-compute scene activate qwen38-mia
|
||||
models:
|
||||
- brain-qwen38-dspark-solo
|
||||
|
||||
budget_gb: 118
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,34 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-dspark
|
||||
version: 1
|
||||
description: "qwen38-mia tuning with the DSpark speculative stack (code/tool-call optimised)."
|
||||
author: karti
|
||||
tags: [brain, vision, voice, sglang, dspark, benchmark]
|
||||
|
||||
# Same model set and same MiaAI-Lab tuning as `qwen38-mia`; only the
|
||||
# speculative stack differs (EAGLE/MTP -> DSpark) plus torch.compile and
|
||||
# continuous-decode-steps 2.
|
||||
#
|
||||
# Benchmark with MiaAI-Lab's own net-decode harness, NOT the prose `performance`
|
||||
# suite: python3 /tmp/ndec_ours.py
|
||||
# qwen38-mia (EAGLE) baseline: code 31.60 / prose 23.97 tok/s
|
||||
# MiaAI-Lab reports for DSpark: code 51.5 / prose 18.3
|
||||
#
|
||||
# DSpark is faster on code/math/tool-calls and SLOWER on free-form prose, so
|
||||
# check both probes before adopting — the squawk/news path is prose.
|
||||
#
|
||||
# Same swap discipline as every other scene: kill stale `scene activate`,
|
||||
# deactivate, wait for free -g >= 110 GB, activate, then activate again to
|
||||
# persist desired.
|
||||
models:
|
||||
- brain-qwen38-dspark
|
||||
- ears
|
||||
- voice-vox
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-mem
|
||||
version: 1
|
||||
description: "MEASUREMENT ONLY — EAGLE at the same 75 GB pool as qwen38-dflash-mem."
|
||||
author: karti
|
||||
tags: [experimental, benchmark, eagle]
|
||||
|
||||
# NOT A PRODUCTION SCENE. Control for qwen38-dflash-mem: same model set, same
|
||||
# 0.62 pool, same quiet box — only the speculative algorithm differs. Do not
|
||||
# leave active; return to `qwen38`.
|
||||
models:
|
||||
- brain-qwen38-mem
|
||||
- ears
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,22 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38-mia
|
||||
version: 1
|
||||
description: "qwen38 with MiaAI-Lab's full EAGLE tuning (X5 cpuset, host net, GDN pool)."
|
||||
author: karti
|
||||
tags: [brain, vision, voice, long-context, sglang, benchmark]
|
||||
|
||||
# Same model set as `qwen38`; the brain adopts MiaAI-Lab's tuning. The point is
|
||||
# to find out how much of our ~22 tok/s vs her reported 34.5 is launch config.
|
||||
# Baseline to beat: EAGLE @ 0.46 decode 22.8 tok/s, toolbench 15/15 @ 1.5 s.
|
||||
models:
|
||||
- brain-qwen38-mia
|
||||
- ears
|
||||
- voice-vox
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,82 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: qwen38
|
||||
version: 2
|
||||
description: "Qwen3.8-27B on SGLang as a seeing brain, plus ears and both narrators."
|
||||
author: karti
|
||||
tags: [brain, vision, voice, long-context, sglang]
|
||||
|
||||
# Created 2026-08-17. A copy of `stock` with two changes:
|
||||
#
|
||||
# 1. brain-nemotron-128k (Lightning 30B-A3B, vLLM) -> brain-qwen38
|
||||
# (Qwen3.8-27B NVFP4+MTP, SGLang EAGLE, in Docker)
|
||||
# 2. `eye` (Cosmos 3 Edge, 14 GB) is DROPPED — the new brain is a native VLM
|
||||
# and SGLang serves its vision tower in-process, so the separate eye is
|
||||
# redundant. This also kills the enable_thinking:false hack the eye needed
|
||||
# because vLLM 0.27.1 has no cosmos3_edge reasoning parser.
|
||||
#
|
||||
# brain-qwen38 56 GB Qwen3.8-27B @ 262k, sees images and video :8001
|
||||
# voice-vox 20 GB VoxCPM 2, quality narrator, capped 3600 chars :8096
|
||||
# ears 10 GB streaming ASR, self-limiting at 6m39s of audio :8006
|
||||
# voice 5 GB Chatterbox TURBO, request-path narrator :8095
|
||||
# -----
|
||||
# 91 GB committed +8 GB Governor safety margin = 99 of 100.
|
||||
#
|
||||
# ⚠️ ORDER IS `listed`, NOT footprint-asc, AND THE BRAIN IS FIRST. SGLang_s
|
||||
# --mem-fraction-static is a fraction of TOTAL memory and is reserved as a
|
||||
# static pool at boot — if ears/voice/vox are already resident it computes its
|
||||
# pool against a box that looks full and dies with a mamba-state-cache error.
|
||||
# This is the same failure mode the primary scene documents for `brain`.
|
||||
#
|
||||
# ⚠️ THE SPEED TRADE vs `stock`. Lightning is a 3B-active MoE and decodes ~46
|
||||
# tok/s class on this box; Qwen3.8-27B is DENSE and decodes ~25.6 tok/s single
|
||||
# stream. You are buying a large quality jump (Terminal-Bench 2.1 63.4 -> 73.0,
|
||||
# OSWorld-Verified 63.9 -> 84.3 over Qwen3.6-27B) with roughly half the
|
||||
# single-stream token rate. Under concurrency the gap closes and reverses:
|
||||
# SGLang EAGLE measured 123.90 tok/s at c8.
|
||||
#
|
||||
# ⚠️ THINKING IS OFF BY DEFAULT (added v2, 2026-08-21). Qwen3.8 thinks by
|
||||
# default and SGLang's auto-detect confirms it
|
||||
# (reasoning_config=ReasoningToggleConfig(toggle_param='enable_thinking',
|
||||
# default_enabled=True)). That silently broke the assistant stack on first activation:
|
||||
# V9's normalizeSpeech (apps/server/src/lib/llm.ts:77) sends max_tokens: 80, the
|
||||
# reasoning block consumed all 80, finish_reason came back "length" and content
|
||||
# was EMPTY — so news and squawk narration produced nothing, with no error.
|
||||
# The registry entry now passes --default-chat-template-kwargs
|
||||
# '{"enable_thinking": false}', matching what every brain-nemotron* entry
|
||||
# already did. Per-request chat_template_kwargs still overrides, so an agent
|
||||
# that wants thinking just asks for it.
|
||||
#
|
||||
# CANARIES after any change to this scene (all three must pass):
|
||||
# 1. 19 x 23 -> 437 (417 means FP8 KV regressed)
|
||||
# 2. max_tokens:80 request -> NON-EMPTY content (guards the above)
|
||||
# 3. chat_template_kwargs {"enable_thinking":true} -> reasoning_content present
|
||||
#
|
||||
# ⚠️ ACTIVATION IS MEMORY-RACY. `docker stop brain-qwen38` followed immediately
|
||||
# by `scene activate` fails: --mem-fraction-static is computed against TOTAL
|
||||
# memory but the stopped container's pages are not released yet, so the static
|
||||
# pool lands short and the container dies at boot with no useful error. The
|
||||
# activate process then hangs on "waiting for exact health" holding the
|
||||
# transition lock, and the next activate returns "another Scene transition is
|
||||
# already running". Wait for `free -g` to show the memory back (~90 GB avail)
|
||||
# before re-activating. Boot then takes ~4 min (FlashInfer autotune + CUDA graph).
|
||||
#
|
||||
# VERIFIED 2026-08-21 on this scene: all three canaries pass; vision reads a
|
||||
# generated test image exactly (shapes, colours and the literal string);
|
||||
# tool calling picks the right tool with a correct command under a 21-tool
|
||||
# payload; the larger local model answered a synthetic container-count prompt correctly with
|
||||
# the command cited — the same question the previous Nemotron brain got wrong
|
||||
# five times running.
|
||||
#
|
||||
# Restore the previous default with `lumbridge-compute scene activate stock`.
|
||||
models:
|
||||
- brain-qwen38
|
||||
- ears
|
||||
- voice-vox
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed # brain FIRST — it reserves a static pool, see above
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,54 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: stock
|
||||
version: 1
|
||||
description: "The default scene: long-context brain, eyes, ears, and both narrators."
|
||||
author: karti
|
||||
tags: [brain, voice, long-context]
|
||||
|
||||
# Created 2026-08-16. THIS IS THE DEFAULT SCENE as of 2026-08-16 — it is what
|
||||
# the box should be running unless someone is deliberately doing something else.
|
||||
# Five models, chosen for capability rather than latency:
|
||||
#
|
||||
# brain-nemotron-128k 41 GB Lightning 30B at 128k (measured 40.1)
|
||||
# ears 10 GB streaming ASR, self-limiting at 6m39s of audio
|
||||
# (measured 9.9)
|
||||
# voice-vox 20 GB VoxCPM 2 at cfg 3.0 / 20 steps, capped at 3600 chars
|
||||
# eye 14 GB Cosmos 3 Edge, still frames only (measured 11.8)
|
||||
# voice 5 GB Chatterbox TURBO on :8095 (measured 3.2 GPU)
|
||||
# ⚠️ but it LEAKS HOST RAM - see the registry
|
||||
# ----
|
||||
# 90 GB committed against a 100 GB budget.
|
||||
#
|
||||
# All footprints above were measured 2026-08-16 with
|
||||
# `nvidia-smi --query-compute-apps`, NOT RSS — RSS does not see GPU memory on
|
||||
# this unified-memory box (the brain reads 8 GB RSS against 40 GB actual).
|
||||
#
|
||||
# TWO NARRATORS, ON PURPOSE. VoxCPM (:8096) is the quality narrator but is
|
||||
# SLOWER THAN REAL TIME: measured RTF 1.54-1.91 warm, 3.07 cold, against the
|
||||
# 1.62 recorded at build. Chatterbox turbo (:8095) is RTF ~0.26. So:
|
||||
# - anything in a request path -> :8095
|
||||
# - anything offline where quality wins -> :8096
|
||||
# Chatterbox is also here because audiobook and downstream voice services
|
||||
# all hardcode :8095; dropping it on 2026-08-16 took all three down until it
|
||||
# was restored here the same day. VoxCPM is NOT a drop-in for them - different
|
||||
# protocol, different voice registry.
|
||||
#
|
||||
# The Chatterbox FULL checkpoint is sunset as of 2026-08-16; `voice` is
|
||||
# turbo-only and the /voice A/B Lab's `full` option now 400s. See the registry.
|
||||
#
|
||||
# 256k -> 128k to make room for the eye: at 57 GB the brain plus the eye needed
|
||||
# 104 GB against a 100 GB budget and the Governor refused the admit.
|
||||
models:
|
||||
- brain-nemotron-128k
|
||||
- ears
|
||||
- voice-vox
|
||||
- eye
|
||||
- voice
|
||||
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: footprint-asc # the small services up first; the 41 GB brain last
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,22 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: studio
|
||||
version: 2
|
||||
description: "Live voice assistant — ears, brain, mouth, and music."
|
||||
author: karti
|
||||
tags: [assistant, voice, always-on]
|
||||
|
||||
# ears 5 + brain 66 + voice 4 + music (XL 4B sft) 24 = 99 GB declared.
|
||||
# Empirically validated on the reference node: all four load with ~20 GB MemAvailable free.
|
||||
# budget bumped to 108 (box is 121 GB) so the Governor admits the measured-safe set.
|
||||
models:
|
||||
- ears
|
||||
- brain
|
||||
- voice
|
||||
- music
|
||||
|
||||
budget_gb: 108
|
||||
activation:
|
||||
order: footprint-asc # small first, so brain's big load spike lands last
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,26 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: tts-eval
|
||||
version: 1
|
||||
description: "Bare box for TTS model evaluation — ears only, for scoring WER."
|
||||
author: karti
|
||||
tags: [eval, tts]
|
||||
|
||||
# 2026-08-15, TEMPORARY. Drops brain-nemotron (38), music (26) and voice (8) so
|
||||
# the candidate TTS models (VoxCPM2 ~8GB, Fish s2-pro ~17GB) have the whole box.
|
||||
# 'ears' stays because transcribing each model's output back to text is how we
|
||||
# measure intelligibility objectively instead of only by listening.
|
||||
#
|
||||
# The candidates run OUTSIDE the registry during the eval, so the governor is
|
||||
# not accounting for them — that is fine on an otherwise-empty box, but do not
|
||||
# leave this scene active with unmanaged models running.
|
||||
#
|
||||
# TO RESTORE PRODUCTION: lumbridge-compute scene activate music
|
||||
models:
|
||||
- ears
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,22 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: voice-gemma
|
||||
version: 1
|
||||
description: "Efficient voice assistant — Nemotron ASR, Gemma 4 31B, and Chatterbox."
|
||||
author: karti
|
||||
tags: [assistant, voice, efficient, multimodal]
|
||||
|
||||
# Gemma's mostly-sliding-window attention keeps its footprint well under brain
|
||||
# and brain-laguna's, leaving real headroom in the budget beyond Lumbridge Compute's 8 GB
|
||||
# admission margin — useful until brain-gemma's footprint is confirmed on
|
||||
# real hardware.
|
||||
models:
|
||||
- ears
|
||||
- brain-gemma
|
||||
- voice
|
||||
|
||||
budget_gb: 80
|
||||
activation:
|
||||
order: footprint-asc
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: voice-laguna
|
||||
version: 1
|
||||
description: "High-capability voice assistant — Nemotron ASR, Laguna S 2.1, and Chatterbox."
|
||||
author: karti
|
||||
tags: [assistant, voice, coding, reasoning]
|
||||
|
||||
# Laguna replaces Qwen and intentionally excludes ACE-Step. The 108 GB scene
|
||||
# budget matches the empirically safe studio ceiling on the reference node while retaining
|
||||
# Lumbridge Compute's separate 8 GB admission margin.
|
||||
models:
|
||||
- ears
|
||||
- brain-laguna
|
||||
- voice
|
||||
|
||||
budget_gb: 108
|
||||
activation:
|
||||
order: footprint-asc
|
||||
wait_healthy: true
|
||||
@@ -0,0 +1,26 @@
|
||||
apiVersion: lumbridge/v1
|
||||
kind: Scene
|
||||
metadata:
|
||||
name: voice-qwen
|
||||
version: 2
|
||||
description: "Superseded by a newer scene on this node. Kept only because it is still the recorded fallback."
|
||||
author: karti
|
||||
tags: [assistant, voice, low-latency, deprecated]
|
||||
|
||||
# DEPRECATED — renamed to "downstream" on 2026-08-02. This file is retained solely
|
||||
# because last_known_good still points at it; delete once the fallback rolls forward.
|
||||
#
|
||||
# Its ordering was made brain-first to match its successor. `brain` now runs MTP, whose
|
||||
# KV-cache profiling spike is not bounded by gpu-memory-utilization; the old
|
||||
# footprint-asc order would start voice+ears first and leave brain short enough
|
||||
# to trip the 3GB watchdog floor. A fallback that cannot come up is worse than
|
||||
# no fallback.
|
||||
models:
|
||||
- brain
|
||||
- ears
|
||||
- voice
|
||||
|
||||
budget_gb: 100
|
||||
activation:
|
||||
order: listed
|
||||
wait_healthy: true
|
||||
Reference in New Issue
Block a user