Lumbridge Compute: telemetry and four-lane operations

scenes/qwen38.scene.yaml

@@ -0,0 +1,82 @@
+apiVersion: lumbridge/v1
+kind: Scene
+metadata:
+  name: qwen38
+  version: 2
+  description: "Qwen3.8-27B on SGLang as a seeing brain, plus ears and both narrators."
+  author: karti
+  tags: [brain, vision, voice, long-context, sglang]
+
+# Created 2026-08-17. A copy of `stock` with two changes:
+#
+#   1. brain-nemotron-128k (Lightning 30B-A3B, vLLM)  ->  brain-qwen38
+#      (Qwen3.8-27B NVFP4+MTP, SGLang EAGLE, in Docker)
+#   2. `eye` (Cosmos 3 Edge, 14 GB) is DROPPED — the new brain is a native VLM
+#      and SGLang serves its vision tower in-process, so the separate eye is
+#      redundant. This also kills the enable_thinking:false hack the eye needed
+#      because vLLM 0.27.1 has no cosmos3_edge reasoning parser.
+#
+#   brain-qwen38  56 GB  Qwen3.8-27B @ 262k, sees images and video   :8001
+#   voice-vox     20 GB  VoxCPM 2, quality narrator, capped 3600 chars :8096
+#   ears          10 GB  streaming ASR, self-limiting at 6m39s of audio :8006
+#   voice          5 GB  Chatterbox TURBO, request-path narrator       :8095
+#                 -----
+#                 91 GB committed +8 GB Governor safety margin = 99 of 100.
+#
+# ⚠️ ORDER IS `listed`, NOT footprint-asc, AND THE BRAIN IS FIRST. SGLang_s
+# --mem-fraction-static is a fraction of TOTAL memory and is reserved as a
+# static pool at boot — if ears/voice/vox are already resident it computes its
+# pool against a box that looks full and dies with a mamba-state-cache error.
+# This is the same failure mode the primary scene documents for `brain`.
+#
+# ⚠️ THE SPEED TRADE vs `stock`. Lightning is a 3B-active MoE and decodes ~46
+# tok/s class on this box; Qwen3.8-27B is DENSE and decodes ~25.6 tok/s single
+# stream. You are buying a large quality jump (Terminal-Bench 2.1 63.4 -> 73.0,
+# OSWorld-Verified 63.9 -> 84.3 over Qwen3.6-27B) with roughly half the
+# single-stream token rate. Under concurrency the gap closes and reverses:
+# SGLang EAGLE measured 123.90 tok/s at c8.
+#
+# ⚠️ THINKING IS OFF BY DEFAULT (added v2, 2026-08-21). Qwen3.8 thinks by
+# default and SGLang's auto-detect confirms it
+# (reasoning_config=ReasoningToggleConfig(toggle_param='enable_thinking',
+# default_enabled=True)). That silently broke the assistant stack on first activation:
+# V9's normalizeSpeech (apps/server/src/lib/llm.ts:77) sends max_tokens: 80, the
+# reasoning block consumed all 80, finish_reason came back "length" and content
+# was EMPTY — so news and squawk narration produced nothing, with no error.
+# The registry entry now passes --default-chat-template-kwargs
+# '{"enable_thinking": false}', matching what every brain-nemotron* entry
+# already did. Per-request chat_template_kwargs still overrides, so an agent
+# that wants thinking just asks for it.
+#
+# CANARIES after any change to this scene (all three must pass):
+#   1. 19 x 23 -> 437          (417 means FP8 KV regressed)
+#   2. max_tokens:80 request -> NON-EMPTY content   (guards the above)
+#   3. chat_template_kwargs {"enable_thinking":true} -> reasoning_content present
+#
+# ⚠️ ACTIVATION IS MEMORY-RACY. `docker stop brain-qwen38` followed immediately
+# by `scene activate` fails: --mem-fraction-static is computed against TOTAL
+# memory but the stopped container's pages are not released yet, so the static
+# pool lands short and the container dies at boot with no useful error. The
+# activate process then hangs on "waiting for exact health" holding the
+# transition lock, and the next activate returns "another Scene transition is
+# already running". Wait for `free -g` to show the memory back (~90 GB avail)
+# before re-activating. Boot then takes ~4 min (FlashInfer autotune + CUDA graph).
+#
+# VERIFIED 2026-08-21 on this scene: all three canaries pass; vision reads a
+# generated test image exactly (shapes, colours and the literal string);
+# tool calling picks the right tool with a correct command under a 21-tool
+# payload; the larger local model answered a synthetic container-count prompt correctly with
+# the command cited — the same question the previous Nemotron brain got wrong
+# five times running.
+#
+# Restore the previous default with `lumbridge-compute scene activate stock`.
+models:
+  - brain-qwen38
+  - ears
+  - voice-vox
+  - voice
+
+budget_gb: 100
+activation:
+  order: listed        # brain FIRST — it reserves a static pool, see above
+  wait_healthy: true