Lumbridge Compute: telemetry and four-lane operations

scenes/qwen38-dflash.scene.yaml

@@ -0,0 +1,76 @@
+apiVersion: lumbridge/v1
+kind: Scene
+metadata:
+  name: qwen38-dflash
+  version: 1
+  description: "qwen38 with the brain on DFlash2 speculative decoding instead of EAGLE."
+  author: karti
+  tags: [brain, vision, voice, long-context, sglang, dflash, experimental]
+
+# ⛔ BLOCKED 2026-08-21 — DO NOT ACTIVATE ON THE CURRENT IMAGE.
+#
+# Our pinned image (lmsysorg/sglang@sha256:3c0abdf4…) ADVERTISES DFLASH as a
+# builtin speculative algorithm — `sglang serve --help` lists
+#   "Builtins: EAGLE, EAGLE3, NEXTN, STANDALONE, NGRAM, DFLASH, DSPARK"
+# and accepts every flag (--speculative-draft-model-path,
+# --mamba-radix-cache-strategy). It boots, loads all 4 shards, then dies at the
+# draft-worker step with:
+#
+#   File ".../speculative/dflash_worker_v2.py", line 195, in __init__
+#   File ".../speculative/draft_worker_common.py", line 83, in build_draft_tp_worker
+#   ValueError: Cannot find model module. 'DFlash2DraftModel' is not a
+#   registered model in the Transformers library ... and 'AutoModel' is not
+#   present in the model config's 'auto_map'
+#
+# i.e. the ALGORITHM is builtin but the DRAFT MODEL ARCHITECTURE is not
+# registered. That is precisely why MiaAI-Lab's recipe builds a DERIVED image
+# from its `patch/` directory for DFlash2 while DSpark runs on the stock image.
+# The flag surface being present is NOT evidence the path works — it boots for
+# ~3 minutes before failing, and with `--rm` the logs vanish with the container.
+#
+# TO UNBLOCK: build MiaAI-Lab/Qwen3.8-27B-SGLang-DGX-Spark's patched image
+# (./start-dflash.sh builds it), then repin the digest in the registry entry.
+#
+# WORTH IT? MiaAI-Lab's measured table for this box class:
+#   probe              EAGLE/MTP   DSpark   DFlash2
+#   code                    34.5     51.5      50.9
+#   long essay              24.1     18.3      25.4
+#   short chat              21.0     23.2      66.6
+# Our EAGLE baseline measured HERE on 2026-08-21: decode 22.8 tok/s mean
+# (TTFT 223ms), tool-calling 15/15, with live squawk traffic on the box —
+# consistent with the 21.0 short-chat column. DSpark is NOT worth the swap for
+# us (23.2 vs 22.8 short chat; its win is coding, which is Claude's job here,
+# not spark's). DFlash2's 66.6 short-chat number is the only prize worth the
+# image build — and it is the number most in need of independent confirmation,
+# since MiaAI-Lab itself footnotes it as "once token-counted correctly".
+#
+# The drafter is already downloaded: /srv/models/Qwen3.8-27B-DFlash2
+# (z-lab/Qwen3.8-27B-DFlash2@50307d4, 3.6 GB).
+#
+# A/B TWIN OF `qwen38`. Identical model set and identical brain flags except the
+# speculative algorithm (EAGLE -> DFLASH) and the drafter mount, so a benchmark
+# delta is attributable to the algorithm alone.
+#
+# Same ordering rule as `qwen38`: brain FIRST. --mem-fraction-static is a
+# fraction of TOTAL memory and is reserved as a static pool at boot.
+#
+# Same activation race too: do not `docker stop` then immediately activate —
+# wait for `free -g` to show the memory back first.
+#
+# Compare with:
+#   lumbridge-compute eval run performance --model brain-qwen38-dflash
+#   python3 /tmp/toolbench.py http://127.0.0.1:8001/v1 brain-qwen38-dflash 3
+# Baseline to beat (EAGLE, measured 2026-08-21, with live squawk traffic on the
+# box): decode 22.8 tok/s mean, TTFT 223ms, tool-calling 15/15.
+#
+# Revert with `lumbridge-compute scene activate qwen38`.
+models:
+  - brain-qwen38-dflash
+  - ears
+  - voice-vox
+  - voice
+
+budget_gb: 100
+activation:
+  order: listed
+  wait_healthy: true