Lumbridge Compute: telemetry and four-lane operations

2026-08-31 16:09:08 -07:00
commit ef9ec1dcd8
59 changed files with 11279 additions and 0 deletions
@@ -0,0 +1,38 @@
+# Lumbridge Compute roadmap
+
+Lumbridge Compute is the resident AI workload control plane for unified-memory machines.
+
+## Core contracts
+
+1. **Models** — local vetted launch recipes with immutable revisions and measured footprints.
+2. **Scenes** — named workload sets activated through admission control.
+3. **Evals** — versioned datasets, prompts, harnesses, assertions, and portable results.
+4. **Artifacts** — resumable weights, adapters, checkpoints, datasets, and result bundles.
+5. **Jobs** — inference, evaluation, download, conversion, and artifact-adoption lifecycles.
+
+Training execution is deliberately outside this control plane. Forge records the handoff to
+Prime-RL and the resulting provenance; Prime Intellect owns rental infrastructure. Compute
+admits and serves an approved artifact after it has been returned, hashed, and registered. It
+does not implement another trainer or become a marketplace client.
+
+## Near term
+
+- Resumable, checksum-verified `lumbridge-compute model pull` with leases and progress state.
+- Eval comparison, regression thresholds, warmup policy, and concurrency/load tests.
+- Extend the shipped server-native Prometheus ingestion and launch-configuration
+  history with TTFT/latency percentile rendering and retention compaction.
+- Scene hooks for preflight, post-activation eval gates, rollback, and schedules.
+- Gateway aliases so clients follow the active scene without configuration changes.
+- A fail-closed artifact adoption command for checkpoints returned by an approved Forge run:
+  verify digest and metadata, measure the local footprint, then promote into the registry.
+- Checkpoint discovery, resumable transfer, retention, and rollback after registry promotion.
+- A four-lane fair-share Jobs scheduler driven by observed queue pressure and
+  latency: three background lanes, one interactive reserve, and safe C4 borrowing.
+
+## Open-source readiness
+
+- Keep public scene/eval manifests command-free; commands remain in the trusted local registry.
+- Add CI across x86_64 and aarch64, unit/integration tests, security policy,
+  contribution guide, code of conduct, changelog, and versioned JSON schemas.
+- Remove private hostnames, voices, paths, and finance datasets from public fixtures;
+  ship generic examples and keep personal overlays outside the repository.