Files
compute/evals/smoke.eval.yaml
T
Karti Tripathi a4490ec80e
ci / rust (push) Successful in 2m26s
Lumbridge Compute
Governed compute for unified-memory AI hardware — machines where CPU and GPU
share one pool and there is no separate VRAM allocation to bounce off.
Over-commit that pool and the box thrashes and wedges, SSH and ping included,
before the OOM killer gets a turn.

Compute does not run inference. It supervises the servers that do: admission
control against both a declared budget and what the machine actually has free,
a 1 Hz watchdog that stops the newest model before thrash, Scenes activated as
one transactional unit with rollback, process ownership bound to
(boot_id, pid, start_time_ticks, pgid) so a reused PID can never be
group-killed, a protocol-transparent gateway, an MCP server, and a read-only
HTTP API for dashboards.

Registry footprints in this release are measured on a live node rather than
estimated.

One binary, six direct dependencies. Apache-2.0.

Generated by scripts/publish-compute.sh, which refuses to publish a tree it
cannot prove clean.
2026-08-03 22:23:56 -07:00

34 lines
955 B
YAML

apiVersion: lumbridge/v1
kind: EvalSuite
metadata:
name: smoke
version: 1
description: "Fast correctness and serving-health gate for every new model."
tags: [smoke, ci]
defaults:
max_tokens: 96
temperature: 0.0
repeat: 1
system: "Follow the requested output format exactly. Do not explain unless asked."
cases:
- id: exact-instruction
category: instruction
prompt: "Reply with exactly: lumbridge ready"
assertions:
- type: exact
value: "lumbridge ready"
- id: arithmetic
category: reasoning
prompt: "A box has 121 GB. The OS reserves 21 GB and models use 75 GB. Reply with only the remaining number."
assertions:
- type: exact
value: "25"
- id: concise-voice
category: voice
prompt: "In at most twelve words, say that risk limits are operating normally. No markdown."
assertions:
- type: max_words
value: 12
- type: not_contains
value: "**"