# Usage telemetry and concurrency history

Lumbridge Compute records operational model usage without recording model input
or output. The runtime remains the source of truth: Compute samples its local
Prometheus endpoint and makes counter resets durable across model and Scene
restarts.

## SGLang launch contract

The managed SGLang recipe enables:

```text
--enable-metrics
--enable-cache-report
--tokenizer-metrics-allowed-custom-labels client agent workload
```

Callers may attach a bounded label dictionary in SGLang's `x-custom-labels`
header, for example:

```text
x-custom-labels: {"client":"cloud-agent","agent":"research-1","workload":"interactive"}
```

Vision callers use `"workload":"vision"`. Compute prefers native encoder
counters when the serving build exports them and otherwise uses this label for
vision call share and vision prompt-token volume. The report includes
`vision_request_source` so this fallback is never ambiguous.

`client`, `agent`, and `workload` must be low-cardinality stable categories.
Never put a request id, conversation id, user id, file name, URL, or job id in a
Prometheus label. Job ids belong in the future Jobs ledger.

## What is stored

The database is `<root>/.compute/usage/telemetry.sqlite3` in WAL mode. It contains:

- time-sampled running and queued requests, configured concurrency, generation
  throughput, and cache hit rate;
- reset-safe request, prompt-token, generation-token, cached-token, and abort
  counters;
- vision request share and vision prompt-token volume, plus multimodal image
  items, encoder tokens, and image-cache hits when the serving build exports
  those native counters;
- the active Scene, model id, and a fingerprint of the launch configuration; and
- the three bounded caller labels above.

It never stores prompts, input token ids, model output, image/audio/video data,
image URLs, arbitrary headers, or sampling parameters. Counter snapshots are
written once per minute. Scheduler gauges are sampled every five seconds by
default, which makes the C0-C4 distribution a time-weighted approximation rather
than a count of request admissions.

## Commands

```bash
lumbridge-compute usage collect
lumbridge-compute usage summary --since 24h
lumbridge-compute usage agents --since 7d
lumbridge-compute usage concurrency --since 30d

# Every report also has machine-readable output.
lumbridge-compute usage summary --since 7d --json
```

The resident agent collects automatically. Use `--no-telemetry` only when the
runtime cannot expose local metrics; failures degrade telemetry and do not stop
the gateway, watchdog, or model supervisor.

The loopback control API exposes the same read-only records:

```text
GET /v1/usage?since=24h
GET /v1/usage/agents?since=7d
GET /v1/usage/concurrency?since=30d
```

## Four-lane scheduling policy

Telemetry is the gate for Jobs scheduling, not a reason to keep all lanes busy
unconditionally. The measured DFlash2 curve on the reference GB10 node is
31.54, 57.09, 81.43, and 85.96 aggregate token/s at C1-C4. C4 adds only 5.6%
over C3.

The initial scheduler policy therefore is:

1. three fair-share background lanes;
2. one latency-reserve lane for interactive work;
3. background borrowing of lane four only while queueing and TTFT remain healthy;
4. weighted round-robin across agents, with job ids in a durable ledger rather
   than metric labels; and
5. one simultaneous vision-prefill job until measured encoder queueing supports
   relaxing the limit.

Compute should own admission and job lifecycle. Bench may ingest aggregate
artifacts, but it should not receive raw operational traffic or request content.