feat(arena): add deterministic headless RL environments

2026-08-19 01:10:30 -07:00
parent 79793faed2
commit 747744fa6c
23 changed files with 2387 additions and 1 deletions
@@ -0,0 +1,106 @@
+# Tera Arena environments
+
+`@lumbridge/tera/arena` is Tera's renderer-independent RL boundary. It imports
+no Three.js scene, canvas, DOM input, network client, or asset code. The four
+shipped environments wrap the same fixed-step controllers and plans used by the
+interactive client:
+
+| Environment | Existing simulator | Goal | Safety terminal |
+|---|---|---|---|
+| `drive-101-v1` | `VehicleController` + California transport pack | complete a short US-101 or I-5 route leg | guardrail contact |
+| `office-nav-v1` | `Plan(FRONTIER_VALLEY)` + `createWalker` | reach a collision-clear office waypoint | repeated collision stall |
+| `crow-nav-v1` | `ActorController` in crow flight mode | reach a 3D waypoint | altitude/horizontal envelope contact |
+| `california-flight-v1` | `AircraftController` | reach a geographic/altitude waypoint | California flight-envelope contact |
+
+## Contract
+
+```ts
+import {
+  Drive101Environment,
+  driveScriptedBaseline,
+} from "@lumbridge/tera/arena";
+
+const env = new Drive101Environment();
+env.reset(115, { split: "train" });
+
+for (;;) {
+  const result = env.step(driveScriptedBaseline());
+  if (result.terminated || result.truncated) break;
+}
+
+const checkpoint = env.snapshot(); // JSON-safe and checksummed
+env.restore(checkpoint);
+
+const trace = env.trace();          // actions, component rewards and per-step checksums
+new Drive101Environment().replay(trace); // throws at the first divergence
+```
+
+Every environment implements `tera.arena/v1`:
+
+- `reset(seed, scenario)` accepts an exact public scenario id or
+  `{ split: "train" | "dev", id? }`. Omitting `id` selects a scenario from the
+  requested split deterministically from the uint32-normalized seed.
+- `step(action)` returns an observation, total reward, named reward components,
+  Gym-style `terminated`/`truncated`, and `info`.
+- `info` pins the semantic environment manifest, materialized scenario, simulator
+  sources, environment sources, seed, step and state checksum.
+- `snapshot`/`restore` round-trip controller state exactly. Snapshots from another
+  version, scenario, or modified payload are rejected.
+- `trace` is a canonical, checksummed action/reward/state envelope. `replay`
+  verifies the initial state, every transition, the cumulative reward and final
+  state. The checksum is FNV-1a-64 over canonical UTF-8 JSON; source pins are
+  SHA-256 values checked from the repository by `npm run arena:source-hashes`.
+
+Stepping a completed episode is an error. A task outcome sets `terminated`;
+only the manifest's step cap sets `truncated` with reason `max-steps`.
+
+## Scenarios and evaluation boundary
+
+Train and dev ids are public, explicit, and disjoint in each manifest. Seeded
+jitter is applied only when a scenario is materialized and is included in its
+hash. This package intentionally contains no “private” split: a private eval
+published in the client package is not private. Evaluation operators should
+hold out their scenario definitions and instantiate the same contract in their
+own package or service.
+
+`ARENA_MANIFESTS` is the machine-readable catalogue. Each manifest declares its
+action and observation fields, component reward meanings, public split ids,
+fixed step, maximum steps, safety terminals, and baseline claims.
+
+## Reward and baseline policy
+
+Rewards are counterweighted rather than progress-only. Progress and sparse
+success compete with time, control/energy, lane/altitude/heading, collision and
+safety costs. A policy therefore cannot collect unbounded reward by circling,
+vibrating controls, leaning on a wall, or remaining still.
+
+The tests execute every train and dev scenario at two seeds and require:
+
+1. the documented inaction action to finish below zero without reaching a goal;
+2. the exported scripted baseline to terminate at the goal with positive return;
+3. scripted return to exceed inaction return;
+4. all safety terminal classes to be reachable;
+5. seeded reset, snapshot continuation, trace replay and tamper rejection to be exact.
+
+These are smoke-proof baselines, not optimal policies. They exist to make reward
+regressions and impossible tasks fail in CI before any training budget is spent.
+
+## Adding an environment
+
+Keep the environment under `src/arena/`, wrap an existing renderer-neutral
+controller or plan, expose train/dev scenario ids and a manifest, and add it to
+`ARENA_MANIFESTS`. Do not import a scene adapter to obtain simulation state.
+Update `scripts/check-arena-source-hashes.mjs`, pin the new SHA-256 values, and
+add the same determinism, floor, scripted, safety, snapshot and replay proofs.
+
+Run the complete gate:
+
+```bash
+npm ci
+npm run arena:source-hashes
+npm test
+npm run typecheck
+npm run build
+npm run provenance
+npm run licenses
+```