lumbridge-code/docs/TESTING.md

# Testing strategy

Lumbridge controls shells, repositories, credentials, agents, and long-lived
processes. A test plan built only from UI clicks would be slow, flaky, and blind
to the failures that matter. The product is designed with programmatic testing
surfaces from the start.

## The central seam

The desktop UI and the test driver use the same versioned local IPC commands and
events exposed by the session runtime. UI tests can therefore:

1. start an isolated runtime with a temporary data directory;
2. issue real commands such as create workspace, split pane, launch harness,
   resize PTY, approve tool, cancel turn, and restore session;
3. wait on semantic events rather than sleeps;
4. query a redacted state snapshot; and
5. separately confirm the visible/accessibility representation where needed.

The test API is not a back door. It uses the same validation and state machine as
the UI, binds locally, requires a per-run capability token, and is absent from
release builds unless explicitly enabled.

## Test layers

### Pure and property tests

Layouts, terminal grid transitions, usage windows, forecasts, event reduction,
path rules, and capability negotiation are deterministic functions. Test them
exhaustively without starting a window. Property tests should target split-tree
invariants, event replay, Unicode width, and usage aggregation.

### Contract tests

Every platform adapter implements the same suites for PTYs, process groups,
signals, resize, secret handles, notifications, paths, and atomic replacement.
Every harness adapter implements discovery, launch, resume, cancel, status, and
usage contracts. Unsupported capability is a valid explicit result.

### Fake harnesses

Small Rust fixture binaries behave like coding agents without network access or
subscriptions. Scripts cover partial frames, noisy stderr, permission prompts,
child processes, crashes, hangs, huge output, invalid bytes, and cancellation.
Tests assert both user-visible state and cleanup of the entire process tree.

### ACP replay

Synthetic ACP transcripts are replayed in both directions against the official
Rust SDK. Golden fixtures cover protocol versions and capability combinations.
Fuzzing targets frame decoding and state transitions; stdout must remain protocol
pure and diagnostics must stay on stderr.

### Terminal conformance

Byte-stream fixtures assert screen cells, styles, cursor, alternate screen,
scrollback, selection, hyperlinks, bracketed paste, keyboard modes, and Unicode.
PTY integration tests exercise real shells on each supported OS. Human typing
and agent automation are stressed concurrently to catch input-order corruption.

### Persistence and recovery

The runtime is interrupted after each durable event and external mutation, then
restarted. Tests prove accepted commands are not lost, incomplete work is
reconciled, migrations are forward-only, and corrupt/truncated tails fail safely.

### UI and rendering

Component tests drive semantic actions and accessibility identifiers. Golden
images cover a small set of high-value layouts at fixed fonts, scale, theme, and
GPU/software renderer. Snapshot updates require review; snapshots never replace
state assertions.

### Performance gates

Criterion and end-to-end probes record input-to-present latency, output ingest,
scrollback search, six-pane frame time, idle CPU, memory per pane, startup, and
restore. CI uses generous regression ceilings; dedicated metal/macOS runs retain
the detailed distributions.

### Packaging smoke tests

Fresh macOS, Ubuntu, and Omarchy/Arch environments install, launch, open a shell,
upgrade, roll back, and uninstall. Artifacts are checked for signatures,
checksums, SBOM, license notices, forbidden secrets, and source revision.

## Developer loop

```bash
# Continuous cargo check. Press t for nextest, c for strict Clippy, v for all.
bacon

# Direct fast test run.
cargo xtest

# Exactly what must pass before a commit.
./scripts/ci.sh
```

`cargo-watch` is no longer the project default because upstream archived it in
2025. `scripts/watch.sh` uses Bacon and retains a cargo-watch fallback for people
who already have the final cargo-watch release installed.

## Implemented vertical-slice coverage

- The framework-neutral six-surface model has deterministic tests for focus,
  direct selection, needs-input transitions, command-palette lifecycle,
  identical GPUI/Floem action replay, bounded output, and workload counters.
- `lumbridge-pty` runs synthetic `/bin/sh` tests for raw output, non-zero exits,
  input, resize, one-chunk backpressure, hung-process termination, invalid
  dimensions, and exclusion of provider credentials from the child environment.
- `lumbridge-runtime` tests ordered actor output, serialized input and resize,
  event polling, invalid configuration, and bounded-time cleanup of a hung PTY.
  A cross-crate integration test drives a real synthetic shell through actor
  output, VT parsing, terminal key encoding, and a 41×101 resize.
- `lumbridge-terminal` fixtures cover styled Unicode, cursor/title state,
  alternate screen, bracketed paste, protocol replies, sanitized title and
  blocked OSC 52 behavior, key encoding, application-cursor mode, and resize.
- `lumbridge-core` tests capability-gated split/select/rename/close commands,
  idempotent request IDs, close-tree promotion, and atomic agentic setup plans.
- The GPUI slice tests its key-event adapter. It renders one real actor-owned VT
  session as plain snapshot rows and keeps five surfaces deterministic; styled
  cell/cursor rendering remains a separate gate.
- The current-GPUI probe compile-checks real AccessKit element wiring and real
  platform input-handler installation. Unit tests cover its semantic tree and
  UTF-16/UTF-8 composed-text mutations. OS screen readers, IME candidate windows,
  and macOS behavior remain platform tests.

## CI matrix as the project grows

Every change runs format, Clippy, unit, contract, integration, and doctests on
Linux. Pull requests that touch platform/UI code also run macOS. Nightly jobs add
MSRV, coverage, dependency/license audit, fuzz smoke, renderer snapshots, soak,
and performance tracking. Release candidates must pass native packaging smoke on
macOS, Ubuntu, and Arch/Omarchy rather than treating cross-compilation as proof.