Replace the footer's placeholder usage with a real observation ledger
The footer showed invented percentages. It now shows what two harnesses actually report, or says it does not know. lumbridge-core gains an append-only per-profile UsageLedger and a projection that labels every derived value estimated, withholds a burn rate from a single sample, withholds a window fraction with no reported ceiling, withholds an exhaustion estimate that lands after the reset, and reports an expired window as rolled over rather than freezing its last percentage. A missing fact renders as missing, never as zero. (0012) lumbridge-harness is the impure side: processes, clocks, and untrusted wire text in, observations out. Three adapters: - Codex's account/rateLimits/read over the app-server's JSON-RPC stdio. The client cannot express a request outside a two-variant enum and answers every server-to-client request with -32601, so a harness asking Lumbridge for a credential is refused by construction. (0013) - Claude Code's session transcripts, as a byte-offset tail follower that reports nothing until the backlog is read to EOF — a partially-read backlog is indistinguishable from a burst of spend, and the first run against 20 MB reported forty-six billion tokens an hour. The parser models four counters, so the conversations in those files are not representable. (0014) - Claude Code's five-hour and seven-day subscription windows, via a bridge installed as its statusLine command. 0014 had claimed no such surface existed; it does, and the record is corrected in place rather than quietly edited. Lumbridge does not read the OAuth credential to call the account usage endpoint, which is what comparable tools do — AGENTS.md forbids it, and 0015 says so rather than leaving the gap unexplained. Also in here: a capability-check ordering fix in the workspace reducer, where the applied-request replay table was consulted before the capability check and so answered questions the caller had no right to ask; the GPUI spike wired to the live probes with per-harness gauges and provenance chips; and a launcher that matches its own window by PID, because GPUI sets WM_NAME but not _NET_WM_NAME and a title match never succeeded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Metal Agent
co-authored by
Claude Opus 5
Claude Opus 5
parent
7fe84f71e2
commit
ef52aa7ce2
@@ -0,0 +1,836 @@
|
||||
//! The Claude Code usage adapter, over two different surfaces.
|
||||
//!
|
||||
//! **Subscription windows** ([`statusline`]) are the real quota. Claude Code
|
||||
//! 2.1.80 and later pipe a `rate_limits` object carrying `five_hour` and
|
||||
//! `seven_day` — each with `used_percentage` and `resets_at` — to the
|
||||
//! configured `statusLine` command on every turn. Those are rate-limit headers
|
||||
//! the CLI already received on its own API responses, so reading them costs
|
||||
//! nothing and they are `ProviderReported`. Lumbridge does not call the
|
||||
//! account's usage endpoint for them: that would mean reading Claude Code's
|
||||
//! OAuth credential, which AGENTS.md forbids. The status line is pushed to a
|
||||
//! command the user installs, so no credential is ever touched.
|
||||
//!
|
||||
//! **Token consumption** ([`transcript`]) comes from the session transcripts
|
||||
//! Claude Code writes under `~/.claude/projects`. Those record the API's
|
||||
//! `usage` object per assistant turn. A running total is the harness's own
|
||||
//! record rather than a provider's statement of account, so it is
|
||||
//! `HarnessReported`, and it carries no ceiling — spend is not a quota.
|
||||
//!
|
||||
//! Both are tail followers. Each remembers a byte offset and reads only what
|
||||
//! was appended since, which is what makes the token total monotonic — the
|
||||
//! ledger rejects an observation that moves a profile backwards.
|
||||
|
||||
mod statusline;
|
||||
mod transcript;
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fs::File;
|
||||
use std::io::{Read, Seek, SeekFrom};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::mpsc::{self, Receiver, SyncSender, TryRecvError};
|
||||
use std::thread::{self, JoinHandle};
|
||||
use std::time::Duration;
|
||||
|
||||
use lumbridge_core::{AccountProfile, UsageObservation, UsageProvenance, UsageUnit};
|
||||
|
||||
use crate::HarnessError;
|
||||
use crate::claude::statusline::parse_feed_line;
|
||||
pub use crate::claude::statusline::{ClaudeWindowKind, WindowReading};
|
||||
use crate::claude::transcript::MAX_RECORD_BYTES;
|
||||
use crate::clock::MonotonicWallClock;
|
||||
use crate::probe::{ProbeHealth, ProbeOutcome, UsageProbe};
|
||||
|
||||
pub use transcript::TokenTally;
|
||||
|
||||
/// Transcripts are appended to constantly; polling faster than this buys
|
||||
/// nothing but disk churn.
|
||||
const MIN_POLL_INTERVAL: Duration = Duration::from_secs(1);
|
||||
/// How long the worker sleeps between filesystem scans.
|
||||
const WORKER_TICK: Duration = Duration::from_millis(250);
|
||||
/// A bound on how much of the tree one scan will walk.
|
||||
const MAX_DEPTH: usize = 6;
|
||||
const MAX_FILES_PER_SCAN: usize = 4_096;
|
||||
/// A bound on how much any one poll will read, so a large backlog is absorbed
|
||||
/// across several polls instead of stalling one.
|
||||
const MAX_BYTES_PER_POLL: usize = 8 * 1024 * 1024;
|
||||
const EVENT_QUEUE: usize = 64;
|
||||
|
||||
/// Where and how often to read transcripts.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct ClaudeCodeProbeOptions {
|
||||
/// The Claude Code projects root. Defaults to `$CLAUDE_CONFIG_DIR/projects`
|
||||
/// or `~/.claude/projects`.
|
||||
pub projects_root: PathBuf,
|
||||
/// Where the Lumbridge status-line bridge appends rate-limit readings.
|
||||
/// Absent until the user installs the bridge, which is why the windows read
|
||||
/// as unavailable rather than as zero before then.
|
||||
pub rate_limit_feed: PathBuf,
|
||||
pub poll_interval: Duration,
|
||||
}
|
||||
|
||||
impl Default for ClaudeCodeProbeOptions {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
projects_root: default_projects_root(),
|
||||
rate_limit_feed: default_rate_limit_feed(),
|
||||
poll_interval: Duration::from_secs(10),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the status-line bridge writes, under Lumbridge's own data root rather
|
||||
/// than inside the harness's configuration.
|
||||
///
|
||||
/// `LUMBRIDGE_CLAUDE_FEED` overrides it, and must be read here as well as in the
|
||||
/// bridge: the two halves have to resolve the same path or the bridge writes
|
||||
/// somewhere the probe never looks and the windows read as unavailable forever.
|
||||
#[must_use]
|
||||
pub fn default_rate_limit_feed() -> PathBuf {
|
||||
if let Some(configured) = std::env::var_os("LUMBRIDGE_CLAUDE_FEED") {
|
||||
return PathBuf::from(configured);
|
||||
}
|
||||
let root = std::env::var_os("XDG_DATA_HOME").map_or_else(
|
||||
|| {
|
||||
std::env::var_os("HOME")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_default()
|
||||
.join(".local")
|
||||
.join("share")
|
||||
},
|
||||
PathBuf::from,
|
||||
);
|
||||
root.join("lumbridge").join("claude-rate-limits.jsonl")
|
||||
}
|
||||
|
||||
/// The documented default location of Claude Code's session transcripts.
|
||||
#[must_use]
|
||||
pub fn default_projects_root() -> PathBuf {
|
||||
if let Some(configured) = std::env::var_os("CLAUDE_CONFIG_DIR") {
|
||||
return PathBuf::from(configured).join("projects");
|
||||
}
|
||||
std::env::var_os("HOME")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_default()
|
||||
.join(".claude")
|
||||
.join("projects")
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum WorkerEvent {
|
||||
Observation(UsageObservation),
|
||||
Health(ProbeHealth),
|
||||
}
|
||||
|
||||
/// A running Claude Code usage probe.
|
||||
pub struct ClaudeCodeProbe {
|
||||
profiles: Vec<AccountProfile>,
|
||||
events: Option<Receiver<WorkerEvent>>,
|
||||
stop: Arc<AtomicBool>,
|
||||
worker: Option<JoinHandle<()>>,
|
||||
health: ProbeHealth,
|
||||
tokens: TokenTally,
|
||||
}
|
||||
|
||||
impl ClaudeCodeProbe {
|
||||
/// The static, account-free profiles this probe reports under: one per
|
||||
/// documented subscription window, plus the transcript token total.
|
||||
///
|
||||
/// The windows come first because they are the quota — what a user means
|
||||
/// by "how much do I have left" — and the token total is the supporting
|
||||
/// fact about spend.
|
||||
fn profiles_for() -> Result<Vec<AccountProfile>, HarnessError> {
|
||||
let mut profiles = Vec::new();
|
||||
for kind in ClaudeWindowKind::ALL {
|
||||
profiles.push(
|
||||
AccountProfile::new(
|
||||
kind.profile_id(),
|
||||
"Claude Code",
|
||||
"Anthropic",
|
||||
kind.scope(),
|
||||
"subscription",
|
||||
)
|
||||
.map_err(|_| HarnessError::EmptyProgram)?,
|
||||
);
|
||||
}
|
||||
profiles.push(
|
||||
AccountProfile::new(
|
||||
"claude-code-transcripts",
|
||||
"Claude Code",
|
||||
"Anthropic",
|
||||
"session transcripts",
|
||||
"subscription",
|
||||
)
|
||||
.map_err(|_| HarnessError::EmptyProgram)?,
|
||||
);
|
||||
Ok(profiles)
|
||||
}
|
||||
|
||||
/// Starts the probe and its worker.
|
||||
///
|
||||
/// A missing projects directory is not an error: the probe reports zero
|
||||
/// and keeps looking, because Claude Code creates it on first use.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns [`HarnessError::ProbeIntervalTooShort`] below one second.
|
||||
pub fn start(options: ClaudeCodeProbeOptions) -> Result<Self, HarnessError> {
|
||||
if options.poll_interval < MIN_POLL_INTERVAL {
|
||||
return Err(HarnessError::ProbeIntervalTooShort);
|
||||
}
|
||||
let profiles = Self::profiles_for()?;
|
||||
let worker_profiles = profiles.clone();
|
||||
let (event_sender, events) = mpsc::sync_channel(EVENT_QUEUE);
|
||||
let stop = Arc::new(AtomicBool::new(false));
|
||||
let worker_stop = Arc::clone(&stop);
|
||||
let worker = thread::Builder::new()
|
||||
.name("lumbridge-claude-probe".to_owned())
|
||||
.spawn(move || {
|
||||
run_worker(&options, &worker_profiles, &event_sender, &worker_stop);
|
||||
})
|
||||
.map_err(|error| HarnessError::SpawnFailed(error.kind()))?;
|
||||
|
||||
Ok(Self {
|
||||
profiles,
|
||||
events: Some(events),
|
||||
stop,
|
||||
worker: Some(worker),
|
||||
health: ProbeHealth::Starting,
|
||||
tokens: TokenTally::default(),
|
||||
})
|
||||
}
|
||||
|
||||
/// The token split behind the latest observation.
|
||||
///
|
||||
/// The ledger holds only the total; this keeps the breakdown available for
|
||||
/// a detail view without a second pass over the transcripts.
|
||||
#[must_use]
|
||||
pub const fn tokens(&self) -> TokenTally {
|
||||
self.tokens
|
||||
}
|
||||
}
|
||||
|
||||
impl UsageProbe for ClaudeCodeProbe {
|
||||
fn poll(&mut self) -> ProbeOutcome {
|
||||
let Some(events) = self.events.as_ref() else {
|
||||
return ProbeOutcome::idle(self.health);
|
||||
};
|
||||
let mut observations = Vec::new();
|
||||
loop {
|
||||
match events.try_recv() {
|
||||
Ok(WorkerEvent::Observation(observation)) => observations.push(observation),
|
||||
Ok(WorkerEvent::Health(health)) => self.health = health,
|
||||
Err(TryRecvError::Empty) => break,
|
||||
Err(TryRecvError::Disconnected) => {
|
||||
if !self.health.is_faulted() {
|
||||
self.health = ProbeHealth::Stopped;
|
||||
}
|
||||
self.events = None;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
ProbeOutcome::new(observations, self.health)
|
||||
}
|
||||
|
||||
fn profiles(&self) -> &[AccountProfile] {
|
||||
&self.profiles
|
||||
}
|
||||
|
||||
fn shutdown(&mut self) {
|
||||
self.stop.store(true, Ordering::Relaxed);
|
||||
self.events = None;
|
||||
if let Some(worker) = self.worker.take() {
|
||||
let _ = worker.join();
|
||||
}
|
||||
if !self.health.is_faulted() {
|
||||
self.health = ProbeHealth::Stopped;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ClaudeCodeProbe {
|
||||
fn drop(&mut self) {
|
||||
self.shutdown();
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-transcript read position. The offset is what makes the total monotonic.
|
||||
#[derive(Default)]
|
||||
struct FollowState {
|
||||
offsets: BTreeMap<PathBuf, u64>,
|
||||
tokens: TokenTally,
|
||||
/// Read position in the status-line feed, and the newest reading seen.
|
||||
feed_offset: u64,
|
||||
windows: BTreeMap<&'static str, WindowReading>,
|
||||
}
|
||||
|
||||
/// Reads whatever the status-line bridge appended and keeps the newest line.
|
||||
///
|
||||
/// The feed is a state snapshot per line, not a stream of deltas, so only the
|
||||
/// last complete line matters — an older one is superseded, never summed.
|
||||
fn follow_feed(path: &Path, state: &mut FollowState, observed_at_ms: u64) {
|
||||
let Ok(metadata) = std::fs::metadata(path) else {
|
||||
return;
|
||||
};
|
||||
let length = metadata.len();
|
||||
if length < state.feed_offset {
|
||||
state.feed_offset = 0;
|
||||
}
|
||||
if length == state.feed_offset {
|
||||
return;
|
||||
}
|
||||
let take = usize::try_from(length - state.feed_offset)
|
||||
.unwrap_or(usize::MAX)
|
||||
.min(MAX_BYTES_PER_POLL);
|
||||
let Ok(mut file) = File::open(path) else {
|
||||
return;
|
||||
};
|
||||
if file.seek(SeekFrom::Start(state.feed_offset)).is_err() {
|
||||
return;
|
||||
}
|
||||
let mut buffer = vec![0_u8; take];
|
||||
let Ok(read) = file.read(&mut buffer) else {
|
||||
return;
|
||||
};
|
||||
buffer.truncate(read);
|
||||
let chunk = String::from_utf8_lossy(&buffer);
|
||||
|
||||
let mut consumed = 0;
|
||||
let mut newest = None;
|
||||
for line in chunk.split_inclusive('\n') {
|
||||
if !line.ends_with('\n') {
|
||||
break;
|
||||
}
|
||||
consumed += line.len();
|
||||
if line.len() <= MAX_RECORD_BYTES
|
||||
&& let Some(readings) = parse_feed_line(line.trim_end(), observed_at_ms)
|
||||
{
|
||||
newest = Some(readings);
|
||||
}
|
||||
}
|
||||
state.feed_offset += consumed as u64;
|
||||
if let Some(readings) = newest {
|
||||
for (kind, reading) in readings {
|
||||
state.windows.insert(kind.profile_id(), reading);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn run_worker(
|
||||
options: &ClaudeCodeProbeOptions,
|
||||
profiles: &[AccountProfile],
|
||||
events: &SyncSender<WorkerEvent>,
|
||||
stop: &AtomicBool,
|
||||
) {
|
||||
let token_profile = profiles
|
||||
.iter()
|
||||
.find(|profile| profile.id().as_str() == "claude-code-transcripts")
|
||||
.cloned();
|
||||
let mut clock = MonotonicWallClock::start();
|
||||
let mut state = FollowState::default();
|
||||
let poll_interval_ms = u64::try_from(options.poll_interval.as_millis()).unwrap_or(u64::MAX);
|
||||
let mut last_poll_ms = 0;
|
||||
let mut last_emitted = TokenTally::default();
|
||||
let mut last_windows: BTreeMap<&'static str, WindowReading> = BTreeMap::new();
|
||||
let mut primed = false;
|
||||
|
||||
loop {
|
||||
if stop.load(Ordering::Relaxed) {
|
||||
break;
|
||||
}
|
||||
let now_ms = clock.now_ms();
|
||||
// While priming, scan every tick to work through the backlog quickly.
|
||||
// After that, respect the poll interval.
|
||||
if primed && now_ms.saturating_sub(last_poll_ms) < poll_interval_ms {
|
||||
thread::sleep(WORKER_TICK);
|
||||
continue;
|
||||
}
|
||||
last_poll_ms = now_ms;
|
||||
|
||||
let caught_up = scan(&options.projects_root, &mut state);
|
||||
// The quota windows are independent of the transcript backlog: a
|
||||
// status-line reading is a complete snapshot, so it can be reported
|
||||
// immediately rather than waiting for priming.
|
||||
follow_feed(&options.rate_limit_feed, &mut state, now_ms);
|
||||
for kind in ClaudeWindowKind::ALL {
|
||||
let Some(reading) = state.windows.get(kind.profile_id()).copied() else {
|
||||
continue;
|
||||
};
|
||||
if last_windows.get(kind.profile_id()) == Some(&reading) {
|
||||
continue;
|
||||
}
|
||||
last_windows.insert(kind.profile_id(), reading);
|
||||
let Some(profile) = profiles
|
||||
.iter()
|
||||
.find(|profile| profile.id().as_str() == kind.profile_id())
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
let observation = window_observation(profile, reading, now_ms);
|
||||
if matches!(
|
||||
events.try_send(WorkerEvent::Observation(observation)),
|
||||
Err(mpsc::TrySendError::Disconnected(_))
|
||||
) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing is reported until the follower has read every existing
|
||||
// transcript to its end. A partially-read backlog looks exactly like
|
||||
// an enormous burst of spend, and the ledger would derive a burn rate
|
||||
// from it — one early run reported forty-six billion tokens an hour,
|
||||
// which was catch-up, not usage. The first reading must be a true
|
||||
// baseline before any rate can mean anything.
|
||||
if !caught_up {
|
||||
thread::sleep(WORKER_TICK);
|
||||
continue;
|
||||
}
|
||||
if !primed {
|
||||
primed = true;
|
||||
let _ = events.try_send(WorkerEvent::Health(ProbeHealth::Ready));
|
||||
}
|
||||
|
||||
// Only speak when the number changed. An unchanged total re-recorded
|
||||
// every poll would flush the ledger's bounded history of real readings.
|
||||
if state.tokens == last_emitted {
|
||||
continue;
|
||||
}
|
||||
last_emitted = state.tokens;
|
||||
let Some(profile) = token_profile.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
let Ok(observation) = UsageObservation::counted(
|
||||
profile.id().clone(),
|
||||
UsageUnit::Tokens,
|
||||
state.tokens.total(),
|
||||
UsageProvenance::HarnessReported,
|
||||
now_ms,
|
||||
) else {
|
||||
continue;
|
||||
};
|
||||
// No `with_limit` and no `with_window`: the transcript states what was
|
||||
// spent and says nothing about a ceiling or a reset. Attaching either
|
||||
// would be an invention.
|
||||
if matches!(
|
||||
events.try_send(WorkerEvent::Observation(observation)),
|
||||
Err(mpsc::TrySendError::Disconnected(_))
|
||||
) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Turns a window reading into the observation the ledger should hold.
|
||||
///
|
||||
/// A usable reading is `ProviderReported`: the CLI is relaying rate-limit
|
||||
/// headers from its own API responses, not computing a number. An absent
|
||||
/// window is an explicit gap — an account with no plan limits, or a window the
|
||||
/// API has stopped reporting — never a zero.
|
||||
fn window_observation(
|
||||
profile: &AccountProfile,
|
||||
reading: WindowReading,
|
||||
observed_at_ms: u64,
|
||||
) -> UsageObservation {
|
||||
let WindowReading::Usable { permille, window } = reading else {
|
||||
return UsageObservation::unavailable(profile.id().clone(), observed_at_ms);
|
||||
};
|
||||
let Ok(observation) = UsageObservation::counted(
|
||||
profile.id().clone(),
|
||||
UsageUnit::WindowPermille,
|
||||
permille,
|
||||
UsageProvenance::ProviderReported,
|
||||
observed_at_ms,
|
||||
) else {
|
||||
return UsageObservation::unavailable(profile.id().clone(), observed_at_ms);
|
||||
};
|
||||
let observation = observation
|
||||
.with_limit(1_000)
|
||||
.unwrap_or_else(|_| UsageObservation::unavailable(profile.id().clone(), observed_at_ms));
|
||||
let Some(window) = window else {
|
||||
return observation;
|
||||
};
|
||||
observation
|
||||
.clone()
|
||||
.with_window(window)
|
||||
.unwrap_or(observation)
|
||||
}
|
||||
|
||||
/// Walks the projects tree and reads whatever was appended since last time.
|
||||
///
|
||||
/// Returns whether every transcript was read to its end. A `false` means the
|
||||
/// per-poll byte budget ran out with work remaining, and the running total is
|
||||
/// therefore mid-catch-up rather than current.
|
||||
fn scan(root: &Path, state: &mut FollowState) -> bool {
|
||||
let mut transcripts = Vec::new();
|
||||
collect_transcripts(root, 0, &mut transcripts);
|
||||
let mut budget = MAX_BYTES_PER_POLL;
|
||||
let mut caught_up = true;
|
||||
for path in transcripts {
|
||||
if budget == 0 {
|
||||
return false;
|
||||
}
|
||||
if !follow(&path, state, &mut budget) {
|
||||
caught_up = false;
|
||||
}
|
||||
}
|
||||
caught_up
|
||||
}
|
||||
|
||||
fn collect_transcripts(directory: &Path, depth: usize, found: &mut Vec<PathBuf>) {
|
||||
if depth > MAX_DEPTH || found.len() >= MAX_FILES_PER_SCAN {
|
||||
return;
|
||||
}
|
||||
let Ok(entries) = std::fs::read_dir(directory) else {
|
||||
return;
|
||||
};
|
||||
for entry in entries.flatten() {
|
||||
if found.len() >= MAX_FILES_PER_SCAN {
|
||||
return;
|
||||
}
|
||||
let path = entry.path();
|
||||
let Ok(file_type) = entry.file_type() else {
|
||||
continue;
|
||||
};
|
||||
// Deliberately not following symlinks: a link inside the projects tree
|
||||
// could otherwise point this reader at an arbitrary file.
|
||||
if file_type.is_dir() {
|
||||
collect_transcripts(&path, depth + 1, found);
|
||||
} else if file_type.is_file() && path.extension().is_some_and(|ext| ext == "jsonl") {
|
||||
found.push(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads one transcript from its remembered offset.
|
||||
///
|
||||
/// Returns whether this file is now read to its end.
|
||||
fn follow(path: &Path, state: &mut FollowState, budget: &mut usize) -> bool {
|
||||
let offset = state.offsets.get(path).copied().unwrap_or(0);
|
||||
let Ok(metadata) = std::fs::metadata(path) else {
|
||||
return true;
|
||||
};
|
||||
let length = metadata.len();
|
||||
if length < offset {
|
||||
// The file shrank, so it is not the file we were reading. Start over
|
||||
// rather than trusting an offset into different content.
|
||||
state.offsets.insert(path.to_path_buf(), 0);
|
||||
return false;
|
||||
}
|
||||
let pending = usize::try_from(length - offset).unwrap_or(usize::MAX);
|
||||
if pending == 0 {
|
||||
return true;
|
||||
}
|
||||
let take = pending.min(*budget);
|
||||
|
||||
let Ok(mut file) = File::open(path) else {
|
||||
return true;
|
||||
};
|
||||
if file.seek(SeekFrom::Start(offset)).is_err() {
|
||||
return true;
|
||||
}
|
||||
let mut buffer = vec![0_u8; take];
|
||||
let Ok(read) = file.read(&mut buffer) else {
|
||||
return true;
|
||||
};
|
||||
buffer.truncate(read);
|
||||
// A transcript is UTF-8 JSON. A chunk boundary can split a multi-byte
|
||||
// character, so decode lossily and stop at the last complete line — the
|
||||
// offset only advances past whole records either way.
|
||||
let chunk = String::from_utf8_lossy(&buffer);
|
||||
let (tally, consumed) = transcript::consume_chunk(&chunk);
|
||||
state.tokens.add(tally.tokens);
|
||||
state
|
||||
.offsets
|
||||
.insert(path.to_path_buf(), offset + consumed as u64);
|
||||
*budget = budget.saturating_sub(consumed);
|
||||
// Caught up only if we took the whole remainder and parsed all of it. A
|
||||
// trailing partial line counts as caught up: the record is still being
|
||||
// written, and waiting for it is correct.
|
||||
take == pending && pending.saturating_sub(consumed) < MAX_RECORD_BYTES
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
ClaudeCodeProbe, ClaudeCodeProbeOptions, FollowState, TokenTally, default_projects_root,
|
||||
scan,
|
||||
};
|
||||
use crate::HarnessError;
|
||||
use crate::probe::{ProbeHealth, UsageProbe};
|
||||
use lumbridge_core::{UsageProvenance, UsageUnit};
|
||||
use std::io::Write;
|
||||
use std::path::PathBuf;
|
||||
use std::time::Duration;
|
||||
|
||||
const RECORD: &str = r#"{"type":"assistant","message":{"model":"claude-opus-5","content":[{"type":"text","text":"private"}],"usage":{"input_tokens":1,"output_tokens":10,"cache_creation_input_tokens":100,"cache_read_input_tokens":1000}}}"#;
|
||||
|
||||
fn scratch(tag: &str) -> PathBuf {
|
||||
let root = std::env::temp_dir().join(format!(
|
||||
"lumbridge-claude-probe-{}-{tag}",
|
||||
std::process::id()
|
||||
));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
std::fs::create_dir_all(root.join("project-a")).expect("scratch dir");
|
||||
root
|
||||
}
|
||||
|
||||
fn append(path: &PathBuf, times: usize) {
|
||||
let mut file = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(path)
|
||||
.expect("transcript is writable");
|
||||
for _ in 0..times {
|
||||
writeln!(file, "{RECORD}").expect("append");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_missing_projects_root_reports_zero_rather_than_failing() {
|
||||
let mut state = FollowState::default();
|
||||
scan(
|
||||
&PathBuf::from("/nonexistent/lumbridge/claude/root"),
|
||||
&mut state,
|
||||
);
|
||||
assert_eq!(state.tokens, TokenTally::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn appended_records_accumulate_and_never_double_count() {
|
||||
let root = scratch("accumulate");
|
||||
let transcript = root.join("project-a").join("session.jsonl");
|
||||
append(&transcript, 2);
|
||||
|
||||
let mut state = FollowState::default();
|
||||
assert!(scan(&root, &mut state));
|
||||
assert_eq!(state.tokens.total(), 2 * 1_111);
|
||||
|
||||
// A second scan with no new bytes must add nothing.
|
||||
assert!(scan(&root, &mut state));
|
||||
assert_eq!(state.tokens.total(), 2 * 1_111);
|
||||
|
||||
// Only the newly appended record is counted.
|
||||
append(&transcript, 1);
|
||||
assert!(scan(&root, &mut state));
|
||||
assert_eq!(state.tokens.total(), 3 * 1_111);
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nested_subagent_transcripts_are_counted_too() {
|
||||
let root = scratch("nested");
|
||||
let nested = root.join("project-a").join("session").join("subagents");
|
||||
std::fs::create_dir_all(&nested).expect("nested dirs");
|
||||
append(&root.join("project-a").join("session.jsonl"), 1);
|
||||
append(&nested.join("agent-1.jsonl"), 1);
|
||||
|
||||
let mut state = FollowState::default();
|
||||
assert!(scan(&root, &mut state));
|
||||
assert_eq!(
|
||||
state.tokens.total(),
|
||||
2 * 1_111,
|
||||
"a subagent's tokens are the user's tokens"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_truncated_transcript_restarts_rather_than_reading_a_stale_offset() {
|
||||
let root = scratch("truncated");
|
||||
let transcript = root.join("project-a").join("session.jsonl");
|
||||
append(&transcript, 3);
|
||||
let mut state = FollowState::default();
|
||||
assert!(scan(&root, &mut state));
|
||||
let before = state.tokens.total();
|
||||
assert_eq!(before, 3 * 1_111);
|
||||
|
||||
std::fs::write(&transcript, "").expect("truncate");
|
||||
// The shrink is reported as not-caught-up on purpose: the offset was
|
||||
// just reset, so the probe must re-scan before it trusts the total
|
||||
// again rather than reporting mid-reset.
|
||||
assert!(
|
||||
!scan(&root, &mut state),
|
||||
"a shrunk file asks to be read again"
|
||||
);
|
||||
assert!(scan(&root, &mut state), "and is caught up on the next pass");
|
||||
assert_eq!(
|
||||
state.tokens.total(),
|
||||
before,
|
||||
"a shrunk file resets its offset without inventing or losing tokens"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_transcript_files_are_ignored() {
|
||||
let root = scratch("ignored");
|
||||
std::fs::write(root.join("project-a").join("notes.md"), RECORD).expect("write");
|
||||
std::fs::write(root.join("project-a").join("history.json"), RECORD).expect("write");
|
||||
let mut state = FollowState::default();
|
||||
assert!(scan(&root, &mut state));
|
||||
assert_eq!(state.tokens, TokenTally::default());
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_too_frequent_poll_interval_is_refused() {
|
||||
let options = ClaudeCodeProbeOptions {
|
||||
poll_interval: Duration::from_millis(10),
|
||||
..ClaudeCodeProbeOptions::default()
|
||||
};
|
||||
assert_eq!(
|
||||
ClaudeCodeProbe::start(options).err(),
|
||||
Some(HarnessError::ProbeIntervalTooShort)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_probe_reports_harness_reported_tokens_with_no_ceiling() {
|
||||
let root = scratch("probe");
|
||||
append(&root.join("project-a").join("session.jsonl"), 4);
|
||||
let mut probe = ClaudeCodeProbe::start(ClaudeCodeProbeOptions {
|
||||
projects_root: root.clone(),
|
||||
rate_limit_feed: root.join("feed.jsonl"),
|
||||
poll_interval: Duration::from_secs(1),
|
||||
})
|
||||
.expect("the probe starts");
|
||||
|
||||
let mut observations = Vec::new();
|
||||
for _ in 0..200 {
|
||||
observations.extend(probe.poll().into_observations());
|
||||
if !observations.is_empty() {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
probe.shutdown();
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
|
||||
let observation = observations.first().expect("a reading arrives");
|
||||
assert_eq!(observation.provenance(), UsageProvenance::HarnessReported);
|
||||
assert_eq!(observation.unit(), Some(UsageUnit::Tokens));
|
||||
assert_eq!(observation.consumed(), 4 * 1_111);
|
||||
assert_eq!(
|
||||
observation.limit(),
|
||||
None,
|
||||
"a transcript states spend, never a ceiling"
|
||||
);
|
||||
assert_eq!(
|
||||
observation.window(),
|
||||
None,
|
||||
"a transcript states spend, never a reset"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_status_line_feed_becomes_provider_reported_windows() {
|
||||
let root = scratch("windows");
|
||||
let feed = root.join("feed.jsonl");
|
||||
// The shape the bridge writes, with a reset far enough ahead that the
|
||||
// window is still open when the probe reads it.
|
||||
let resets_at = (std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.expect("a clock after the epoch")
|
||||
.as_secs())
|
||||
+ 3_600;
|
||||
std::fs::write(
|
||||
&feed,
|
||||
format!(
|
||||
"{{\"rate_limits_available\":true,\"rate_limits\":{{\"five_hour\":{{\"used_percentage\":42,\"resets_at\":{resets_at}}},\"seven_day\":{{\"used_percentage\":8.5,\"resets_at\":{resets_at}}}}}}}\n"
|
||||
),
|
||||
)
|
||||
.expect("feed is writable");
|
||||
|
||||
let mut probe = ClaudeCodeProbe::start(ClaudeCodeProbeOptions {
|
||||
projects_root: root.clone(),
|
||||
rate_limit_feed: feed,
|
||||
poll_interval: Duration::from_secs(1),
|
||||
})
|
||||
.expect("the probe starts");
|
||||
|
||||
let mut observations = Vec::new();
|
||||
for _ in 0..200 {
|
||||
observations.extend(probe.poll().into_observations());
|
||||
if observations.len() >= 2 {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
probe.shutdown();
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
|
||||
let five = observations
|
||||
.iter()
|
||||
.find(|observation| observation.profile().as_str() == "claude-code-five-hour")
|
||||
.expect("the five-hour window is reported");
|
||||
assert_eq!(
|
||||
five.provenance(),
|
||||
UsageProvenance::ProviderReported,
|
||||
"the CLI relays the provider's own rate-limit headers"
|
||||
);
|
||||
assert_eq!(five.unit(), Some(UsageUnit::WindowPermille));
|
||||
assert_eq!(five.consumed(), 420);
|
||||
assert_eq!(
|
||||
five.limit(),
|
||||
Some(1_000),
|
||||
"a percentage is a share of one whole"
|
||||
);
|
||||
assert!(five.window().is_some(), "a documented window has a reset");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_account_without_plan_limits_reports_a_gap_not_a_zero() {
|
||||
let root = scratch("nolimits");
|
||||
let feed = root.join("feed.jsonl");
|
||||
std::fs::write(&feed, "{\"rate_limits_available\":false}\n").expect("writable");
|
||||
let mut probe = ClaudeCodeProbe::start(ClaudeCodeProbeOptions {
|
||||
projects_root: root.clone(),
|
||||
rate_limit_feed: feed,
|
||||
poll_interval: Duration::from_secs(1),
|
||||
})
|
||||
.expect("starts");
|
||||
let mut observations = Vec::new();
|
||||
for _ in 0..200 {
|
||||
observations.extend(probe.poll().into_observations());
|
||||
if !observations.is_empty() {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
probe.shutdown();
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
let window = observations
|
||||
.iter()
|
||||
.find(|observation| observation.profile().as_str() == "claude-code-five-hour")
|
||||
.expect("the window is still named");
|
||||
assert_eq!(window.provenance(), UsageProvenance::Unavailable);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shutdown_is_prompt_and_idempotent() {
|
||||
let root = scratch("shutdown");
|
||||
let mut probe = ClaudeCodeProbe::start(ClaudeCodeProbeOptions {
|
||||
projects_root: root.clone(),
|
||||
rate_limit_feed: root.join("feed.jsonl"),
|
||||
poll_interval: Duration::from_secs(60),
|
||||
})
|
||||
.expect("starts");
|
||||
let started = std::time::Instant::now();
|
||||
probe.shutdown();
|
||||
probe.shutdown();
|
||||
assert!(started.elapsed() < Duration::from_secs(5));
|
||||
assert!(matches!(
|
||||
probe.poll().health(),
|
||||
ProbeHealth::Stopped | ProbeHealth::Starting | ProbeHealth::Ready
|
||||
));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_default_root_follows_the_documented_location() {
|
||||
let root = default_projects_root();
|
||||
assert!(root.ends_with("projects"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,337 @@
|
||||
//! Claude Code's status-line rate-limit payload: the subscription window.
|
||||
//!
|
||||
//! Claude Code 2.1.80 and later pipe a `rate_limits` object to the configured
|
||||
//! `statusLine` command on every turn. The shape is documented inside the CLI
|
||||
//! itself, which describes `five_hour` and `seven_day` as
|
||||
//!
|
||||
//! ```text
|
||||
//! "five_hour": { // present only while the API reports it and its resets_at has not passed
|
||||
//! "used_percentage": number, // Percentage of limit used (0-100)
|
||||
//! "resets_at": number // Unix epoch seconds when this window resets
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! These are the real quota windows, and they cost nothing to read: the CLI is
|
||||
//! relaying rate-limit headers that already came back on its own API responses.
|
||||
//! That makes them `ProviderReported` under decision 0013's test — the harness
|
||||
//! forwards the provider's number rather than computing one.
|
||||
//!
|
||||
//! `rate_limits_available` is the honest-gap signal, and the CLI documents it:
|
||||
//! "False when plan rate limits do not apply (API key, Bedrock, Vertex, or
|
||||
//! missing profile scope) — `rate_limits` will be null." An API-key user has no
|
||||
//! subscription window, and this field says so rather than leaving us guessing.
|
||||
//!
|
||||
//! Nothing else from the status-line payload is modelled. It also carries the
|
||||
//! session's cost, its transcript path, the current working directory, and the
|
||||
//! model — none of which this parser can represent.
|
||||
|
||||
use lumbridge_core::UsageWindow;
|
||||
use serde::Deserialize;
|
||||
|
||||
/// Percent is 0-100 on the wire; a spend limit may exceed 100 once breached.
|
||||
const MAX_PERCENT: f64 = 100.0;
|
||||
const PERMILLE_PER_PERCENT: f64 = 10.0;
|
||||
const MILLIS_PER_SECOND: i64 = 1_000;
|
||||
/// Documented window lengths, used to give each reading a window start so the
|
||||
/// ledger can scope a burn rate to one quota period.
|
||||
const FIVE_HOUR_MS: u64 = 5 * 60 * 60 * 1_000;
|
||||
const SEVEN_DAY_MS: u64 = 7 * 24 * 60 * 60 * 1_000;
|
||||
|
||||
/// One line of the feed a Lumbridge status-line bridge appends.
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub(crate) struct FeedLineWire {
|
||||
#[serde(default)]
|
||||
pub(crate) rate_limits_available: Option<bool>,
|
||||
#[serde(default)]
|
||||
pub(crate) rate_limits: Option<RateLimitsWire>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Deserialize)]
|
||||
pub(crate) struct RateLimitsWire {
|
||||
#[serde(default)]
|
||||
pub(crate) five_hour: Option<WindowWire>,
|
||||
#[serde(default)]
|
||||
pub(crate) seven_day: Option<WindowWire>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub(crate) struct WindowWire {
|
||||
/// The documented field. `utilization` is accepted as a sibling because the
|
||||
/// OAuth usage endpoint spells the same quantity that way, and a rename
|
||||
/// should degrade rather than silently go dark.
|
||||
#[serde(default)]
|
||||
pub(crate) used_percentage: Option<f64>,
|
||||
#[serde(default)]
|
||||
pub(crate) utilization: Option<f64>,
|
||||
#[serde(default)]
|
||||
pub(crate) resets_at: Option<i64>,
|
||||
}
|
||||
|
||||
/// Which documented subscription window a reading describes.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum ClaudeWindowKind {
|
||||
FiveHour,
|
||||
SevenDay,
|
||||
}
|
||||
|
||||
impl ClaudeWindowKind {
|
||||
pub(crate) const ALL: [Self; 2] = [Self::FiveHour, Self::SevenDay];
|
||||
|
||||
/// Static, account-free profile identifiers.
|
||||
pub(crate) const fn profile_id(self) -> &'static str {
|
||||
match self {
|
||||
Self::FiveHour => "claude-code-five-hour",
|
||||
Self::SevenDay => "claude-code-seven-day",
|
||||
}
|
||||
}
|
||||
|
||||
/// Named for the window, not for a model: these limits are account-scoped.
|
||||
pub(crate) const fn scope(self) -> &'static str {
|
||||
match self {
|
||||
Self::FiveHour => "five-hour window",
|
||||
Self::SevenDay => "seven-day window",
|
||||
}
|
||||
}
|
||||
|
||||
const fn length_ms(self) -> u64 {
|
||||
match self {
|
||||
Self::FiveHour => FIVE_HOUR_MS,
|
||||
Self::SevenDay => SEVEN_DAY_MS,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What one window in the feed means at a point in time.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum WindowReading {
|
||||
/// A usable share of the window.
|
||||
Usable {
|
||||
permille: u64,
|
||||
window: Option<UsageWindow>,
|
||||
},
|
||||
/// The window is absent, or the account has no plan limits at all.
|
||||
Absent,
|
||||
}
|
||||
|
||||
/// Reads one window against the moment it was observed.
|
||||
///
|
||||
/// Total: every input produces a decision and none of them panics.
|
||||
pub(crate) fn parse_window(
|
||||
wire: &WindowWire,
|
||||
kind: ClaudeWindowKind,
|
||||
observed_at_ms: u64,
|
||||
) -> WindowReading {
|
||||
let Some(percent) = wire.used_percentage.or(wire.utilization) else {
|
||||
return WindowReading::Absent;
|
||||
};
|
||||
if !percent.is_finite() || percent < 0.0 {
|
||||
return WindowReading::Absent;
|
||||
}
|
||||
// A spend limit can report over 100 once breached; a quota share cannot
|
||||
// exceed the whole, so clamp rather than letting the ledger hold >100%.
|
||||
//
|
||||
// The value is clamped to 0..=100 and scaled by ten before rounding, so it
|
||||
// is a whole number in 0..=1000 — well inside u64 and inside the range f64
|
||||
// represents exactly. The conversion cannot truncate or lose a sign here.
|
||||
#[expect(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
reason = "a rounded whole number clamped to 0..=1000"
|
||||
)]
|
||||
let permille = (percent.clamp(0.0, MAX_PERCENT) * PERMILLE_PER_PERCENT).round() as u64;
|
||||
|
||||
let window = wire
|
||||
.resets_at
|
||||
.and_then(unix_seconds_to_millis)
|
||||
// The CLI documents that a window is present only while its resets_at
|
||||
// has not passed. One that has is stale, so its start-and-reset pair is
|
||||
// dropped and only the percentage survives.
|
||||
.filter(|resets_at_ms| *resets_at_ms > observed_at_ms)
|
||||
.and_then(|resets_at_ms| {
|
||||
let started_at_ms = resets_at_ms.saturating_sub(kind.length_ms());
|
||||
if started_at_ms <= observed_at_ms {
|
||||
UsageWindow::new(started_at_ms, resets_at_ms).ok()
|
||||
} else {
|
||||
// The local clock disagrees with the provider about the start.
|
||||
// Keep the reset, which is the half that bounds a forecast.
|
||||
Some(UsageWindow::until(resets_at_ms))
|
||||
}
|
||||
});
|
||||
|
||||
WindowReading::Usable { permille, window }
|
||||
}
|
||||
|
||||
fn unix_seconds_to_millis(seconds: i64) -> Option<u64> {
|
||||
u64::try_from(seconds.checked_mul(MILLIS_PER_SECOND)?).ok()
|
||||
}
|
||||
|
||||
/// Reads one feed line into a reading per documented window.
|
||||
///
|
||||
/// `rate_limits_available: false` is a positive statement that this account has
|
||||
/// no plan limits, so every window reads as absent rather than unknown.
|
||||
pub(crate) fn parse_feed_line(
|
||||
line: &str,
|
||||
observed_at_ms: u64,
|
||||
) -> Option<[(ClaudeWindowKind, WindowReading); 2]> {
|
||||
let feed: FeedLineWire = serde_json::from_str(line).ok()?;
|
||||
if feed.rate_limits_available == Some(false) {
|
||||
return Some([
|
||||
(ClaudeWindowKind::FiveHour, WindowReading::Absent),
|
||||
(ClaudeWindowKind::SevenDay, WindowReading::Absent),
|
||||
]);
|
||||
}
|
||||
let limits = feed.rate_limits?;
|
||||
let five = limits
|
||||
.five_hour
|
||||
.as_ref()
|
||||
.map_or(WindowReading::Absent, |w| {
|
||||
parse_window(w, ClaudeWindowKind::FiveHour, observed_at_ms)
|
||||
});
|
||||
let seven = limits
|
||||
.seven_day
|
||||
.as_ref()
|
||||
.map_or(WindowReading::Absent, |w| {
|
||||
parse_window(w, ClaudeWindowKind::SevenDay, observed_at_ms)
|
||||
});
|
||||
Some([
|
||||
(ClaudeWindowKind::FiveHour, five),
|
||||
(ClaudeWindowKind::SevenDay, seven),
|
||||
])
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{ClaudeWindowKind, WindowReading, parse_feed_line};
|
||||
|
||||
const HOUR_MS: u64 = 3_600_000;
|
||||
const RESETS_AT_SECONDS: i64 = 1_800_000_000;
|
||||
const RESETS_AT_MS: u64 = 1_800_000_000_000;
|
||||
|
||||
fn line(five: &str, seven: &str) -> String {
|
||||
format!(
|
||||
r#"{{"rate_limits_available":true,"rate_limits":{{"five_hour":{five},"seven_day":{seven}}}}}"#
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_documented_shape_yields_both_windows() {
|
||||
let body = line(
|
||||
&format!(r#"{{"used_percentage":42,"resets_at":{RESETS_AT_SECONDS}}}"#),
|
||||
&format!(r#"{{"used_percentage":8.5,"resets_at":{RESETS_AT_SECONDS}}}"#),
|
||||
);
|
||||
let readings = parse_feed_line(&body, RESETS_AT_MS - HOUR_MS).expect("a valid feed line");
|
||||
let WindowReading::Usable { permille, window } = readings[0].1 else {
|
||||
panic!("the five-hour window is usable");
|
||||
};
|
||||
assert_eq!(readings[0].0, ClaudeWindowKind::FiveHour);
|
||||
assert_eq!(permille, 420);
|
||||
let window = window.expect("a documented window length builds a window");
|
||||
assert_eq!(window.resets_at_ms(), RESETS_AT_MS);
|
||||
assert_eq!(
|
||||
window.started_at_ms(),
|
||||
Some(RESETS_AT_MS - 5 * HOUR_MS),
|
||||
"the five-hour window starts five hours before it resets"
|
||||
);
|
||||
|
||||
let WindowReading::Usable { permille, .. } = readings[1].1 else {
|
||||
panic!("the seven-day window is usable");
|
||||
};
|
||||
assert_eq!(permille, 85, "a fractional percent keeps its tenth");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_api_key_account_reports_absent_rather_than_zero() {
|
||||
// The CLI documents rate_limits_available as false for API key,
|
||||
// Bedrock, Vertex, or a missing profile scope.
|
||||
let readings = parse_feed_line(
|
||||
r#"{"rate_limits_available":false,"rate_limits":null}"#,
|
||||
RESETS_AT_MS,
|
||||
)
|
||||
.expect("a valid feed line");
|
||||
assert!(readings.iter().all(|(_, r)| *r == WindowReading::Absent));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_missing_window_is_absent_without_affecting_the_other() {
|
||||
let body = line(
|
||||
&format!(r#"{{"used_percentage":10,"resets_at":{RESETS_AT_SECONDS}}}"#),
|
||||
"null",
|
||||
);
|
||||
let readings = parse_feed_line(&body, RESETS_AT_MS - HOUR_MS).expect("valid");
|
||||
assert!(matches!(readings[0].1, WindowReading::Usable { .. }));
|
||||
assert_eq!(readings[1].1, WindowReading::Absent);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_reset_that_has_passed_keeps_the_percentage_and_drops_the_window() {
|
||||
let body = line(
|
||||
&format!(r#"{{"used_percentage":99,"resets_at":{RESETS_AT_SECONDS}}}"#),
|
||||
"null",
|
||||
);
|
||||
let readings = parse_feed_line(&body, RESETS_AT_MS + 1).expect("valid");
|
||||
assert_eq!(
|
||||
readings[0].1,
|
||||
WindowReading::Usable {
|
||||
permille: 990,
|
||||
window: None
|
||||
},
|
||||
"a window whose reset has passed is stale, not current"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_oauth_spelling_of_the_same_field_is_accepted() {
|
||||
let body = line(
|
||||
&format!(r#"{{"utilization":25,"resets_at":{RESETS_AT_SECONDS}}}"#),
|
||||
"null",
|
||||
);
|
||||
let readings = parse_feed_line(&body, RESETS_AT_MS - HOUR_MS).expect("valid");
|
||||
assert!(matches!(
|
||||
readings[0].1,
|
||||
WindowReading::Usable { permille: 250, .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_over_range_or_nonsense_percentage_is_clamped_or_refused() {
|
||||
let over = line(r#"{"used_percentage":140}"#, "null");
|
||||
assert!(matches!(
|
||||
parse_feed_line(&over, 0).expect("valid")[0].1,
|
||||
WindowReading::Usable {
|
||||
permille: 1_000,
|
||||
window: None
|
||||
}
|
||||
));
|
||||
for bad in ["-1", "null"] {
|
||||
let body = line(&format!(r#"{{"used_percentage":{bad}}}"#), "null");
|
||||
assert_eq!(
|
||||
parse_feed_line(&body, 0).expect("valid")[0].1,
|
||||
WindowReading::Absent
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_or_unrelated_lines_are_refused() {
|
||||
for body in ["", "not json", "{}", r#"{"rate_limits":{}}"#] {
|
||||
let parsed = parse_feed_line(body, 0);
|
||||
assert!(
|
||||
parsed.is_none_or(|readings| readings
|
||||
.iter()
|
||||
.all(|(_, r)| *r == WindowReading::Absent)),
|
||||
"{body:?} must not produce a number"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn window_kinds_have_distinct_account_free_identifiers() {
|
||||
assert_ne!(
|
||||
ClaudeWindowKind::FiveHour.profile_id(),
|
||||
ClaudeWindowKind::SevenDay.profile_id()
|
||||
);
|
||||
for kind in ClaudeWindowKind::ALL {
|
||||
assert!(!kind.profile_id().contains('@'));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,286 @@
|
||||
//! The Claude Code transcript reader: token counts in, nothing else out.
|
||||
//!
|
||||
//! Claude Code writes one JSONL record per event to
|
||||
//! `~/.claude/projects/<encoded-cwd>/<session>.jsonl`. Assistant records carry
|
||||
//! the API's `usage` object verbatim, which is the only documented local
|
||||
//! surface that reports what the harness actually spent.
|
||||
//!
|
||||
//! **These files contain the user's conversations.** This module is built so
|
||||
//! that reading one cannot surface their content: the wire types below model
|
||||
//! the record type, the model name, and the four token counters, and nothing
|
||||
//! else. Serde discards every unmodelled field while parsing, so message text,
|
||||
//! tool inputs, and file contents are never deserialized into a Lumbridge value
|
||||
//! at all. That is a structural guarantee, not a convention — there is no
|
||||
//! field here that could hold them.
|
||||
|
||||
use serde::Deserialize;
|
||||
|
||||
/// A transcript line is a single JSON object. Anything longer than this is not
|
||||
/// a record we can use, and buffering it would let one file grow this process.
|
||||
pub(crate) const MAX_RECORD_BYTES: usize = 1024 * 1024;
|
||||
|
||||
/// The only fields this crate reads from a transcript record.
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub(crate) struct RecordWire {
|
||||
#[serde(rename = "type")]
|
||||
pub(crate) record_type: Option<String>,
|
||||
#[serde(default)]
|
||||
pub(crate) message: Option<MessageWire>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub(crate) struct MessageWire {
|
||||
#[serde(default)]
|
||||
pub(crate) model: Option<String>,
|
||||
#[serde(default)]
|
||||
pub(crate) usage: Option<UsageWire>,
|
||||
}
|
||||
|
||||
/// The API usage object as Claude Code records it.
|
||||
///
|
||||
/// `cache_creation`, `server_tool_use`, `iterations`, `service_tier`, `speed`,
|
||||
/// and `inference_geo` are all present on the wire and all deliberately
|
||||
/// unmodelled: a probe should not be able to read account or routing metadata
|
||||
/// it has no use for.
|
||||
///
|
||||
/// The wire names are pinned with explicit renames so the Rust field names can
|
||||
/// read naturally without the contract drifting from what Claude Code writes.
|
||||
#[derive(Debug, Default, Deserialize)]
|
||||
pub(crate) struct UsageWire {
|
||||
#[serde(default, rename = "input_tokens")]
|
||||
pub(crate) input: u64,
|
||||
#[serde(default, rename = "output_tokens")]
|
||||
pub(crate) output: u64,
|
||||
#[serde(default, rename = "cache_creation_input_tokens")]
|
||||
pub(crate) cache_creation: u64,
|
||||
#[serde(default, rename = "cache_read_input_tokens")]
|
||||
pub(crate) cache_read: u64,
|
||||
}
|
||||
|
||||
/// Tokens the harness recorded, kept split by kind.
|
||||
///
|
||||
/// The four counters are priced very differently and mean different things, so
|
||||
/// they are carried separately rather than collapsed at the point of parsing.
|
||||
/// The ledger holds one number per profile; [`TokenTally::total`] is what it
|
||||
/// gets, and the split stays available for a detail view.
|
||||
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
|
||||
pub struct TokenTally {
|
||||
pub input: u64,
|
||||
pub output: u64,
|
||||
pub cache_creation: u64,
|
||||
pub cache_read: u64,
|
||||
}
|
||||
|
||||
impl TokenTally {
|
||||
/// Every token the API processed.
|
||||
///
|
||||
/// Cache reads are included because they are billed and do count against a
|
||||
/// subscription window, even though they are re-reads of context already
|
||||
/// counted once. Excluding them would understate consumption; weighting
|
||||
/// them by price would require a price list this adapter does not have and
|
||||
/// would turn a counted fact into an estimate.
|
||||
#[must_use]
|
||||
pub const fn total(self) -> u64 {
|
||||
self.input
|
||||
.saturating_add(self.output)
|
||||
.saturating_add(self.cache_creation)
|
||||
.saturating_add(self.cache_read)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub const fn is_empty(self) -> bool {
|
||||
self.total() == 0
|
||||
}
|
||||
|
||||
pub(crate) fn add(&mut self, other: Self) {
|
||||
self.input = self.input.saturating_add(other.input);
|
||||
self.output = self.output.saturating_add(other.output);
|
||||
self.cache_creation = self.cache_creation.saturating_add(other.cache_creation);
|
||||
self.cache_read = self.cache_read.saturating_add(other.cache_read);
|
||||
}
|
||||
}
|
||||
|
||||
/// What one transcript line contributed.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq)]
|
||||
pub(crate) struct RecordTally {
|
||||
pub(crate) tokens: TokenTally,
|
||||
/// The model that produced the record, when it named one.
|
||||
pub(crate) model: Option<String>,
|
||||
}
|
||||
|
||||
/// Reads one transcript line.
|
||||
///
|
||||
/// Total: a line that is not JSON, not an assistant record, or carries no
|
||||
/// usage contributes nothing. A malformed line is skipped rather than faulting
|
||||
/// the probe, because a transcript is append-only and the last line of a file
|
||||
/// being written is routinely incomplete.
|
||||
pub(crate) fn parse_record(line: &str) -> Option<RecordTally> {
|
||||
let record: RecordWire = serde_json::from_str(line).ok()?;
|
||||
if record.record_type.as_deref() != Some("assistant") {
|
||||
return None;
|
||||
}
|
||||
let message = record.message?;
|
||||
let usage = message.usage?;
|
||||
let tokens = TokenTally {
|
||||
input: usage.input,
|
||||
output: usage.output,
|
||||
cache_creation: usage.cache_creation,
|
||||
cache_read: usage.cache_read,
|
||||
};
|
||||
if tokens.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(RecordTally {
|
||||
tokens,
|
||||
model: message.model,
|
||||
})
|
||||
}
|
||||
|
||||
/// Sums the complete lines in a chunk, reporting how many bytes were consumed.
|
||||
///
|
||||
/// A trailing partial line is left unconsumed so the next read picks it up
|
||||
/// whole. This is what makes the running total monotonic: every byte is
|
||||
/// counted exactly once, and a file that is still being appended to never
|
||||
/// double-counts or loses its last record.
|
||||
pub(crate) fn consume_chunk(chunk: &str) -> (RecordTally, usize) {
|
||||
let mut tally = RecordTally::default();
|
||||
let mut consumed = 0;
|
||||
for line in chunk.split_inclusive('\n') {
|
||||
if !line.ends_with('\n') {
|
||||
break; // A partial record. Leave it for the next read.
|
||||
}
|
||||
consumed += line.len();
|
||||
if line.len() > MAX_RECORD_BYTES {
|
||||
continue;
|
||||
}
|
||||
if let Some(record) = parse_record(line.trim_end()) {
|
||||
tally.tokens.add(record.tokens);
|
||||
if record.model.is_some() {
|
||||
tally.model = record.model;
|
||||
}
|
||||
}
|
||||
}
|
||||
(tally, consumed)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{TokenTally, consume_chunk, parse_record};
|
||||
|
||||
/// Shaped exactly like a real assistant record, with the content fields a
|
||||
/// real one carries so the test proves they are ignored rather than absent.
|
||||
///
|
||||
/// Deliberately one line: a transcript is one JSON object per line, and a
|
||||
/// fixture that wrapped would not exercise the chunk boundary logic.
|
||||
const ASSISTANT: &str = concat!(
|
||||
r#"{"type":"assistant","uuid":"u1","sessionId":"s1","cwd":"/home/x","#,
|
||||
r#""message":{"id":"msg_1","role":"assistant","model":"claude-opus-5","#,
|
||||
r#""content":[{"type":"text","text":"SECRET CONVERSATION CONTENT"}],"#,
|
||||
r#""usage":{"input_tokens":10,"output_tokens":200,"#,
|
||||
r#""cache_creation_input_tokens":1000,"cache_read_input_tokens":50000,"#,
|
||||
r#""service_tier":"standard","cache_creation":{"ephemeral_5m_input_tokens":7}}}}"#,
|
||||
);
|
||||
|
||||
#[test]
|
||||
fn an_assistant_record_yields_its_four_counters() {
|
||||
let record = parse_record(ASSISTANT).expect("an assistant record with usage counts");
|
||||
assert_eq!(
|
||||
record.tokens,
|
||||
TokenTally {
|
||||
input: 10,
|
||||
output: 200,
|
||||
cache_creation: 1_000,
|
||||
cache_read: 50_000,
|
||||
}
|
||||
);
|
||||
assert_eq!(record.model.as_deref(), Some("claude-opus-5"));
|
||||
assert_eq!(record.tokens.total(), 51_210);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conversation_content_is_not_representable() {
|
||||
// The record above carries message text. The parser's own output type
|
||||
// has nowhere to put it, so this is a property of the types rather
|
||||
// than of the parsing.
|
||||
let record = parse_record(ASSISTANT).expect("parsed");
|
||||
let rendered = format!("{record:?}");
|
||||
assert!(
|
||||
!rendered.contains("SECRET"),
|
||||
"no debug rendering of a parsed record may include message content"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_assistant_records_contribute_nothing() {
|
||||
for line in [
|
||||
r#"{"type":"user","message":{"role":"user","content":"hello"}}"#,
|
||||
r#"{"type":"attachment","attachment":{"content":"a file"}}"#,
|
||||
r#"{"type":"system","subtype":"hook"}"#,
|
||||
] {
|
||||
assert!(parse_record(line).is_none(), "{line} must not be counted");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_assistant_record_without_usage_contributes_nothing() {
|
||||
assert!(
|
||||
parse_record(r#"{"type":"assistant","message":{"model":"claude-opus-5"}}"#).is_none()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_lines_are_skipped_rather_than_faulting() {
|
||||
for line in ["", "not json", "{", "[]", "\"a string\""] {
|
||||
assert!(parse_record(line).is_none(), "{line:?} must not panic");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_partial_trailing_line_is_left_for_the_next_read() {
|
||||
let chunk = format!("{ASSISTANT}\n{{\"type\":\"assis");
|
||||
let (tally, consumed) = consume_chunk(&chunk);
|
||||
assert_eq!(tally.tokens.total(), 51_210);
|
||||
assert_eq!(
|
||||
consumed,
|
||||
ASSISTANT.len() + 1,
|
||||
"only the complete record may be consumed"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resuming_from_the_offset_counts_every_byte_exactly_once() {
|
||||
let whole = format!("{ASSISTANT}\n{ASSISTANT}\n");
|
||||
let (first, consumed) = consume_chunk(&whole[..ASSISTANT.len() + 4]);
|
||||
let (second, _) = consume_chunk(&whole[consumed..]);
|
||||
let mut total = first.tokens;
|
||||
total.add(second.tokens);
|
||||
assert_eq!(
|
||||
total.total(),
|
||||
2 * 51_210,
|
||||
"a split read must not double-count or drop a record"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_oversized_line_is_consumed_but_not_parsed() {
|
||||
let line = format!("{}\n", "x".repeat(super::MAX_RECORD_BYTES + 1));
|
||||
let (tally, consumed) = consume_chunk(&line);
|
||||
assert_eq!(tally.tokens.total(), 0);
|
||||
assert_eq!(
|
||||
consumed,
|
||||
line.len(),
|
||||
"the offset must still advance past it"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn totals_saturate_rather_than_overflowing() {
|
||||
let tally = TokenTally {
|
||||
input: u64::MAX,
|
||||
output: u64::MAX,
|
||||
cache_creation: 0,
|
||||
cache_read: 0,
|
||||
};
|
||||
assert_eq!(tally.total(), u64::MAX);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user