Generate the theme catalog from real themes instead of a hand table

The derivation landed in decision 0018 with the catalog and the terminal ANSI
palette still fixed tables written by hand. tools/theme-gen reads the
TextMate themes under assets/themes/ and emits the catalog and the reference
vectors, so the anchors a palette is derived from are the ones the theme
actually ships rather than the ones somebody transcribed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SPYebLiN2w4TqnHUYGdECq
This commit is contained in:
Metal Agent
2026-09-01 12:51:25 -07:00
co-authored by Claude Opus 5
parent 7bee985279
commit 401760d670
108 changed files with 72648 additions and 12 deletions
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "theme-gen"
description = "Dev-only generator that turns the vendored TextMate themes into lumbridge-theme's catalog"
publish = false
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
lumbridge-theme = { path = "../../crates/lumbridge-theme" }
serde_json = "1.0"
[lints]
workspace = true
+60
View File
@@ -0,0 +1,60 @@
# theme-gen
Turns the theme files vendored in `assets/themes/` into
`crates/lumbridge-theme/src/generated/catalog.rs` and
`assets/themes/NOTICE-THEMES.md`.
```bash
cargo run -p theme-gen # rewrite both files
cargo run -p theme-gen -- --check # fail if either is out of date
```
Dev-only. Nothing in the workspace depends on it at build time, and `--check` is
the hook a future CI step would use.
## Why it is a binary and not a `build.rs`
Two reasons, in order of weight.
The colours a user sees should be reviewable as text in a pull request. If an
upstream theme changes a hex digit, that should arrive as a diff someone reads,
not as a silently different build. Everything this tool produces is checked in
for exactly that reason, and `--check` is what keeps the checked-in copy honest.
And a build that reads `assets/themes/` makes the asset tree a build input.
Trimming assets in a packaging step would then break compilation rather than
breaking a theme picker, which is the wrong failure.
## What it decides, and what it only measures
The tool makes no aesthetic choices. It applies rules and records what they
produced:
- **Anchors.** `editor.background`, `editor.foreground`, the comment token's
foreground, and the git-decoration colours, by the rules in `src/extract.rs`.
- **Terminal palette.** The theme's `terminal.ansi*` keys where it has them, a
syntax token that stands for the same role where it does not, and a documented
hue derived from the theme's own foreground where it has neither.
- **Contrast gate.** A theme whose body text measures below 4.5:1 against the
surface Lumbridge derives for it is held back, and the measurement is written
into the generated file's header. Three themes currently fail.
- **Distinctness.** Two escape codes that would paint the same colour, and two
role colours that would make decision 0012's provenance shades identical, are
forced apart — and every such substitution is written down twice: as a comment
above the theme, and in a machine-readable constant the tests read back.
Licence exclusion happens **before** this tool, at the vendoring step, so a GPL
theme is never in the tree at all. See `assets/themes/SOURCE.md`.
## `reference/`
`reference/emit-reference-vectors.mjs` regenerates
`crates/lumbridge-theme/src/generated/reference_vectors.rs` by running Buzz's
original TypeScript under Node over the same vendored theme files. It needs the
Buzz checkout, which lives outside this repository and is pinned in
`docs/RESEARCH_SNAPSHOTS.md`, so it is not a step CI can run — which is why its
output is checked in and the test that reads it runs everywhere.
Run it only when the derivation itself changes. Decision 0018 explains at some
length why these vectors must come from running the original and never from
re-deriving them.
@@ -0,0 +1,211 @@
/**
* Emits the golden vectors the Rust derivation is tested against, by running
* Buzz's original TypeScript under Node.
*
* Decision 0018 records why this exists as a script rather than as a table
* someone typed in. A re-implementation of the derivation in Python reports
* `#191c20` for github-dark's chrome; the real answer is `#171a1d`. Python's
* `round` is banker's rounding and JavaScript's `Math.round` is half-up, they
* disagree on exactly one channel value — 22.5 — and that one channel decides
* whether the luminance bisection's convergence test trips a step early. A
* research pass once claimed to have reproduced these byte-exactly and had not.
* So: nothing here reimplements anything. The reference modules are imported
* from the pinned Buzz checkout and called.
*
* node tools/theme-gen/reference/emit-reference-vectors.mjs \
* [--buzz <path to the block/buzz checkout>] \
* [--out <path to reference_vectors.rs>]
*
* Buzz is pinned in docs/RESEARCH_SNAPSHOTS.md at commit cb3144999beb and lives
* outside this repository, so this is not a step anyone can run in CI. That is
* the point of checking the output in: the vectors travel with the repository
* and the test runs everywhere, while regenerating them needs the reference.
*
* Two adaptations, both mechanical, both necessary, neither of them a rewrite:
*
* - `adaptive-theme.ts` keeps `mix`, `adjust` and `calculateChromeColors`
* module-private and only exports a shadcn CSS variable map, whose colours
* have been through a lossy hex→HSL string conversion. A `load` hook appends
* an export statement to the module source so the exact functions can be
* called; the source itself is untouched on disk.
* - Buzz feeds `extractThemeInfo` a theme loaded through Shiki, and reads
* token rules from `theme.settings`. Shiki 4.1.0's bundled themes expose
* that array as `tokenColors`, so the script renames the field before
* handing the document over. Without the rename the reference reports every
* theme's comment colour as its foreground — which is what Buzz itself does
* at the pinned commit, and is a bug there rather than the rule this catalog
* is built on.
*/
import { registerHooks } from "node:module";
import { readFileSync, readdirSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = resolve(HERE, "../../..");
function argument(name, fallback) {
const index = process.argv.indexOf(`--${name}`);
return index >= 0 && process.argv[index + 1] ? process.argv[index + 1] : fallback;
}
const BUZZ = argument(
"buzz",
resolve(ROOT, "../Research/agent-ides/buzz"),
);
const ASSETS = argument("assets", join(ROOT, "assets/themes"));
const OUT = argument(
"out",
join(ROOT, "crates/lumbridge-theme/src/generated/reference_vectors.rs"),
);
const THEME_DIR = `${BUZZ}/desktop/src/shared/theme`;
// TypeScript source in the reference checkout uses extensionless relative
// imports, which Node does not resolve, and keeps the derivation's building
// blocks private. Both are fixed at load time so the files on disk stay as
// upstream wrote them.
registerHooks({
resolve(specifier, context, next) {
if (specifier.startsWith(".") && !/\.[cm]?[jt]sx?$/.test(specifier)) {
return next(`${specifier}.ts`, context);
}
return next(specifier, context);
},
load(url, context, next) {
const result = next(url, context);
if (!url.endsWith("/adaptive-theme.ts")) return result;
const source = result.source.toString();
return {
...result,
source: `${source}\nexport { mix as __mix, adjust as __adjust, calculateChromeColors as __chromeColors, hexToRgb as __hexToRgb, rgbToHex as __rgbToHex };\n`,
};
},
});
const { extractThemeInfo } = await import(`${THEME_DIR}/theme-loader.ts`);
const adaptive = await import(`${THEME_DIR}/adaptive-theme.ts`);
const { luminance, __mix: mix, __adjust: adjust, __chromeColors: chromeColors } = adaptive;
const normalise = (hex) => adaptive.__rgbToHex(adaptive.__hexToRgb(hex));
const names = readdirSync(join(ASSETS, "tm-themes"))
.filter((file) => file.endsWith(".json"))
.map((file) => file.replace(/\.json$/, ""))
.sort();
const vectors = names.map((name) => {
const document = JSON.parse(
readFileSync(join(ASSETS, "tm-themes", `${name}.json`), "utf8"),
);
const info = extractThemeInfo(name, {
...document,
settings: document.tokenColors,
});
// calculateChromeColors + the role formulas, exactly as createThemeVars runs
// them, but keeping hex instead of the CSS-variable HSL strings.
const bg = normalise(info.bg);
const fg = normalise(info.fg);
const { chrome, primary } = chromeColors(info.bg);
const isDark = luminance(info.bg) < 0.5;
const elevate = (amount) => adjust(primary, (isDark ? 1 : -1) * amount);
const border = mix(primary, info.fg, isDark ? 0.15 : 0.12);
return {
name,
bg,
fg,
comment: normalise(info.comment),
added: info.added && normalise(info.added),
deleted: info.deleted && normalise(info.deleted),
modified: info.modified && normalise(info.modified),
chrome: normalise(chrome),
surface: normalise(primary),
surfaceRaised: normalise(elevate(0.04)),
surfaceActive: normalise(elevate(0.06)),
surfaceOverlay: normalise(elevate(0.08)),
surfaceBetween: normalise(mix(chrome, primary, 0.5)),
border: normalise(border),
borderQuiet: normalise(mix(primary, border, 0.5)),
isDark,
ansi: Object.values(info.terminalPalette.ansi).map(normalise),
};
});
const hex = (value) => `Srgb::from_hex(0x${value.replace("#", "")})`;
const option = (value) => (value ? `Some(${hex(value)})` : "None");
const lines = [];
lines.push("//! Golden vectors from Buzz's original TypeScript, run under Node.");
lines.push("//!");
lines.push("//! Generated by `tools/theme-gen/reference/emit-reference-vectors.mjs` against");
lines.push("//! the Buzz checkout pinned in `docs/RESEARCH_SNAPSHOTS.md` (block/buzz,");
lines.push("//! Apache-2.0, commit `cb3144999beb`) and the theme files vendored under");
lines.push("//! `assets/themes/`. Do not edit by hand, and do not \"correct\" a value here to");
lines.push("//! make a test pass: these are the answers, and a disagreement means the Rust");
lines.push("//! port drifted. Decision 0018 records what it cost to learn that.");
lines.push("//!");
lines.push("//! Test-only: the shipped catalog carries the same colours, so this is a");
lines.push("//! second copy that exists to be compared against, not to be read from.");
lines.push("");
lines.push("use crate::color::Srgb;");
lines.push("");
lines.push("/// What the reference produces for one theme.");
lines.push("///");
lines.push("/// `ansi` is the terminal palette *before* Lumbridge forces colliding base hues");
lines.push("/// apart, so a slot the catalog substituted will differ here on purpose;");
lines.push("/// `catalog::ANSI_SUBSTITUTIONS` says which.");
lines.push("pub struct ReferenceVector {");
for (const field of [
"name: &'static str",
"bg: Srgb",
"fg: Srgb",
"comment: Srgb",
"added: Option<Srgb>",
"deleted: Option<Srgb>",
"modified: Option<Srgb>",
"chrome: Srgb",
"surface: Srgb",
"surface_raised: Srgb",
"surface_active: Srgb",
"surface_overlay: Srgb",
"surface_between: Srgb",
"border: Srgb",
"border_quiet: Srgb",
"is_dark: bool",
"ansi: [Srgb; 16]",
]) {
lines.push(` pub ${field},`);
}
lines.push("}");
lines.push("");
lines.push("/// Every vendored theme, including the ones the catalog holds back.");
lines.push("pub const REFERENCE_VECTORS: &[ReferenceVector] = &[");
for (const vector of vectors) {
lines.push(" ReferenceVector {");
lines.push(` name: ${JSON.stringify(vector.name)},`);
lines.push(` bg: ${hex(vector.bg)},`);
lines.push(` fg: ${hex(vector.fg)},`);
lines.push(` comment: ${hex(vector.comment)},`);
lines.push(` added: ${option(vector.added)},`);
lines.push(` deleted: ${option(vector.deleted)},`);
lines.push(` modified: ${option(vector.modified)},`);
lines.push(` chrome: ${hex(vector.chrome)},`);
lines.push(` surface: ${hex(vector.surface)},`);
lines.push(` surface_raised: ${hex(vector.surfaceRaised)},`);
lines.push(` surface_active: ${hex(vector.surfaceActive)},`);
lines.push(` surface_overlay: ${hex(vector.surfaceOverlay)},`);
lines.push(` surface_between: ${hex(vector.surfaceBetween)},`);
lines.push(` border: ${hex(vector.border)},`);
lines.push(` border_quiet: ${hex(vector.borderQuiet)},`);
lines.push(` is_dark: ${vector.isDark},`);
lines.push(" ansi: [");
for (const color of vector.ansi) lines.push(` ${hex(color)},`);
lines.push(" ],");
lines.push(" },");
}
lines.push("];");
lines.push("");
writeFileSync(OUT, lines.join("\n"));
console.error(`wrote ${vectors.length} reference vectors to ${OUT}`);
+339
View File
@@ -0,0 +1,339 @@
//! Rendering the generated catalog and the attribution table.
//!
//! Both outputs are written as text and checked in, which is the reason this is
//! a binary rather than a `build.rs`: a colour that changes because a theme file
//! changed should turn up in a review diff, not inside a build directory. The
//! same property is what makes `--check` worth having.
use std::fmt::Write as _;
use lumbridge_theme::Srgb;
use lumbridge_theme::terminal::AnsiColor;
use crate::extract::{CommentSource, Extracted};
use crate::source::ThemeMetadata;
/// A theme that survived the gates, ready to be written out.
pub struct Shipped<'a> {
pub metadata: &'a ThemeMetadata,
pub extracted: &'a Extracted,
}
/// A theme that did not, and the measurement that decided it.
pub struct Excluded<'a> {
pub metadata: &'a ThemeMetadata,
pub reason: String,
}
/// The generated Rust catalog.
#[must_use]
pub fn catalog(shipped: &[Shipped<'_>], excluded: &[Excluded<'_>], version: &str) -> String {
let mut out = String::with_capacity(shipped.len() * 1600);
header(&mut out, shipped, excluded, version);
out.push_str("use crate::color::Srgb;\n");
out.push_str("use crate::derive::{RoleOverrides, ThemeAnchors};\n");
out.push_str("use crate::terminal::TerminalPalette;\n\n");
catalog_constant(&mut out, shipped);
comment_source_constant(&mut out, shipped);
skipped_git_key_constant(&mut out, shipped);
substitution_constant(&mut out, shipped);
secondary_text_constant(&mut out, shipped);
out
}
fn comment_source_constant(out: &mut String, shipped: &[Shipped<'_>]) {
out.push_str(
"/// Themes whose secondary-text colour is not the one the reference rule finds.\n\
///\n\
/// The reference matches the exact scope `comment` against a token entry's\n\
/// scope *array*, and falls back to the foreground colour when nothing matches.\n\
/// A theme that spells its scope list as one comma-separated string, or that\n\
/// colours comments as its foreground at partial alpha, therefore ends up with\n\
/// secondary text identical to body text — which would collapse two of decision\n\
/// 0012's provenance colours into one. These are the themes where Lumbridge\n\
/// read further, and what it did.\n\
///\n\
/// The golden-vector test uses this list in both directions: a theme named here\n\
/// must be one the reference could not read, and a theme not named here must\n\
/// match the reference exactly.\n\
pub const COMMENT_SOURCES: &[(&str, &str)] = &[\n",
);
for theme in shipped {
let note = match theme.extracted.comment_source {
CommentSource::ExactScope => continue,
CommentSource::CommaSeparatedScope => {
"scope list spelled as one comma-separated string"
}
CommentSource::DerivedFromForeground => {
"no comment colour distinct from the foreground; mixed toward the background"
}
};
let _ = writeln!(out, " ({:?}, {note:?}),", theme.metadata.name);
}
out.push_str("];\n\n");
}
fn skipped_git_key_constant(out: &mut String, shipped: &[Shipped<'_>]) {
out.push_str(
"/// Git-decoration keys passed over because their colour was already spoken\n\
/// for, as `(theme, role, key)`.\n\
///\n\
/// A theme is free to draw added files in its plain foreground colour — Gruvbox\n\
/// does — or to give \"added\" and \"modified\" the same green — `LaserWave` does.\n\
/// Both are reasonable in a file tree and useless as *colours* here, where\n\
/// decision 0012 needs the provenance shades apart. The theme's next declaration\n\
/// for the same role is read instead. The golden-vector test allows the catalog\n\
/// to differ from the reference on exactly these roles and no others.\n\
pub const SKIPPED_GIT_KEYS: &[(&str, &str, &str)] = &[\n",
);
for theme in shipped {
for skipped in &theme.extracted.skipped_git_keys {
let _ = writeln!(
out,
" ({:?}, {:?}, {:?}),",
theme.metadata.name, skipped.role, skipped.key
);
}
}
out.push_str("];\n\n");
}
fn substitution_constant(out: &mut String, shipped: &[Shipped<'_>]) {
out.push_str(
"/// The ANSI slots that do not carry the colour their theme asked for, as\n\
/// `(theme, SGR index)`.\n\
///\n\
/// A base slot is here when its extracted colour repeated an earlier slot's,\n\
/// which would have made two different escape codes paint the same pixels. A\n\
/// bright slot is here when the theme does not declare it and it is therefore\n\
/// lifted from a base that moved. The comment above each theme in `CATALOG` says\n\
/// what was replaced and with what; this is the machine-readable half of it. The\n\
/// golden-vector test compares the catalog against the reference implementation\n\
/// slot by slot, and this is the list of places the two are allowed to\n\
/// disagree — so a substitution that was never intended cannot hide behind a\n\
/// comment.\n\
pub const ANSI_SUBSTITUTIONS: &[(&str, usize)] = &[\n",
);
for theme in shipped {
for slot in &theme.extracted.diverging_ansi_slots {
let _ = writeln!(
out,
" ({:?}, {slot}), // {:?}",
theme.metadata.name,
AnsiColor::ALL[*slot]
);
}
}
out.push_str("];\n\n");
}
fn header(out: &mut String, shipped: &[Shipped<'_>], excluded: &[Excluded<'_>], version: &str) {
out.push_str("//! The theme catalog, generated by `tools/theme-gen`.\n//!\n");
out.push_str("//! Do not edit by hand. Run `cargo run -p theme-gen` after changing the\n");
out.push_str("//! vendored theme set, and review the diff: every colour here is a fact\n");
out.push_str("//! about a file in `assets/themes/`, not a judgement call.\n//!\n");
let _ = writeln!(
out,
"//! Source: `tm-themes@{version}`, vendored under `assets/themes/`; provenance\n\
//! and licences in `assets/themes/SOURCE.md` and `NOTICE-THEMES.md`.\n//!\n\
//! {} themes ship. The rest were held back:\n//!",
shipped.len()
);
for entry in excluded {
let _ = writeln!(out, "//! - `{}`: {}", entry.metadata.name, entry.reason);
}
out.push('\n');
}
fn catalog_constant(out: &mut String, shipped: &[Shipped<'_>]) {
out.push_str(
"/// Every theme extracted from the vendored set, alphabetically by identifier.\n\
///\n\
/// The first-party theme is not here: `lib.rs` puts it first, because decision\n\
/// 0018 requires the default theme to keep reproducing the appearance it\n\
/// replaced, and that theme carries overrides no generated theme may carry.\n\
pub const CATALOG: &[ThemeAnchors] = &[\n",
);
for theme in shipped {
theme_entry(out, theme);
}
out.push_str("];\n\n");
}
fn theme_entry(out: &mut String, theme: &Shipped<'_>) {
let meta = theme.metadata;
let _ = writeln!(
out,
" // {} — {}, {}.",
meta.display_name,
meta.spdx,
upstream_slug(&meta.source)
);
match theme.extracted.comment_source {
CommentSource::ExactScope => {}
CommentSource::CommaSeparatedScope => {
out.push_str(
" // comment: the scope list is one comma-separated string, which the\n\
\x20 // reference rule reads past; split it and the theme's own comment\n\
\x20 // colour is there.\n",
);
}
CommentSource::DerivedFromForeground => {
out.push_str(
" // comment: the theme has no comment colour distinct from its\n\
\x20 // foreground, so secondary text is mixed 40% toward the background\n\
\x20 // rather than left identical to body text.\n",
);
}
}
for skipped in &theme.extracted.skipped_git_keys {
let _ = writeln!(
out,
" // {}: `{}` is {}, which would make two role colours identical;\n\
\x20 // read the theme's next declaration for this role instead.",
skipped.role, skipped.key, skipped.collided_with
);
}
for substitution in &theme.extracted.substitutions {
let _ = writeln!(
out,
" // ansi {:?} was {:06x}, the same as {:?}; substituted the derived hue {:06x}\n\
\x20 // so the two stay distinguishable.",
substitution.slot,
substitution.original.to_hex(),
substitution.collided_with,
substitution.replacement.to_hex()
);
}
let extracted = theme.extracted;
let _ = writeln!(out, " ThemeAnchors {{");
let _ = writeln!(out, " name: {:?},", meta.name);
let _ = writeln!(out, " display_name: {:?},", meta.display_name);
let _ = writeln!(out, " bg: {},", hex(extracted.bg));
let _ = writeln!(out, " fg: {},", hex(extracted.fg));
let _ = writeln!(out, " comment: {},", hex(extracted.comment));
let _ = writeln!(out, " added: {},", optional(extracted.added));
let _ = writeln!(out, " deleted: {},", optional(extracted.deleted));
let _ = writeln!(out, " modified: {},", optional(extracted.modified));
terminal_literal(out, theme);
let _ = writeln!(out, " overrides: RoleOverrides::NONE,");
let _ = writeln!(out, " }},");
}
fn terminal_literal(out: &mut String, theme: &Shipped<'_>) {
let terminal = &theme.extracted.terminal;
let _ = writeln!(out, " terminal: TerminalPalette {{");
let _ = writeln!(out, " background: {},", hex(terminal.background));
let _ = writeln!(out, " foreground: {},", hex(terminal.foreground));
let _ = writeln!(out, " cursor: {},", hex(terminal.cursor));
let _ = writeln!(
out,
" cursor_text: {},",
hex(terminal.cursor_text)
);
let _ = writeln!(out, " ansi: [");
for slot in AnsiColor::ALL {
let _ = writeln!(
out,
" {}, // {:?}",
hex(terminal.color(slot)),
slot
);
}
let _ = writeln!(out, " ],");
let _ = writeln!(out, " }},");
}
fn secondary_text_constant(out: &mut String, shipped: &[Shipped<'_>]) {
out.push_str(
"/// Themes whose comment colour measures below 3:1 against their own surface,\n\
/// with the ratio each one measures.\n\
///\n\
/// These are not defects in the extraction — they are what the theme's author\n\
/// chose, and a comment is meant to recede. Lumbridge paints secondary text in\n\
/// that colour, so on these themes secondary text is quieter than WCAG's\n\
/// large-text floor, and this is the list that says so out loud. The catalog\n\
/// test reads it as an allowlist and fails on a theme that is below the floor\n\
/// without being named here, and on a name here that is no longer below it, so\n\
/// the list cannot grow or rot unnoticed.\n\
pub const SECONDARY_TEXT_BELOW_3_TO_1: &[(&str, f64)] = &[\n",
);
for theme in shipped {
if theme.extracted.secondary_contrast < 3.0 {
let _ = writeln!(
out,
" ({:?}, {:.3}),",
theme.metadata.name, theme.extracted.secondary_contrast
);
}
}
out.push_str("];\n");
}
fn hex(color: Srgb) -> String {
format!("Srgb::from_hex(0x{:06x})", color.to_hex())
}
fn optional(color: Option<Srgb>) -> String {
color.map_or_else(
|| "None".to_owned(),
|value| format!("Some({})", hex(value)),
)
}
/// `owner/repo` out of a GitHub blob URL, for the one-line credit above a theme.
fn upstream_slug(source: &str) -> String {
source.strip_prefix("https://github.com/").map_or_else(
|| source.to_owned(),
|rest| rest.split('/').take(2).collect::<Vec<_>>().join("/"),
)
}
/// The attribution table.
#[must_use]
pub fn notice(shipped: &[Shipped<'_>], excluded: &[Excluded<'_>], version: &str) -> String {
let mut out = String::new();
out.push_str("# Theme attribution\n\n");
let _ = writeln!(
out,
"Generated by `tools/theme-gen` from `assets/themes/metadata.json`. Every theme\n\
below is adapted from an upstream TextMate/VS Code theme, redistributed here\n\
under its own licence and vendored from `tm-themes@{version}`. Lumbridge stores\n\
five anchor colours and sixteen terminal colours per theme; the licence covers\n\
the theme file those were read from, which is checked in beside this table.\n"
);
out.push_str("\n## Shipped\n\n");
out.push_str("| Theme | Identifier | Upstream | SPDX | Licence text |\n");
out.push_str("|---|---|---|---|---|\n");
for theme in shipped {
let meta = theme.metadata;
let _ = writeln!(
out,
"| {} | `{}` | [{}]({}) | [`{}`]({}) | [`{}`]({}) |",
meta.display_name,
meta.name,
upstream_slug(&meta.source),
meta.source,
meta.spdx,
meta.license_url,
meta.license_file,
meta.license_file
);
}
out.push_str("\n## Held back\n\n");
out.push_str("Vendored so the decision can be re-measured, but absent from the catalog.\n\n");
out.push_str("| Theme | Identifier | SPDX | Why |\n|---|---|---|---|\n");
for entry in excluded {
let _ = writeln!(
out,
"| {} | `{}` | `{}` | {} |",
entry.metadata.display_name, entry.metadata.name, entry.metadata.spdx, entry.reason
);
}
out.push_str(
"\nOne theme is not vendored at all: `aurora-x` is GPL-3.0, and the research\n\
boundary in `AGENTS.md` forbids shipping GPL-derived assets from an Apache-2.0\n\
repository. It is excluded at the vendoring step, so no copy of it exists here.\n",
);
out
}
+718
View File
@@ -0,0 +1,718 @@
//! Turning a `TextMate` theme document into the five anchors and sixteen ANSI
//! colours the catalog stores.
//!
//! The extraction rules are Buzz's, read out of
//! `desktop/src/shared/theme/theme-loader.ts` and
//! `desktop/src/shared/theme/terminal-palette.ts` (block/buzz, Apache-2.0,
//! Copyright 2026 Block, Inc.) and reimplemented here so that Lumbridge's
//! catalog contains the same colours a Buzz user would see. No implementation
//! code was copied; `tools/theme-gen/reference/` holds a script that runs the
//! originals under Node and writes the vectors the Rust side is tested against,
//! which is what keeps the two honest. See decision 0018.
//!
//! Three details in those rules are easy to get wrong and are load-bearing:
//!
//! - The comment scope match is **exact**, and upstream compares against array
//! elements only, so an entry whose scope is the single string
//! `"comment, punctuation.definition.comment"` does not match it. Four themes
//! spell it that way and upstream reads their comment colour as their
//! foreground. Lumbridge runs upstream's rule first and only widens when it
//! comes back with nothing usable — see [`CommentSource`].
//! - The terminal extractor's hex parser is **stricter** than the anchor
//! extractor's: it wants a leading `#` and 3, 6, or 8 digits, and rejects the
//! 4-digit form the anchor parser accepts. A theme spelling a terminal colour
//! any other way falls through to the derived hue rather than being coerced.
//! - An absent colour is absent. Where upstream has no value it derives one
//! from colours the theme does carry; it never substitutes a stock palette.
use lumbridge_theme::color::{contrast_ratio, mix, relative_luminance};
use lumbridge_theme::terminal::{AnsiColor, TerminalPalette};
use lumbridge_theme::{Srgb, ThemeAnchors, derive};
use serde_json::Value;
/// The six chromatic hues used when a theme names neither a terminal colour nor
/// a syntax token that could stand in for one. Upstream's table, kept because a
/// different one would silently repaint every theme that relies on it.
const FALLBACK_HUES: [(AnsiColor, u32); 6] = [
(AnsiColor::Red, 0xd75f5f),
(AnsiColor::Green, 0x5faf87),
(AnsiColor::Yellow, 0xd7af5f),
(AnsiColor::Blue, 0x5f87d7),
(AnsiColor::Magenta, 0xaf87d7),
(AnsiColor::Cyan, 0x5fafd7),
];
/// Which syntax scopes stand in for which terminal hue. Upstream's table.
const SCOPE_ANCHORS: [(AnsiColor, &[&str]); 6] = [
(AnsiColor::Red, &["invalid", "keyword", "deleted"]),
(AnsiColor::Green, &["string", "inserted", "tag"]),
(AnsiColor::Yellow, &["type", "class", "escape"]),
(AnsiColor::Blue, &["function", "call", "method"]),
(AnsiColor::Magenta, &["numeric", "constant", "operator"]),
(AnsiColor::Cyan, &["attribute", "parameter", "property"]),
];
/// How much of the foreground is left in a comment colour derived for a theme
/// that does not usefully declare one. Comments recede; this is how far.
const DERIVED_COMMENT_MIX: f64 = 0.40;
/// Where a theme's secondary-text colour came from.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum CommentSource {
/// A `tokenColors` entry whose scope array holds the exact string
/// `comment`. Upstream's rule, and by far the common case.
ExactScope,
/// The same rule, applied to an entry that spells its scope list as one
/// comma-separated string rather than as an array.
///
/// `TextMate` allows both spellings and several themes use the second one.
/// Upstream compares against array elements only and so reads straight past
/// them, which leaves the comment colour equal to the foreground — and a
/// theme whose secondary text is identical to its body text has lost a
/// distinction the interface depends on. Splitting the string first reads
/// the colour the theme's author actually wrote down.
CommaSeparatedScope,
/// The theme declares no comment colour distinguishable from its
/// foreground, so one is mixed from the theme's own foreground and
/// background. Recorded per theme in the generated catalog.
DerivedFromForeground,
}
/// A git-decoration key that was passed over because its colour was already in
/// use for something else.
pub struct SkippedGitKey {
/// `added`, `deleted`, or `modified`.
pub role: &'static str,
/// The theme key that was skipped.
pub key: &'static str,
/// What already had that colour.
pub collided_with: &'static str,
}
/// A base hue that had to be replaced because it repeated an earlier one.
///
/// Recorded rather than applied silently: a reader looking at a theme whose
/// blue is not the blue upstream shipped deserves to see why in the same file.
pub struct Substitution {
pub slot: AnsiColor,
pub collided_with: AnsiColor,
pub original: Srgb,
pub replacement: Srgb,
}
/// Everything the catalog needs about one theme, plus the measurements the
/// generator gates on.
pub struct Extracted {
pub bg: Srgb,
pub fg: Srgb,
pub comment: Srgb,
pub comment_source: CommentSource,
pub added: Option<Srgb>,
pub deleted: Option<Srgb>,
pub modified: Option<Srgb>,
pub terminal: TerminalPalette,
pub substitutions: Vec<Substitution>,
/// Every ANSI slot whose colour is not the one the extraction rules alone
/// would produce: the base hues that were forced apart, plus the bright
/// slots that are derived from them and therefore moved with them.
pub diverging_ansi_slots: Vec<usize>,
pub skipped_git_keys: Vec<SkippedGitKey>,
/// Body text against the surface the interface will actually paint, which
/// is the theme's background except on the near-black themes where the
/// derivation lifts the surface instead of sinking the frame.
pub text_contrast: f64,
/// The comment colour against that same surface.
pub secondary_contrast: f64,
}
/// Reads one theme document.
#[must_use]
pub fn extract(document: &Value) -> Extracted {
let colors = document.get("colors");
let bg = anchor(colors, "editor.background", 0x1e_1e1e);
let fg = anchor(colors, "editor.foreground", 0xd4_d4d4);
let tokens = token_entries(document);
let (comment, comment_source) = comment_anchor(tokens, bg, fg);
let (terminal, substitutions, diverging_ansi_slots) = terminal_palette(document, tokens);
let ([added, deleted, modified], skipped_git_keys) = git_colors(colors, fg, comment);
let mut extracted = Extracted {
bg,
fg,
comment,
comment_source,
added,
deleted,
modified,
terminal,
substitutions,
diverging_ansi_slots,
skipped_git_keys,
text_contrast: 0.0,
secondary_contrast: 0.0,
};
let surface = surface_of(&extracted);
extracted.text_contrast = contrast_ratio(fg, surface);
extracted.secondary_contrast = contrast_ratio(comment, surface);
extracted
}
/// The work surface the interface derives for this theme.
fn surface_of(extracted: &Extracted) -> Srgb {
let anchors = ThemeAnchors {
name: "",
display_name: "",
bg: extracted.bg,
fg: extracted.fg,
comment: extracted.comment,
added: extracted.added,
deleted: extracted.deleted,
modified: extracted.modified,
terminal: extracted.terminal,
overrides: lumbridge_theme::RoleOverrides::default(),
};
derive(&anchors, extracted.fg).surface
}
const GIT_ADDED_KEYS: &[&str] = &[
"gitDecoration.addedResourceForeground",
"editorGutter.addedBackground",
"diffEditor.insertedTextBackground",
];
const GIT_DELETED_KEYS: &[&str] = &[
"gitDecoration.deletedResourceForeground",
"editorGutter.deletedBackground",
"diffEditor.removedTextBackground",
];
const GIT_MODIFIED_KEYS: &[&str] = &[
"gitDecoration.modifiedResourceForeground",
"editorGutter.modifiedBackground",
];
/// The three git-decoration colours, each the first the theme declares that is
/// not a colour some other role has already taken.
///
/// Two themes force this. Gruvbox sets `gitDecoration.addedResourceForeground`
/// to its editor foreground — a fine choice in a file tree, where "added" files
/// are simply not dimmed, but useless as a *colour*: it would make a
/// provider-reported usage number and a locally computed one the same shade.
/// `LaserWave` gives "added" and "modified" the same mint green. Decision 0012
/// needs these apart, so a key whose colour is already spoken for is passed over
/// and the theme's next declaration for the same role is read instead — which on
/// every theme here is the colour it paints that role in the gutter. Reading
/// further, not inventing.
fn git_colors(
colors: Option<&Value>,
foreground: Srgb,
comment: Srgb,
) -> ([Option<Srgb>; 3], Vec<SkippedGitKey>) {
let mut taken = vec![
(foreground, "the foreground colour"),
(comment, "the comment colour"),
];
let mut skipped = Vec::new();
let mut chosen = [None; 3];
let roles: [(&'static str, &[&'static str]); 3] = [
("added", GIT_ADDED_KEYS),
("deleted", GIT_DELETED_KEYS),
("modified", GIT_MODIFIED_KEYS),
];
for (index, (role, keys)) in roles.into_iter().enumerate() {
for key in keys {
let Some(color) = color_string(colors, key).and_then(loose_hex) else {
continue;
};
if let Some((_, owner)) = taken.iter().find(|(other, _)| *other == color) {
skipped.push(SkippedGitKey {
role,
key,
collided_with: owner,
});
continue;
}
taken.push((color, role));
chosen[index] = Some(color);
break;
}
}
(chosen, skipped)
}
/// `editor.background` and friends, with upstream's fallback when absent.
fn anchor(colors: Option<&Value>, key: &str, fallback: u32) -> Srgb {
color_string(colors, key)
.and_then(loose_hex)
.unwrap_or_else(|| Srgb::from_hex(fallback))
}
fn color_string<'a>(colors: Option<&'a Value>, key: &str) -> Option<&'a str> {
colors?
.get(key)
.and_then(Value::as_str)
.filter(|value| !value.is_empty())
}
/// The anchor parser: 3, 4, 6, or 8 hex digits, `#` optional, alpha discarded.
fn loose_hex(text: &str) -> Option<Srgb> {
Srgb::parse(text)
}
/// The terminal parser, which is deliberately stricter — see the module note.
fn strict_hex(text: &str) -> Option<Srgb> {
let trimmed = text.trim();
let digits = trimmed.strip_prefix('#')?;
if !matches!(digits.len(), 3 | 6 | 8) {
return None;
}
Srgb::parse(digits)
}
fn token_entries(document: &Value) -> &[Value] {
document
.get("tokenColors")
.or_else(|| document.get("settings"))
.and_then(Value::as_array)
.map_or(&[], Vec::as_slice)
}
/// The theme's secondary-text colour, and where it came from.
///
/// Upstream's rule runs first and unchanged. Only when it produces nothing the
/// interface can use — either no match at all, or a match that is the
/// foreground colour over again — do the two fallbacks run, and each one is
/// recorded so the catalog says which themes took them. A theme whose secondary
/// text is the same colour as its body text would make decision 0012's
/// provenance colouring unreadable, because "local" and "unavailable" are those
/// two roles.
fn comment_anchor(tokens: &[Value], bg: Srgb, fg: Srgb) -> (Srgb, CommentSource) {
if let Some(color) = comment_color(tokens, false).filter(|color| *color != fg) {
return (color, CommentSource::ExactScope);
}
if let Some(color) = comment_color(tokens, true).filter(|color| *color != fg) {
return (color, CommentSource::CommaSeparatedScope);
}
(
mix(fg, bg, DERIVED_COMMENT_MIX),
CommentSource::DerivedFromForeground,
)
}
/// The first token entry that both names the exact scope `comment` and sets a
/// foreground. An entry that names the scope but sets no foreground is skipped
/// rather than ending the search, which is upstream's behaviour and matters for
/// themes that style comments in two passes.
fn comment_color(tokens: &[Value], split_commas: bool) -> Option<Srgb> {
tokens.iter().find_map(|entry| {
let foreground = entry.get("settings")?.get("foreground")?.as_str()?;
if scopes_of(entry, split_commas).any(|scope| scope == "comment") {
loose_hex(foreground)
} else {
None
}
})
}
/// A token entry's scopes, whether it spelled them as a string or an array.
///
/// `split_commas` decides whether a single string holding a comma-separated
/// list counts as several scopes or as one. Upstream treats it as one; see
/// [`CommentSource::CommaSeparatedScope`]. It makes no difference to the
/// terminal-hue search, which splits on punctuation again afterwards.
fn scopes_of(entry: &Value, split_commas: bool) -> impl Iterator<Item = &str> {
let scope = entry.get("scope");
let single = scope
.and_then(Value::as_str)
.map_or(Vec::new(), |text| {
if split_commas {
text.split(',').map(str::trim).collect()
} else {
vec![text]
}
})
.into_iter();
let many = scope
.and_then(Value::as_array)
.map_or(&[][..], Vec::as_slice)
.iter()
.filter_map(Value::as_str);
single.chain(many)
}
/// The sixteen terminal colours, and the base hues that had to be moved.
///
/// The order of the three passes matters. Bases are read, *then* forced apart,
/// *then* the bright half is lifted from them — because a bright slot the theme
/// does not declare is derived from its base, and deriving it before the base
/// moved would leave bright cyan sitting on bright blue after the dim pair had
/// already been separated.
fn terminal_palette(
document: &Value,
tokens: &[Value],
) -> (TerminalPalette, Vec<Substitution>, Vec<usize>) {
let colors = document.get("colors");
// The terminal extractor re-reads the background and foreground through its
// own stricter parser, so a theme whose `editor.background` is spelled in
// the 4-digit form gets the stock terminal background while the interface
// still uses the parsed one. Reproduced rather than tidied: tidying it would
// change colours users already have.
let background = color_string(colors, "editor.background")
.and_then(strict_hex)
.unwrap_or_else(|| Srgb::from_hex(0x1e_1e1e));
let foreground = color_string(colors, "editor.foreground")
.and_then(strict_hex)
.unwrap_or_else(|| Srgb::from_hex(0xd4_d4d4));
let mut ansi = [background; 16];
ansi[AnsiColor::Black as usize] =
ansi_key(colors, AnsiColor::Black).unwrap_or_else(|| mix(background, foreground, 0.16));
ansi[AnsiColor::White as usize] =
ansi_key(colors, AnsiColor::White).unwrap_or_else(|| mix(background, foreground, 0.82));
for (slot, anchors) in SCOPE_ANCHORS {
let anchored = scope_color(tokens, anchors).and_then(strict_hex);
ansi[slot as usize] = ansi_key(colors, slot)
.or(anchored)
.unwrap_or_else(|| derived_hue(slot, background, foreground));
}
let substitutions = decollide(&mut ansi, background, foreground);
let mut diverging: Vec<usize> = substitutions
.iter()
.map(|substitution| substitution.slot as usize)
.collect();
let toward = if relative_luminance(background) > relative_luminance(foreground) {
Srgb::BLACK
} else {
Srgb::WHITE
};
for index in 8..16 {
let slot = AnsiColor::ALL[index];
if let Some(declared) = ansi_key(colors, slot) {
ansi[index] = declared;
continue;
}
ansi[index] = mix(ansi[index - 8], toward, 0.28);
if diverging.contains(&(index - 8)) {
diverging.push(index);
}
}
let palette = TerminalPalette {
background,
foreground,
cursor: color_string(colors, "terminalCursor.foreground")
.and_then(strict_hex)
.unwrap_or(foreground),
cursor_text: color_string(colors, "terminalCursor.background")
.and_then(strict_hex)
.unwrap_or(background),
ansi,
};
(palette, substitutions, diverging)
}
fn ansi_key(colors: Option<&Value>, slot: AnsiColor) -> Option<Srgb> {
let key = format!("terminal.ansi{}", slot.key_suffix());
color_string(colors, &key).and_then(strict_hex)
}
/// The hue a slot falls back to: the stock hue pulled 18% toward the theme's own
/// foreground, so it sits in the theme's light rather than beside it.
fn derived_hue(slot: AnsiColor, background: Srgb, foreground: Srgb) -> Srgb {
match slot {
AnsiColor::Black => mix(background, foreground, 0.16),
AnsiColor::White => mix(background, foreground, 0.82),
_ => {
let hue = FALLBACK_HUES
.iter()
.find(|(name, _)| *name == slot)
.map_or(0x80_8080, |(_, hue)| *hue);
mix(Srgb::from_hex(hue), foreground, 0.18)
}
}
}
/// The first token colour whose scope mentions one of `anchors`.
fn scope_color<'a>(tokens: &'a [Value], anchors: &[&str]) -> Option<&'a str> {
tokens.iter().find_map(|entry| {
let foreground = entry.get("settings")?.get("foreground")?.as_str()?;
let matched = scopes_of(entry, false).any(|scope| {
scope
.to_ascii_lowercase()
.split(|character: char| {
character == '.' || character == ',' || character.is_whitespace()
})
.any(|part| anchors.iter().any(|anchor| part.contains(anchor)))
});
matched.then_some(foreground)
})
}
/// Forces the eight base hues apart.
///
/// Themes collide here for two different reasons and both are handled the same
/// way: some derive two slots from one syntax token by accident, and at least
/// one (poimandres) ships `terminal.ansiBlue` and `terminal.ansiCyan` set to the
/// same value on purpose. Upstream's own choice loses either way, because a
/// terminal that cannot tell blue from cyan has dropped information the running
/// program encoded; what upstream chose is written into the catalog as a comment
/// so the trade is visible.
fn decollide(ansi: &mut [Srgb; 16], background: Srgb, foreground: Srgb) -> Vec<Substitution> {
let mut substitutions = Vec::new();
for index in 0..8 {
let slot = AnsiColor::ALL[index];
let Some(earlier) = ansi[..index].iter().position(|hue| *hue == ansi[index]) else {
continue;
};
let original = ansi[index];
let replacement = separate(&ansi[..index], slot, background, foreground);
ansi[index] = replacement;
substitutions.push(Substitution {
slot,
collided_with: AnsiColor::ALL[earlier],
original,
replacement,
});
}
substitutions
}
/// The nearest distinct stand-in for a colliding slot.
///
/// The derived hue is tried first. If that also repeats an earlier slot the hue
/// is walked toward the foreground in fixed steps, which keeps the result inside
/// the theme's own range instead of reaching for an unrelated colour. The walk
/// is bounded; running out returns the last candidate and the caller's assertion
/// catches it, because shipping a duplicate quietly is the failure this whole
/// routine exists to prevent.
fn separate(taken: &[Srgb], slot: AnsiColor, background: Srgb, foreground: Srgb) -> Srgb {
let base = derived_hue(slot, background, foreground);
let mut candidate = base;
for step in 0..=5 {
candidate = mix(base, foreground, f64::from(step) * 0.06);
if !taken.contains(&candidate) {
return candidate;
}
}
candidate
}
#[cfg(test)]
mod tests {
use super::{CommentSource, extract};
use lumbridge_theme::Srgb;
use lumbridge_theme::terminal::AnsiColor;
use serde_json::{Value, json};
/// A theme document with the colours a test cares about and nothing else.
fn document(colors: &Value, tokens: &Value) -> Value {
json!({ "colors": colors, "tokenColors": tokens })
}
fn comment_token(scope: &Value, foreground: &str) -> Value {
json!({ "scope": scope, "settings": { "foreground": foreground } })
}
#[test]
fn a_theme_with_no_colours_at_all_falls_back_to_the_stock_pair() {
let extracted = extract(&json!({}));
assert_eq!(
extracted.bg.to_hex(),
0x1e1e1e,
"an absent editor.background must use the documented stock value"
);
assert_eq!(extracted.fg.to_hex(), 0xd4d4d4);
assert_eq!(
extracted.added, None,
"a theme that declares no git colours must report none, not a guess"
);
}
#[test]
fn the_comment_colour_comes_from_an_exact_scope_match() {
let extracted = extract(&document(
&json!({ "editor.background": "#101010", "editor.foreground": "#eeeeee" }),
&json!([
comment_token(&json!("comment.line"), "#111111"),
comment_token(&json!(["punctuation", "comment"]), "#777777"),
]),
));
assert_eq!(
extracted.comment.to_hex(),
0x777777,
"`comment.line` is not the exact scope `comment` and must not match"
);
assert_eq!(extracted.comment_source, CommentSource::ExactScope);
}
#[test]
fn a_comma_separated_scope_list_is_read_only_when_the_exact_rule_finds_nothing() {
let extracted = extract(&document(
&json!({ "editor.background": "#101010", "editor.foreground": "#eeeeee" }),
&json!([comment_token(
&json!("comment, punctuation.definition.comment"),
"#858585"
)]),
));
assert_eq!(
extracted.comment.to_hex(),
0x858585,
"a scope list spelled as one string still names the comment scope"
);
assert_eq!(
extracted.comment_source,
CommentSource::CommaSeparatedScope,
"reading past upstream's rule must be recorded, not silent"
);
}
#[test]
fn a_comment_colour_equal_to_the_foreground_is_replaced_rather_than_kept() {
let extracted = extract(&document(
&json!({ "editor.background": "#000000", "editor.foreground": "#ffffff" }),
&json!([comment_token(&json!(["comment"]), "#ffffffaa")]),
));
assert_ne!(
extracted.comment, extracted.fg,
"secondary text identical to body text would collapse two provenance colours"
);
assert_eq!(
extracted.comment_source,
CommentSource::DerivedFromForeground
);
assert_eq!(
extracted.comment.to_hex(),
0x999999,
"40% toward the background"
);
}
#[test]
fn a_git_key_holding_the_foreground_colour_is_passed_over_for_the_next_one() {
let extracted = extract(&document(
&json!({
"editor.background": "#101010",
"editor.foreground": "#ebdbb2",
"gitDecoration.addedResourceForeground": "#ebdbb2",
"editorGutter.addedBackground": "#b8bb26",
}),
&json!([comment_token(&json!(["comment"]), "#928374")]),
));
assert_eq!(
extracted.added.map(Srgb::to_hex),
Some(0xb8bb26),
"the first key is the foreground colour and carries no information as a role"
);
let skipped = &extracted.skipped_git_keys;
assert_eq!(skipped.len(), 1, "the skip must be recorded");
assert_eq!(skipped[0].role, "added");
assert_eq!(skipped[0].key, "gitDecoration.addedResourceForeground");
}
#[test]
fn two_git_roles_may_not_end_up_the_same_colour() {
let extracted = extract(&document(
&json!({
"editor.background": "#101010",
"editor.foreground": "#ffffff",
"editorGutter.addedBackground": "#74dfc4",
"gitDecoration.modifiedResourceForeground": "#74dfc4",
"editorGutter.modifiedBackground": "#40b4c4",
}),
&json!([comment_token(&json!(["comment"]), "#888888")]),
));
assert_eq!(extracted.added.map(Srgb::to_hex), Some(0x74dfc4));
assert_eq!(
extracted.modified.map(Srgb::to_hex),
Some(0x40b4c4),
"a modified colour equal to the added colour must read the next key"
);
}
#[test]
fn a_terminal_colour_spelled_in_a_form_the_upstream_parser_rejects_is_not_coerced() {
let extracted = extract(&document(
&json!({
"editor.background": "#101010",
"editor.foreground": "#eeeeee",
// Four digits: valid to the anchor parser, rejected by the
// terminal parser, so this must not become the red slot.
"terminal.ansiRed": "#f00f",
"terminal.ansiGreen": "#00ff00",
}),
&json!([comment_token(&json!(["comment"]), "#888888")]),
));
assert_eq!(
extracted.terminal.color(AnsiColor::Green).to_hex(),
0x00ff00,
"a six-digit terminal colour is read as written"
);
assert_ne!(
extracted.terminal.color(AnsiColor::Red).to_hex(),
0xff0000,
"a four-digit terminal colour must fall through rather than be coerced"
);
}
#[test]
fn two_base_slots_declared_with_the_same_colour_are_forced_apart() {
let extracted = extract(&document(
&json!({
"editor.background": "#101010",
"editor.foreground": "#eeeeee",
"terminal.ansiBlue": "#89ddff",
"terminal.ansiCyan": "#89ddff",
}),
&json!([comment_token(&json!(["comment"]), "#888888")]),
));
let blue = extracted.terminal.color(AnsiColor::Blue);
let cyan = extracted.terminal.color(AnsiColor::Cyan);
assert_eq!(
blue.to_hex(),
0x89ddff,
"the earlier slot keeps what it asked for"
);
assert_ne!(
cyan, blue,
"a terminal that cannot tell blue from cyan has dropped information"
);
let recorded = &extracted.substitutions;
assert_eq!(recorded.len(), 1, "the substitution must be recorded");
assert_eq!(recorded[0].slot, AnsiColor::Cyan);
assert_eq!(recorded[0].collided_with, AnsiColor::Blue);
assert_eq!(recorded[0].original.to_hex(), 0x89ddff);
}
#[test]
fn a_bright_slot_is_lifted_from_the_base_slot_that_replaced_a_collision() {
let extracted = extract(&document(
&json!({
"editor.background": "#101010",
"editor.foreground": "#eeeeee",
"terminal.ansiBlue": "#89ddff",
"terminal.ansiCyan": "#89ddff",
}),
&json!([comment_token(&json!(["comment"]), "#888888")]),
));
assert_ne!(
extracted.terminal.color(AnsiColor::BrightCyan),
extracted.terminal.color(AnsiColor::BrightBlue),
"a bright slot derived from a substituted base must follow the substitution"
);
}
#[test]
fn contrast_is_measured_against_the_surface_the_interface_will_paint() {
// Pitch black leaves no room below it, so the derivation lifts the
// surface instead of sinking the frame; the measurement has to follow.
let extracted = extract(&document(
&json!({ "editor.background": "#000000", "editor.foreground": "#ffffff" }),
&json!([comment_token(&json!(["comment"]), "#777777")]),
));
assert!(
extracted.text_contrast < 21.0,
"white on a lifted surface is not the 21:1 of white on pure black"
);
assert!(extracted.text_contrast > 15.0);
}
}
+200
View File
@@ -0,0 +1,200 @@
//! Generates `lumbridge-theme`'s catalog from the vendored `TextMate` themes.
//!
//! Run it by hand, review the diff, commit the result:
//!
//! ```text
//! cargo run -p theme-gen # rewrite the generated files
//! cargo run -p theme-gen -- --check # fail if they are out of date
//! ```
//!
//! It is deliberately **not** a `build.rs`. Two reasons, in order of weight.
//! The colours a user sees should be reviewable as text in a pull request, and
//! a colour that moved because an upstream theme moved should show up as a diff
//! rather than as a silently different build. And a build that reads
//! `assets/themes/` makes the asset tree a build input, which means a packaging
//! step that trims assets breaks compilation instead of breaking a picker.
//!
//! Two gates decide what ships, and both are measurements rather than opinions:
//!
//! - **Licence.** GPL-licensed themes are excluded at the vendoring step, not
//! here, so no copy of one is in the tree at all. `AGENTS.md` forbids shipping
//! GPL-derived assets from this Apache-2.0 repository.
//! - **Contrast.** A theme whose body text measures below 4.5:1 against the
//! surface Lumbridge will paint under it is held back. That is WCAG AA for
//! body text, and the interface uses `editor.foreground` for real reading
//! rather than for syntax accents, so a theme that fails it is unusable here
//! even where it is fine in an editor that only ever puts code on that
//! background.
// The generator is almost entirely colour hex, and `0x00d7_5f5f` is harder to
// check against a theme file than `0xd75f5f`, not easier. Same reasoning, and
// same wording, as the crate it writes for.
#![allow(
clippy::unreadable_literal,
reason = "six-digit colour hex reads whole"
)]
mod emit;
mod extract;
mod source;
use std::fs;
use std::io::{self, Write as _};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode, Stdio};
use crate::emit::{Excluded, Shipped};
/// The pinned upstream package, echoed into both generated files.
const TM_THEMES_VERSION: &str = "1.12.10";
/// WCAG AA for body text.
const MINIMUM_TEXT_CONTRAST: f64 = 4.5;
fn main() -> ExitCode {
let check_only = std::env::args().any(|argument| argument == "--check");
match run(&source::repository_root(), check_only) {
Ok(Outcome::UpToDate | Outcome::Written) => ExitCode::SUCCESS,
Ok(Outcome::Stale(paths)) => {
for path in paths {
eprintln!("out of date: {}", path.display());
}
eprintln!("run `cargo run -p theme-gen` and commit the result");
ExitCode::FAILURE
}
Err(error) => {
eprintln!("theme-gen: {error}");
ExitCode::FAILURE
}
}
}
/// What a run did, so `--check` and a normal run can share one code path.
enum Outcome {
Written,
UpToDate,
Stale(Vec<PathBuf>),
}
/// Reads the vendored themes and writes (or checks) the generated files.
///
/// # Errors
///
/// Fails if the vendored set cannot be read, if `rustfmt` cannot be run over the
/// generated Rust, or if a generated file cannot be written. It also fails if a
/// theme's terminal palette still has two identical base hues after
/// substitution, which would mean the de-collision walk ran out of room; that is
/// a bug in the walk rather than a fact about the theme, and shipping the
/// duplicate would defeat the guarantee the type documents.
fn run(root: &Path, check_only: bool) -> io::Result<Outcome> {
let assets = root.join("assets/themes");
let themes = source::load(&assets)?;
let extracted: Vec<_> = themes
.iter()
.map(|theme| (&theme.metadata, extract::extract(&theme.document)))
.collect();
let mut shipped = Vec::new();
let mut excluded = Vec::new();
for (metadata, item) in &extracted {
if item.text_contrast < MINIMUM_TEXT_CONTRAST {
excluded.push(Excluded {
metadata,
reason: format!(
"body text measures {:.2}:1 against its own surface, below the {MINIMUM_TEXT_CONTRAST}:1 floor",
item.text_contrast
),
});
continue;
}
assert_distinct_hues(metadata.name.as_str(), item)?;
shipped.push(Shipped {
metadata,
extracted: item,
});
}
let catalog_path = root.join("crates/lumbridge-theme/src/generated/catalog.rs");
let notice_path = assets.join("NOTICE-THEMES.md");
let catalog = rustfmt(&emit::catalog(&shipped, &excluded, TM_THEMES_VERSION))?;
let notice = emit::notice(&shipped, &excluded, TM_THEMES_VERSION);
if check_only {
let stale: Vec<_> = [(catalog_path, catalog), (notice_path, notice)]
.into_iter()
.filter(|(path, wanted)| fs::read_to_string(path).ok().as_ref() != Some(wanted))
.map(|(path, _)| path)
.collect();
return Ok(if stale.is_empty() {
Outcome::UpToDate
} else {
Outcome::Stale(stale)
});
}
if let Some(parent) = catalog_path.parent() {
fs::create_dir_all(parent)?;
}
fs::write(&catalog_path, catalog)?;
fs::write(&notice_path, notice)?;
println!(
"wrote {} themes ({} held back) to {}",
shipped.len(),
excluded.len(),
catalog_path.display()
);
Ok(Outcome::Written)
}
/// The guarantee `TerminalPalette` documents, checked where it is established.
fn assert_distinct_hues(name: &str, extracted: &extract::Extracted) -> io::Result<()> {
let hues = extracted.terminal.base_hues();
for (index, first) in hues.iter().enumerate() {
if hues[index + 1..].contains(first) {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!(
"{name}: base ANSI hue {:06x} still repeats after substitution",
first.to_hex()
),
));
}
}
Ok(())
}
/// Formats generated Rust with the toolchain's own `rustfmt`.
///
/// Not cosmetic: `--check` compares the generated text against the checked-in
/// file, and the checked-in file has been through `cargo fmt`. Emitting
/// unformatted code would make the two disagree forever.
fn rustfmt(source: &str) -> io::Result<String> {
let mut child = Command::new("rustfmt")
.arg("--edition=2024")
.arg("--emit=stdout")
.arg("--quiet")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.map_err(|error| {
io::Error::new(
error.kind(),
format!("could not run rustfmt ({error}); install it with `rustup component add rustfmt`"),
)
})?;
child
.stdin
.take()
.ok_or_else(|| io::Error::other("rustfmt stdin was not piped"))?
.write_all(source.as_bytes())?;
let output = child.wait_with_output()?;
if !output.status.success() {
return Err(io::Error::other(format!(
"rustfmt rejected the generated catalog ({})",
output.status
)));
}
String::from_utf8(output.stdout)
.map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))
}
+106
View File
@@ -0,0 +1,106 @@
//! Reading the vendored theme set off disk.
//!
//! Everything this module returns comes from `assets/themes/`, which is a
//! byte-for-byte copy of the npm package recorded in `assets/themes/SOURCE.md`.
//! Nothing is fetched: a generator that reaches the network cannot be reviewed
//! by reading its output diff.
use std::fs;
use std::io;
use std::path::{Path, PathBuf};
use serde_json::Value;
/// One theme's row in `assets/themes/metadata.json`.
///
/// The fields are the ones attribution needs plus the display name, which is
/// upstream's own spelling of the theme's title and is not derivable from the
/// file name (`github-dark-dimmed` is "GitHub Dark Dimmed", not "Github Dark
/// Dimmed").
pub struct ThemeMetadata {
pub name: String,
pub display_name: String,
pub source: String,
pub spdx: String,
pub license_url: String,
pub license_file: String,
}
/// A theme file paired with its attribution row.
pub struct SourceTheme {
pub metadata: ThemeMetadata,
pub document: Value,
}
/// Everything under `assets/themes/`, in the order the catalog will list it.
///
/// # Errors
///
/// Fails if the assets directory is missing, if `metadata.json` is not the
/// array of objects this expects, or if a theme named in the metadata has no
/// JSON file beside it. Each of those means the vendored set and its manifest
/// have drifted apart, and generating from a half-known set would put themes in
/// the catalog with attribution that does not describe them.
pub fn load(assets: &Path) -> io::Result<Vec<SourceTheme>> {
let manifest_path = assets.join("metadata.json");
let manifest = read_json(&manifest_path)?;
let rows = manifest
.as_array()
.ok_or_else(|| invalid(&manifest_path, "expected a JSON array of theme rows"))?;
let mut themes = Vec::with_capacity(rows.len());
for row in rows {
let metadata = read_metadata(row, &manifest_path)?;
let theme_path = assets
.join("tm-themes")
.join(format!("{}.json", metadata.name));
let document = read_json(&theme_path)?;
themes.push(SourceTheme { metadata, document });
}
themes.sort_by(|left, right| left.metadata.name.cmp(&right.metadata.name));
Ok(themes)
}
fn read_metadata(row: &Value, manifest_path: &Path) -> io::Result<ThemeMetadata> {
let field = |key: &str| -> io::Result<String> {
row.get(key)
.and_then(Value::as_str)
.map(str::to_owned)
.ok_or_else(|| invalid(manifest_path, &format!("a row is missing `{key}`")))
};
Ok(ThemeMetadata {
name: field("name")?,
display_name: field("displayName")?,
source: field("source")?,
spdx: field("spdx")?,
license_url: field("licenseUrl")?,
license_file: field("licenseFile")?,
})
}
fn read_json(path: &Path) -> io::Result<Value> {
let text = fs::read_to_string(path)
.map_err(|error| io::Error::new(error.kind(), format!("{}: {error}", path.display())))?;
serde_json::from_str(&text).map_err(|error| invalid(path, &error.to_string()))
}
fn invalid(path: &Path, message: &str) -> io::Error {
io::Error::new(
io::ErrorKind::InvalidData,
format!("{}: {message}", path.display()),
)
}
/// The repository root, inferred from where this crate was compiled.
///
/// `tools/theme-gen` sits two directories below the root, so the tool can be
/// run from anywhere without a path argument. An explicit argument still wins,
/// which is what the tests and any future relocation use.
#[must_use]
pub fn repository_root() -> PathBuf {
let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
manifest
.parent()
.and_then(Path::parent)
.map_or(manifest.clone(), Path::to_path_buf)
}