Build interactive native workspace vertical slice
CI / rust (push) Successful in 1m43s

This commit is contained in:
2026-08-31 16:15:32 -07:00
parent de89b015bc
commit 27beb69ff8
19 changed files with 10535 additions and 200 deletions
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,14 @@
[package]
name = "lumbridge-spike-gpui-accessibility"
description = "Isolated proof of current GPUI accessibility and IME input for Lumbridge"
version = "0.0.1"
edition = "2024"
rust-version = "1.97.1"
license = "Apache-2.0"
publish = false
[dependencies]
gpui = { git = "https://github.com/zed-industries/zed.git", rev = "ce48461eaadd16c65c31f835511ab96bd3b6e746", default-features = false, features = ["wayland", "x11"] }
gpui_platform = { git = "https://github.com/zed-industries/zed.git", rev = "ce48461eaadd16c65c31f835511ab96bd3b6e746", default-features = false, features = ["font-kit", "wayland", "x11"] }
[workspace]
+83
View File
@@ -0,0 +1,83 @@
# GPUI accessibility and input probe
This independent workspace answers one narrow question: can the GPUI version
currently used by Zed expose the minimum semantic tree Lumbridge needs for a
workspace and terminal pane, while accepting real platform IME input?
## Reproducibility
- Upstream: `https://github.com/zed-industries/zed`
- Commit: `ce48461eaadd16c65c31f835511ab96bd3b6e746`
- GPUI license at this commit: Apache-2.0
- Rust: 1.97.1, matching the upstream `rust-toolchain.toml`
Both `gpui` and `gpui_platform` are pinned to the full commit. This directory is
an independent Cargo workspace, so its toolchain and dependency graph do not
change Lumbridge's root MSRV or release graph.
## Semantics under test
The rendered tree contains:
- an application named `Lumbridge accessibility probe`;
- a stable, externally identifiable `Region` named `Lumbridge workspace`;
- a stable, focusable `Pane` named `Local terminal pane`;
- `selected = true` on the pane;
- real GPUI keyboard focus tracked on the pane;
- the description `Needs input: choose whether to run the proposed command`;
- a `Terminal` child named `Terminal output` for the pane's content.
Current GPUI exposes these through its AccessKit integration using `role`,
`accessibility_id`, `aria_label`, `aria_description`, `aria_selected`,
`focusable`, and `track_focus`. The deterministic unit test checks the exact
AccessKit role and properties. The binary compile-checks GPUI's element wiring;
a platform screen reader remains necessary for end-to-end AT-SPI/VoiceOver
validation.
## Input and IME proof
The focused pane also installs a real `ElementInputHandler<ImeTextReceiver>`
during element paint. `ImeTextReceiver` implements GPUI's
`EntityInputHandler`, including text queries, UTF-16 selection, marked-text
composition, replacement, unmarking, selection updates, editable length, and
IME candidate bounds. This is the code path GPUI's platform adapters call for
composed operating-system text input.
The backing buffer stores UTF-8 only at valid scalar boundaries and translates
the platform's UTF-16 ranges before mutation. Deterministic tests cover:
- replacing a selected decomposed `e` plus combining acute accent with `é`;
- a marked decomposed-accent IME composition and commit;
- inserting, selecting, and replacing the multi-codepoint grapheme `👩🏽‍💻`;
- collapsed selection placement after every replacement.
These tests prove range conversion and composed-text preservation without
splitting UTF-8. An actual keyboard IME on X11/Wayland and macOS remains an
end-to-end platform test, not a unit-test claim.
## Commands
```bash
cargo check --locked
cargo test --locked
cargo clippy --locked --all-targets -- -D warnings
```
The first command installs the pinned Rust toolchain if rustup does not already
have it and downloads Zed's GPUI dependency closure.
On Ubuntu hosts that have only the runtime libraries, checks work with
`RUST_FONTCONFIG_DLOPEN=1`. Linking a test or binary additionally needs the
unversioned linker names normally installed by `libxcb1-dev`,
`libxkbcommon-dev`, and `libxkbcommon-x11-dev`. The amd-server validation used
temporary symlinks under the ignored `target/native-libs` directory and set
`LIBRARY_PATH` to that directory; no system packages or root files changed.
## Observed local cost
On amd-server, the first successful check required roughly 90 seconds after
installing/fetching, with some Cargo-cache contention from concurrent work. The
lockfile contains 691 packages. After check, test, Clippy, and a debug build,
the isolated target directory was 4.4 GiB and the debug binary was 535 MiB. The
minimal Rust 1.97.1 toolchain occupies 624 MiB. These are development costs, not
optimized release measurements.
@@ -0,0 +1,4 @@
[toolchain]
channel = "1.97.1"
profile = "minimal"
components = ["clippy", "rustfmt"]
+486
View File
@@ -0,0 +1,486 @@
use std::ops::Range;
use gpui::{
App, Bounds, ClipboardItem, Context, Element, ElementId, ElementInputHandler, Entity,
EntityInputHandler, FocusHandle, GlobalElementId, LayoutId, Pixels, Point, Role, Style,
UTF16Selection, Window, WindowBounds, WindowOptions, div, prelude::*, px, relative, rgb, size,
text,
};
use gpui_platform::application;
const WORKSPACE_LABEL: &str = "Lumbridge workspace";
const PANE_LABEL: &str = "Local terminal pane";
const PANE_DESCRIPTION: &str = "Needs input: choose whether to run the proposed command";
const PANE_ACCESSIBILITY_ID: &str = "lumbridge.pane.local-terminal";
struct AccessibilityProbe {
pane_focus: FocusHandle,
input: Entity<ImeTextReceiver>,
}
impl AccessibilityProbe {
fn new(window: &mut Window, cx: &mut Context<Self>) -> Self {
let pane_focus = cx.focus_handle().tab_stop(true);
window.focus(&pane_focus, cx);
Self {
pane_focus,
input: cx.new(|_| ImeTextReceiver::default()),
}
}
}
#[derive(Clone, Copy)]
enum BoundaryBias {
Floor,
Ceil,
}
#[derive(Debug, Default, PartialEq, Eq)]
struct TextBuffer {
text: String,
selection: Range<usize>,
marked: Option<Range<usize>>,
}
impl TextBuffer {
fn utf16_to_utf8(text: &str, offset: usize, bias: BoundaryBias) -> usize {
let mut utf16_offset = 0;
for (utf8_offset, character) in text.char_indices() {
if utf16_offset == offset {
return utf8_offset;
}
let next_utf16 = utf16_offset + character.len_utf16();
if offset < next_utf16 {
return match bias {
BoundaryBias::Floor => utf8_offset,
BoundaryBias::Ceil => utf8_offset + character.len_utf8(),
};
}
utf16_offset = next_utf16;
}
text.len()
}
fn utf8_to_utf16(text: &str, offset: usize) -> usize {
text[..offset].encode_utf16().count()
}
fn range_from_utf16(text: &str, range: &Range<usize>) -> Range<usize> {
if range.is_empty() {
let offset = Self::utf16_to_utf8(text, range.start, BoundaryBias::Floor);
return offset..offset;
}
Self::utf16_to_utf8(text, range.start, BoundaryBias::Floor)
..Self::utf16_to_utf8(text, range.end, BoundaryBias::Ceil)
}
fn range_to_utf16(&self, range: &Range<usize>) -> Range<usize> {
Self::utf8_to_utf16(&self.text, range.start)..Self::utf8_to_utf16(&self.text, range.end)
}
fn selection_utf16(&self) -> Range<usize> {
self.range_to_utf16(&self.selection)
}
fn marked_utf16(&self) -> Option<Range<usize>> {
self.marked.as_ref().map(|range| self.range_to_utf16(range))
}
fn set_selection_utf16(&mut self, range: Range<usize>) {
self.selection = Self::range_from_utf16(&self.text, &range);
}
fn replacement_range(&self, range_utf16: Option<&Range<usize>>) -> Range<usize> {
range_utf16
.map(|range| Self::range_from_utf16(&self.text, range))
.or_else(|| self.marked.clone())
.unwrap_or_else(|| self.selection.clone())
}
fn replace_text_in_range(&mut self, range_utf16: Option<Range<usize>>, new_text: &str) {
let range = self.replacement_range(range_utf16.as_ref());
let caret = range.start + new_text.len();
self.text.replace_range(range, new_text);
self.selection = caret..caret;
self.marked = None;
}
fn replace_and_mark_text_in_range(
&mut self,
range_utf16: Option<Range<usize>>,
new_text: &str,
new_selection_utf16: Option<Range<usize>>,
) {
let range = self.replacement_range(range_utf16.as_ref());
let insertion_start = range.start;
self.text.replace_range(range, new_text);
let inserted = insertion_start..insertion_start + new_text.len();
self.marked = (!new_text.is_empty()).then(|| inserted.clone());
self.selection = new_selection_utf16
.map(|selection| Self::range_from_utf16(new_text, &selection))
.map(|selection| insertion_start + selection.start..insertion_start + selection.end)
.unwrap_or(inserted.end..inserted.end);
}
}
#[derive(Debug, Default)]
struct ImeTextReceiver {
buffer: TextBuffer,
}
impl EntityInputHandler for ImeTextReceiver {
fn text_for_range(
&mut self,
range_utf16: Range<usize>,
adjusted_range: &mut Option<Range<usize>>,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<String> {
let range = TextBuffer::range_from_utf16(&self.buffer.text, &range_utf16);
adjusted_range.replace(self.buffer.range_to_utf16(&range));
Some(self.buffer.text[range].to_owned())
}
fn selected_text_range(
&mut self,
_ignore_disabled_input: bool,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<UTF16Selection> {
Some(UTF16Selection {
range: self.buffer.selection_utf16(),
reversed: false,
})
}
fn marked_text_range(
&self,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<Range<usize>> {
self.buffer.marked_utf16()
}
fn unmark_text(&mut self, _window: &mut Window, cx: &mut Context<Self>) {
self.buffer.marked = None;
cx.notify();
}
fn paste(&mut self, item: ClipboardItem, window: &mut Window, cx: &mut Context<Self>) {
if let Some(text) = item.text() {
self.replace_text_in_range(None, &text, window, cx);
}
}
fn replace_text_in_range(
&mut self,
range_utf16: Option<Range<usize>>,
text: &str,
_window: &mut Window,
cx: &mut Context<Self>,
) {
self.buffer.replace_text_in_range(range_utf16, text);
cx.notify();
}
fn replace_and_mark_text_in_range(
&mut self,
range_utf16: Option<Range<usize>>,
new_text: &str,
new_selected_range_utf16: Option<Range<usize>>,
_window: &mut Window,
cx: &mut Context<Self>,
) {
self.buffer
.replace_and_mark_text_in_range(range_utf16, new_text, new_selected_range_utf16);
cx.notify();
}
fn bounds_for_range(
&mut self,
_range_utf16: Range<usize>,
element_bounds: Bounds<Pixels>,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<Bounds<Pixels>> {
Some(element_bounds)
}
fn character_index_for_point(
&mut self,
_point: Point<Pixels>,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<usize> {
Some(self.buffer.selection_utf16().end)
}
fn set_selected_text_range(
&mut self,
range_utf16: Range<usize>,
_window: &mut Window,
cx: &mut Context<Self>,
) {
self.buffer.set_selection_utf16(range_utf16);
cx.notify();
}
fn text_length_utf16(
&mut self,
_window: &mut Window,
_cx: &mut Context<Self>,
) -> Option<usize> {
Some(self.buffer.text.encode_utf16().count())
}
}
struct InputCaptureElement {
input: Entity<ImeTextReceiver>,
focus: FocusHandle,
}
impl IntoElement for InputCaptureElement {
type Element = Self;
fn into_element(self) -> Self::Element {
self
}
}
impl Element for InputCaptureElement {
type RequestLayoutState = ();
type PrepaintState = ();
fn id(&self) -> Option<ElementId> {
None
}
fn source_location(&self) -> Option<&'static core::panic::Location<'static>> {
None
}
fn request_layout(
&mut self,
_id: Option<&GlobalElementId>,
_inspector_id: Option<&gpui::InspectorElementId>,
window: &mut Window,
cx: &mut App,
) -> (LayoutId, Self::RequestLayoutState) {
let mut style = Style::default();
style.size.width = relative(1.).into();
style.size.height = px(32.).into();
(window.request_layout(style, [], cx), ())
}
fn prepaint(
&mut self,
_id: Option<&GlobalElementId>,
_inspector_id: Option<&gpui::InspectorElementId>,
_bounds: Bounds<Pixels>,
_request_layout: &mut Self::RequestLayoutState,
_window: &mut Window,
_cx: &mut App,
) -> Self::PrepaintState {
}
fn paint(
&mut self,
_id: Option<&GlobalElementId>,
_inspector_id: Option<&gpui::InspectorElementId>,
bounds: Bounds<Pixels>,
_request_layout: &mut Self::RequestLayoutState,
_prepaint: &mut Self::PrepaintState,
window: &mut Window,
cx: &mut App,
) {
window.handle_input(
&self.focus,
ElementInputHandler::new(bounds, self.input.clone()),
cx,
);
}
}
impl Render for AccessibilityProbe {
fn render(&mut self, _window: &mut Window, cx: &mut Context<Self>) -> impl IntoElement {
let input_text = self.input.read(cx).buffer.text.clone();
div()
.id("lumbridge-application")
.role(Role::Application)
.aria_label("Lumbridge accessibility probe")
.size_full()
.p_4()
.bg(rgb(0x0c0e13))
.text_color(rgb(0xd9e2f2))
.child(
div()
.id("workspace")
.accessibility_id("lumbridge.workspace.primary")
.role(Role::Region)
.aria_label(WORKSPACE_LABEL)
.size_full()
.p_3()
.border_1()
.border_color(rgb(0x293244))
.rounded_md()
.child(
div()
.id("pane-local-terminal")
.accessibility_id(PANE_ACCESSIBILITY_ID)
.role(Role::Pane)
.aria_label(PANE_LABEL)
.aria_description(PANE_DESCRIPTION)
.aria_selected(true)
.focusable()
.track_focus(&self.pane_focus)
.size_full()
.p_3()
.bg(rgb(0x121722))
.rounded_md()
.child(
div()
.id("terminal-content")
.role(Role::Terminal)
.aria_label("Terminal output")
.flex()
.flex_col()
.gap_2()
.child(text!("codex is waiting for approval"))
.child(
div()
.id("ime-input-status")
.role(Role::Status)
.aria_label("Composed input")
.child(if input_text.is_empty() {
"Type composed Unicode here…".to_owned()
} else {
input_text
}),
)
.child(InputCaptureElement {
input: self.input.clone(),
focus: self.pane_focus.clone(),
}),
),
),
)
}
}
fn main() {
application().run(|cx: &mut App| {
let bounds = Bounds::centered(None, size(px(760.0), px(420.0)), cx);
cx.open_window(
WindowOptions {
window_bounds: Some(WindowBounds::Windowed(bounds)),
titlebar: Some(gpui::TitlebarOptions {
title: Some("Lumbridge · GPUI accessibility probe".into()),
..Default::default()
}),
..Default::default()
},
|window, cx| cx.new(|cx| AccessibilityProbe::new(window, cx)),
)
.expect("GPUI accessibility probe window should open");
cx.activate(true);
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn workspace_and_focused_pane_map_to_an_accesskit_tree() {
use gpui::accesskit::{Node, NodeId, Tree, TreeId, TreeUpdate};
let workspace_id = NodeId(1);
let pane_id = NodeId(2);
let mut workspace = Node::new(Role::Region);
workspace.set_label(WORKSPACE_LABEL.to_owned());
workspace.set_children(vec![pane_id]);
let mut pane = Node::new(Role::Pane);
pane.set_author_id(PANE_ACCESSIBILITY_ID.to_owned());
pane.set_label(PANE_LABEL.to_owned());
pane.set_description(PANE_DESCRIPTION.to_owned());
pane.set_selected(true);
let update = TreeUpdate {
nodes: vec![(workspace_id, workspace), (pane_id, pane)],
tree: Some(Tree::new(workspace_id)),
tree_id: TreeId::ROOT,
focus: pane_id,
};
let pane = &update.nodes[1].1;
assert_eq!(
update.tree.as_ref().map(|tree| tree.root),
Some(workspace_id)
);
assert_eq!(update.focus, pane_id);
assert_eq!(update.nodes[0].1.role(), Role::Region);
assert_eq!(update.nodes[0].1.label(), Some(WORKSPACE_LABEL));
assert_eq!(pane.role(), Role::Pane);
assert_eq!(pane.author_id(), Some(PANE_ACCESSIBILITY_ID));
assert_eq!(pane.label(), Some(PANE_LABEL));
assert_eq!(pane.description(), Some(PANE_DESCRIPTION));
assert_eq!(pane.is_selected(), Some(true));
}
#[test]
fn replace_selected_composed_accent_without_splitting_utf8() {
let mut buffer = TextBuffer {
text: "Cafe\u{301}".to_owned(),
selection: 0..0,
marked: None,
};
buffer.set_selection_utf16(3..5);
buffer.replace_text_in_range(None, "é");
assert_eq!(buffer.text, "Café");
assert_eq!(buffer.selection_utf16(), 4..4);
assert!(buffer.text.is_char_boundary(buffer.selection.start));
}
#[test]
fn composed_ime_and_multi_codepoint_grapheme_preserve_selection() {
let grapheme = "👩🏽‍💻";
let grapheme_utf16_len = grapheme.encode_utf16().count();
let mut buffer = TextBuffer::default();
buffer.replace_and_mark_text_in_range(Some(0..0), "e\u{301}", Some(2..2));
assert_eq!(buffer.text, "e\u{301}");
assert_eq!(buffer.marked_utf16(), Some(0..2));
assert_eq!(buffer.selection_utf16(), 2..2);
buffer.replace_text_in_range(None, "é");
assert_eq!(buffer.text, "é");
assert_eq!(buffer.marked_utf16(), None);
assert_eq!(buffer.selection_utf16(), 1..1);
buffer.set_selection_utf16(0..1);
buffer.replace_text_in_range(None, grapheme);
assert_eq!(buffer.text, grapheme);
assert_eq!(
buffer.selection_utf16(),
grapheme_utf16_len..grapheme_utf16_len
);
assert!(buffer.text.is_char_boundary(buffer.selection.start));
buffer.set_selection_utf16(0..grapheme_utf16_len);
buffer.replace_text_in_range(None, "done");
assert_eq!(buffer.text, "done");
assert_eq!(buffer.selection_utf16(), 4..4);
}
}