feat(frontend): shared pod-wide test audio

Make the "Test audio" connectivity check pod-wide instead of local to the
publisher. Shared on/off state is derived from the presence of the podman-beat
track across participants (self-syncing on join/leave). Any participant can
stop it: the owner unpublishes directly; non-owners send a new additive
BEAT_STOP data message that the owner honors. Drives the button label, status
line, and status waveform for everyone, and guards against rapid-double-click
double-publish and mid-start unmount leaks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Kartikeya
2026-06-27 23:39:34 -07:00
parent e0757088ef
commit 7269098ea3
4 changed files with 230 additions and 24 deletions
+64
View File
@@ -0,0 +1,64 @@
# Shared Test Audio — pod-wide connectivity check
> Spec for the `frontend/src/livekit/useBeat.ts` + `PodView.tsx` test-audio
> behavior and the additive `BEAT_STOP` data message. Satisfies the
> documentation-first gate for those files.
## Why
The **Test audio** button is PodMan's pre-flight check that the LiveKit audio
path works for the whole pod — the same path the urgent Gemini-TTS voice
escalation rides on. Today the beat is published correctly but its on/off state
is **local to the publisher**: teammates can't see it's playing and can't stop
it. This makes it a shared, pod-wide toggle so a judge sees the state flip on
every screen at once.
## Behavior
- Any participant clicks **Test audio** → they publish the `podman-beat` audio
track (Web Audio, `lib/beat.ts`). Everyone auto-subscribes and hears it.
- The shared on/off state is **derived from the track's presence**, not a synced
flag — so it self-syncs across joins/leaves and can't drift from reality. The
publisher is the **owner**.
- Anyone can stop it:
- Owner clicks **Stop audio** → unpublishes its own track directly.
- Non-owner clicks **Stop (`<owner>`'s)** → sends `BEAT_STOP`; the owner
unpublishes. (LiveKit forbids unpublishing another participant's track, so a
request is the only way.)
- The Status card shows `publishing` / `<owner> playing` / `ready`, and the
waveform animates (`active`) for everyone while the test is live.
## State derivation (source of truth = the track)
`useBeat(room)` returns `{ on, by, mine }`, recomputed from the presence of a
track named `podman-beat` across `localParticipant` + `remoteParticipants` on
these events: `LocalTrackPublished/Unpublished`, `TrackPublished/Unpublished`,
`TrackSubscribed/Unsubscribed`, `ParticipantConnected/Disconnected`. Owner
disconnect and late-join sync therefore need no extra messaging.
## Contract (additive)
`shared/src/messages.ts` — one new message on the existing `podman.intervention`
data topic:
| { type: 'BEAT_STOP' } // any participant → owner: stop the shared beat
Additive to the `DataMessage` union; existing consumers ignore unknown types.
**No backend / API change.**
## Known limitation (LiveKit constraint)
A client can only unpublish **its own** tracks, so a non-owner's **Stop** is a
`BEAT_STOP` _request_ the owner must honor. If the owner disconnects **uncleanly**
(crash / network drop), the SFU keeps the track published until it times the
participant out — during that window the beat keeps playing and non-owners can't
stop it. A clean disconnect clears it immediately via `ParticipantDisconnected`.
Demo mitigation: have the same person who starts the test also stop it.
## Files
- `shared/src/messages.ts``BEAT_STOP` message (additive).
- `frontend/src/livekit/useBeat.ts``useBeat(room)` hook.
- `frontend/src/components/PodView.tsx` — button label, status line, waveform
`active` driven by the hook.
- `frontend/src/lib/beat.ts` — unchanged (existing Web-Audio beat source).
+13 -23
View File
@@ -22,7 +22,7 @@ import {
} from 'lucide-react'; } from 'lucide-react';
import type { Room, RemoteTrack, RemoteTrackPublication, RemoteParticipant } from 'livekit-client'; import type { Room, RemoteTrack, RemoteTrackPublication, RemoteParticipant } from 'livekit-client';
import type { Pod, PodActivityEvent, PodActivityKind, PodActivitySource } from '@podman/shared'; import type { Pod, PodActivityEvent, PodActivityKind, PodActivitySource } from '@podman/shared';
import { startBeat, type BeatHandle } from '../lib/beat.js'; import { useBeat } from '../livekit/useBeat.js';
import { useInterventions, primeSpeech } from '../livekit/useInterventions.js'; import { useInterventions, primeSpeech } from '../livekit/useInterventions.js';
import { usePodActivity } from '../hooks/use-pod-activity.js'; import { usePodActivity } from '../hooks/use-pod-activity.js';
import LiveWaveform from '@/components/ruixen/live-waveform'; import LiveWaveform from '@/components/ruixen/live-waveform';
@@ -109,7 +109,6 @@ export function PodView({
}) { }) {
const [participants, setParticipants] = useState<PInfo[]>([]); const [participants, setParticipants] = useState<PInfo[]>([]);
const [sharing, setSharing] = useState(false); const [sharing, setSharing] = useState(false);
const [playingBeat, setPlayingBeat] = useState(false);
const [note, setNote] = useState<string | null>(null); const [note, setNote] = useState<string | null>(null);
const [audioBlocked, setAudioBlocked] = useState(false); const [audioBlocked, setAudioBlocked] = useState(false);
const [leftStreamOpen, setLeftStreamOpen] = useState(() => const [leftStreamOpen, setLeftStreamOpen] = useState(() =>
@@ -119,10 +118,10 @@ export function PodView({
readStoredBool('podman.teamStreamOpen', true), readStoredBool('podman.teamStreamOpen', true),
); );
const { active, hermes, voiceCue, actionUrl, respond } = useInterventions(room); const { active, hermes, voiceCue, actionUrl, respond } = useInterventions(room);
const { beat, toggleBeat: runBeat } = useBeat(room);
const activity = usePodActivity(team.id, me); const activity = usePodActivity(team.id, me);
const audioRef = useRef<HTMLDivElement>(null); const audioRef = useRef<HTMLDivElement>(null);
const beatRef = useRef<BeatHandle | null>(null);
const screenTrackRef = useRef<MediaStreamTrack | null>(null); const screenTrackRef = useRef<MediaStreamTrack | null>(null);
const onLeaveRef = useRef(onLeave); const onLeaveRef = useRef(onLeave);
onLeaveRef.current = onLeave; onLeaveRef.current = onLeave;
@@ -179,8 +178,6 @@ export function PodView({
useEffect(() => { useEffect(() => {
return () => { return () => {
beatRef.current?.stop();
beatRef.current = null;
screenTrackRef.current?.stop(); screenTrackRef.current?.stop();
screenTrackRef.current = null; screenTrackRef.current = null;
}; };
@@ -194,22 +191,10 @@ export function PodView({
localStorage.setItem('podman.teamStreamOpen', String(rightStreamOpen)); localStorage.setItem('podman.teamStreamOpen', String(rightStreamOpen));
}, [rightStreamOpen]); }, [rightStreamOpen]);
async function toggleBeat() { async function onToggleBeat() {
if (!room) return;
setNote(null); setNote(null);
try { try {
if (playingBeat) { await runBeat();
if (beatRef.current) await room.localParticipant.unpublishTrack(beatRef.current.track);
beatRef.current?.stop();
beatRef.current = null;
setPlayingBeat(false);
} else {
await room.startAudio().catch(() => {});
const handle = startBeat();
beatRef.current = handle;
await room.localParticipant.publishTrack(handle.track, { name: 'podman-beat' });
setPlayingBeat(true);
}
} catch (e) { } catch (e) {
setNote(`Audio test failed: ${(e as Error).message}`); setNote(`Audio test failed: ${(e as Error).message}`);
} }
@@ -328,9 +313,9 @@ export function PodView({
</div> </div>
<div className="grid grid-cols-2 gap-2 sm:flex sm:items-center"> <div className="grid grid-cols-2 gap-2 sm:flex sm:items-center">
<Button variant="outline" onClick={toggleBeat} disabled={!room}> <Button variant="outline" onClick={onToggleBeat} disabled={!room}>
<Volume2Icon data-icon="inline-start" /> <Volume2Icon data-icon="inline-start" />
{playingBeat ? 'Stop audio' : 'Test audio'} {beat.on ? (beat.mine ? 'Stop audio' : `Stop (${beat.by})`) : 'Test audio'}
</Button> </Button>
<Button onClick={toggleScreen} disabled={!room}> <Button onClick={toggleScreen} disabled={!room}>
<MonitorUpIcon data-icon="inline-start" /> <MonitorUpIcon data-icon="inline-start" />
@@ -508,7 +493,7 @@ export function PodView({
<CardContent> <CardContent>
<LiveWaveform <LiveWaveform
processing={!!room} processing={!!room}
active={false} active={beat.on}
height={32} height={32}
barWidth={2} barWidth={2}
barGap={3} barGap={3}
@@ -517,7 +502,12 @@ export function PodView({
<div className="flex flex-col gap-3"> <div className="flex flex-col gap-3">
<StatusLine label="LiveKit" value={room ? 'connected' : 'offline'} /> <StatusLine label="LiveKit" value={room ? 'connected' : 'offline'} />
<StatusLine label="Screen" value={sharing ? 'published' : 'not shared'} /> <StatusLine label="Screen" value={sharing ? 'published' : 'not shared'} />
<StatusLine label="Audio" value={playingBeat ? 'publishing' : 'ready'} /> <StatusLine
label="Audio"
value={
beat.on ? (beat.mine ? 'publishing' : `${beat.by} playing`) : 'ready'
}
/>
<StatusLine label="Agent" value={podmanPresent ? 'watching' : 'waiting'} /> <StatusLine label="Agent" value={podmanPresent ? 'watching' : 'waiting'} />
</div> </div>
</CardContent> </CardContent>
+150
View File
@@ -0,0 +1,150 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { RoomEvent, type Room } from 'livekit-client';
import { DATA_TOPIC, type DataMessage } from '@podman/shared';
import { startBeat, type BeatHandle } from '../lib/beat.js';
/** Name of the test-audio track; its presence in the room IS the shared state. */
export const BEAT_TRACK = 'podman-beat';
export interface BeatState {
/** Is the test audio playing anywhere in the pod? */
on: boolean;
/** Display name of the participant who started it (the owner). */
by: string | null;
/** Do I own the beat track (so I can stop it directly)? */
mine: boolean;
}
const OFF: BeatState = { on: false, by: null, mine: false };
/**
* Shared, pod-wide test audio. One participant publishes the `podman-beat`
* track; everyone hears it and sees the same on/off state, derived directly
* from the track's presence (self-syncing across joins/leaves). Any participant
* can stop it: non-owners send BEAT_STOP and the owner unpublishes.
*/
export function useBeat(room: Room | null) {
const [beat, setBeat] = useState<BeatState>(OFF);
const beatRef = useRef<BeatHandle | null>(null);
const stopLocal = useCallback(async () => {
const handle = beatRef.current;
if (!handle) return;
beatRef.current = null;
try {
await room?.localParticipant.unpublishTrack(handle.track);
} finally {
handle.stop();
}
}, [room]);
// Derive the shared state from the podman-beat track across all participants.
useEffect(() => {
if (!room) {
setBeat(OFF);
return;
}
const recompute = () => {
const lp = room.localParticipant;
const localPub = [...lp.trackPublications.values()].find((p) => p.trackName === BEAT_TRACK);
if (localPub) {
setBeat({ on: true, by: lp.name || lp.identity, mine: true });
return;
}
for (const p of room.remoteParticipants.values()) {
const pub = [...p.trackPublications.values()].find((tp) => tp.trackName === BEAT_TRACK);
if (pub) {
setBeat({ on: true, by: p.name || p.identity, mine: false });
return;
}
}
setBeat(OFF);
};
recompute();
const events = [
RoomEvent.LocalTrackPublished,
RoomEvent.LocalTrackUnpublished,
RoomEvent.TrackPublished,
RoomEvent.TrackUnpublished,
RoomEvent.TrackSubscribed,
RoomEvent.TrackUnsubscribed,
RoomEvent.ParticipantConnected,
RoomEvent.ParticipantDisconnected,
] as const;
events.forEach((e) => room.on(e, recompute));
return () => {
events.forEach((e) => room.off(e, recompute));
};
}, [room]);
// The owner honors stop requests from any participant.
useEffect(() => {
if (!room) return;
const onData = (payload: Uint8Array, _p: unknown, _k: unknown, topic?: string) => {
if (topic !== DATA_TOPIC) return;
let msg: DataMessage;
try {
msg = JSON.parse(new TextDecoder().decode(payload)) as DataMessage;
} catch {
return;
}
if (msg.type === 'BEAT_STOP') void stopLocal();
};
room.on(RoomEvent.DataReceived, onData);
return () => {
room.off(RoomEvent.DataReceived, onData);
};
}, [room, stopLocal]);
// Tear down my own track on unmount (e.g. leaving the pod).
const stopLocalRef = useRef(stopLocal);
stopLocalRef.current = stopLocal;
const startingRef = useRef(false);
const unmountedRef = useRef(false);
useEffect(() => {
return () => {
unmountedRef.current = true;
void stopLocalRef.current();
};
}, []);
const toggleBeat = useCallback(async () => {
if (!room) return;
// `beatRef` is the synchronous source of truth for "do I own it" — `beat.mine`
// lags behind the LiveKit track events that recompute it, so gate on the ref.
if (beatRef.current) {
await stopLocal();
return;
}
if (beat.on) {
// Someone else owns it — can't unpublish their track, so ask them to stop.
await room.startAudio().catch(() => {});
await room.localParticipant.publishData(
new TextEncoder().encode(JSON.stringify({ type: 'BEAT_STOP' } satisfies DataMessage)),
{ reliable: true, topic: DATA_TOPIC },
);
return;
}
// Start it. Guard against rapid double-clicks publishing two tracks before the
// LocalTrackPublished event has had a chance to update state.
if (startingRef.current) return;
startingRef.current = true;
try {
await room.startAudio().catch(() => {}); // unlock playback from this gesture
if (unmountedRef.current) return;
const handle = startBeat();
beatRef.current = handle;
await room.localParticipant.publishTrack(handle.track, { name: BEAT_TRACK });
if (unmountedRef.current) await stopLocal(); // left mid-publish — clean up
} catch (e) {
beatRef.current?.stop();
beatRef.current = null;
throw e;
} finally {
startingRef.current = false;
}
}, [room, beat, stopLocal]);
return { beat, toggleBeat };
}
+3 -1
View File
@@ -10,7 +10,9 @@ export type DataMessage =
| { type: 'HERMES_MESSAGE'; message: HermesMessage } | { type: 'HERMES_MESSAGE'; message: HermesMessage }
| { type: 'VOICE_CUE'; text: string } | { type: 'VOICE_CUE'; text: string }
| { type: 'ACK'; interventionId: string; status: InterventionStatus; note?: string } | { type: 'ACK'; interventionId: string; status: InterventionStatus; note?: string }
| { type: 'GIT_REPORT'; report: LocalGitReport }; | { type: 'GIT_REPORT'; report: LocalGitReport }
/** Any participant → the current test-audio owner: stop publishing the shared beat. */
| { type: 'BEAT_STOP' };
/** A targeted teammate/project-channel notification from the Hermes action layer. */ /** A targeted teammate/project-channel notification from the Hermes action layer. */
export interface HermesMessage { export interface HermesMessage {