LiveKit Integration Spec

LiveKit is the real-time backbone for PodMan. It handles room presence and voice delivery. It is load-bearing — not decorative.

Room structure

One LiveKit room per project pod: room = podId
Engineers join as named participants (e.g. alice, bob)
Hermes joins as podman-hermes
All participants stay connected for the duration of the session

Engineer side (PWA)

Joining:

PWA calls POST /pods/:podId/token → receives { token, url }
LiveKit client connects to the room with the token
PWA publishes screen track via getDisplayMedia
PWA sets mic enabled for ambient presence

Receiving:

LiveKit client subscribes to remote Hermes audio tracks and attaches them to a hidden audio sink in the DOM.
Browser autoplay restrictions still apply. The PWA calls room.startAudio() from user gestures such as first room click, Enable audio, Test PodMan voice, and Share screen.
PWA also listens for data channel messages from Hermes for UI card updates and VOICE_CUE fallback text.

Data channel listener (PWA):

room.on(RoomEvent.DataReceived, (payload, participant) => {
  if (participant?.identity !== 'podman-hermes') return;
  const intervention = JSON.parse(new TextDecoder().decode(payload));
  // intervention: COLLISION, HERMES_MESSAGE, VOICE_CUE, ACK, or GIT_REPORT
  appendInterventionToFeed(intervention);
});

Hermes side (PodMan LiveKit participant)

Framework: @livekit/rtc-node

Startup:

Hermes mints its own token via the same createPodToken function with identity: 'podman-hermes'
Connects to the configured room as podman-hermes
Publishes data-channel cards/messages and Gemini TTS audio tracks

Voice delivery:

Urgent intervention text is ready (from Gemini text generation)
Hermes sends a natural-speaking prompt to Gemini TTS
Gemini returns PCM audio using the configured voice
Hermes publishes the audio as a LiveKit microphone-source track
Hermes keeps the track published for the generated audio duration plus tail silence and a hold window. This avoids browser-side cutoff when LiveKit's queued playout signal returns before subscribers finish playing buffered audio.
All participants hear it after browser audio has been unlocked

Data channel message (sent alongside audio):

const intervention = {
  type: 'DEPENDENCY_READY' | 'BLOCKER_DETECTED' | 'DUPLICATE_WORK',
  message: string,        // the spoken text
  involvedEngineers: string[],
  file: string | null,
  sentAt: string,         // ISO timestamp
};
room.localParticipant.publishData(
  new TextEncoder().encode(JSON.stringify(intervention)),
  { reliable: true }
);

Token endpoint

Already implemented at POST /api/token.

Hermes uses the same endpoint. Grants:

roomJoin: true
canPublish: true (for audio track)
canPublishData: true (for data channel)
canSubscribe: true

Gemini voice model

Model ID: gemini-3.1-flash-tts-preview
Default voice: Charon (GEMINI_TTS_VOICE)
Hermes generates Gemini TTS audio and publishes it as a LiveKit audio track.
Voice publishing logs generated frame count, estimated duration, queued playout, and the final subscriber hold time for diagnostics.
The backend keeps a Gemini Live path for future model availability, but the verified deployment path uses TTS.

What LiveKit does NOT do in PodMan

No video tracks from Hermes
No mic transcription (not needed for v1)
No SFU mixing — standard room behavior is sufficient

3.5 KiB Raw Blame History