LiveKit Integration Spec

LiveKit is the real-time backbone for PodMan. It handles room presence and voice delivery. It is load-bearing — not decorative.

Room structure

One LiveKit room per project pod: room = podId
Engineers join as named participants (e.g. alice, bob)
Hermes joins as podman-hermes
All participants stay connected for the duration of the session

Engineer side (PWA)

Joining:

PWA calls POST /pods/:podId/token → receives { token, url }
LiveKit client connects to the room with the token
PWA publishes screen track via getDisplayMedia
PWA sets mic enabled for ambient presence

Receiving:

LiveKit client automatically receives Hermes audio track
No special subscription needed — LiveKit delivers audio to all participants
PWA also listens for data channel messages from Hermes for UI card updates

Data channel listener (PWA):

room.on(RoomEvent.DataReceived, (payload, participant) => {
  if (participant?.identity !== 'podman-hermes') return;
  const nudge = JSON.parse(new TextDecoder().decode(payload));
  // nudge: { type, message, involvedEngineers, file, sentAt }
  appendNudgeToFeed(nudge);
});

Hermes side (LiveKit Agent)

Framework: LiveKit Agents (Node.js)

Startup:

Hermes mints its own token via the same createPodToken function with identity: 'podman-hermes'
Connects to the configured room as podman-hermes
Registers as a LiveKit Agent with Gemini Live 2.5 as voice provider

Voice delivery:

Nudge message text is ready (from Gemini text generation)
Hermes passes text to Gemini Live 2.5 via LiveKit Agents voice pipeline
Audio streams into the room in real-time
All participants hear it

Data channel message (sent alongside audio):

const nudge = {
  type: 'DEPENDENCY_READY' | 'BLOCKER_DETECTED' | 'DUPLICATE_WORK',
  message: string,        // the spoken text
  involvedEngineers: string[],
  file: string | null,
  sentAt: string,         // ISO timestamp
};
room.localParticipant.publishData(
  new TextEncoder().encode(JSON.stringify(nudge)),
  { reliable: true }
);

Token endpoint

Already implemented at POST /api/token.

Hermes uses the same endpoint. Grants:

roomJoin: true
canPublish: true (for audio track)
canPublishData: true (for data channel)
canSubscribe: true

Gemini voice model

Model ID: gemini-3.1-flash-tts-preview
Hermes generates Gemini TTS audio and publishes it as a LiveKit audio track.
The backend keeps a Gemini Live path for future model availability, but the verified deployment path uses TTS.

What LiveKit does NOT do in PodMan

No video tracks from Hermes
No mic transcription (not needed for v1)
No SFU mixing — standard room behavior is sufficient

2.8 KiB Raw Permalink Blame History