LiveKit Integration Spec

LiveKit is the real-time backbone for PodMan. It handles room presence and voice delivery. It is load-bearing — not decorative.

Room structure

One LiveKit room per project pod: room = podId
Engineers join as named participants (e.g. alice, bob)
Hermes joins as podman-hermes
All participants stay connected for the duration of the session

Engineer side (PWA)

Joining:

PWA calls POST /pods/:podId/token → receives { token, url }
LiveKit client connects to the room with the token
PWA publishes screen track via getDisplayMedia
PWA sets mic enabled for ambient presence

Receiving:

LiveKit client automatically receives Hermes audio track
No special subscription needed — LiveKit delivers audio to all participants
PWA also listens for data channel messages from Hermes for UI card updates

Data channel listener (PWA):

room.on(RoomEvent.DataReceived, (payload, participant) => {
  if (participant?.identity !== 'podman-hermes') return;
  const nudge = JSON.parse(new TextDecoder().decode(payload));
  // nudge: { type, message, involvedEngineers, file, sentAt }
  appendNudgeToFeed(nudge);
});

Hermes side (PodMan LiveKit participant)

Framework: @livekit/rtc-node

Startup:

Hermes mints its own token via the same createPodToken function with identity: 'podman-hermes'
Connects to the configured room as podman-hermes
Publishes data-channel cards/messages and short Gemini TTS audio tracks

Voice delivery:

Nudge message text is ready (from Gemini text generation)
Hermes sends a natural-speaking prompt to Gemini TTS
Gemini returns PCM audio using the configured voice
Hermes publishes the audio as a short LiveKit track
All participants hear it

Data channel message (sent alongside audio):

const nudge = {
  type: 'DEPENDENCY_READY' | 'BLOCKER_DETECTED' | 'DUPLICATE_WORK',
  message: string,        // the spoken text
  involvedEngineers: string[],
  file: string | null,
  sentAt: string,         // ISO timestamp
};
room.localParticipant.publishData(
  new TextEncoder().encode(JSON.stringify(nudge)),
  { reliable: true }
);

Token endpoint

Already implemented at POST /api/token.

Hermes uses the same endpoint. Grants:

roomJoin: true
canPublish: true (for audio track)
canPublishData: true (for data channel)
canSubscribe: true

Gemini voice model

Model ID: gemini-3.1-flash-tts-preview
Default voice: Charon (GEMINI_TTS_VOICE)
Hermes generates Gemini TTS audio and publishes it as a LiveKit audio track.
The backend keeps a Gemini Live path for future model availability, but the verified deployment path uses TTS.

What LiveKit does NOT do in PodMan

No video tracks from Hermes
No mic transcription (not needed for v1)
No SFU mixing — standard room behavior is sufficient

2.9 KiB Raw Blame History