OSS/podman
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge origin/main into feat/graph-data-viz

Browse Source 
Resolve conflicts:
- backend/src/server.ts: keep main's getPresence + closeRoom (room auto-cleanup)
  and add the graph imports/routes (additive).
- frontend/src/App.tsx: take main's shadcn command-center UI and re-integrate
  GraphView — import, graphPodId state, an early-return render branch, and a
  shadcn "Team memory" button next to Refresh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
sb-iam
2026-06-27 18:41:56 -07:00
parent 32bff6a3ab 5185845090
commit eedfa935b1
113 changed files with 13878 additions and 493 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/demo-setup.md
+18 -12
View File
@@ -18,18 +18,21 @@ Pre-stage checklist for the 3-minute live demo. Do this on all 3 laptops before
## Laptop setup (all 3 machines)
### Editor settings
- Font size: **18pt or larger** — Gemini Vision must read file names and code
- Single editor window — no split panes, no overlapping terminals
- File tab visible with full file name shown (not truncated)
- Light or dark theme is fine — avoid low-contrast themes
### Browser
- Chrome (best `getDisplayMedia` support)
- PWA tab open and joined to `demo-pod`
- Earbuds / headphones plugged in and tested
- Volume: medium — PodMan voice should be clearly audible but not startle
### Screen layout
- Editor takes 2/3 of screen
- Terminal takes bottom 1/3 (always visible)
- No other windows on top
@@ -41,14 +44,17 @@ Pre-stage checklist for the 3-minute live demo. Do this on all 3 laptops before
Pre-create these files in the demo repo before the demo:
**Alice's machine:**
- Open `auth/middleware.ts` — has visible function stubs
- Terminal shows nothing running initially, then `Server running on :3001` at the right moment
**Bob's machine:**
- Open `frontend/login.tsx` — has visible form component code
- Terminal idle
**Carol's machine:**
- Open `frontend/integration.ts` or similar
- Terminal shows: `curl http://localhost:3001/auth` → `curl: (7) Failed to connect`
@@ -56,18 +62,18 @@ Pre-create these files in the demo repo before the demo:
## Demo script timing
| Time | Action | Who |
|---|---|---|
| 0:00 | All three join `demo-pod` | All |
| 0:05 | PodMan greets by voice | Hermes auto |
| 0:20 | Alice opens `auth/middleware.ts`, starts typing | Alice |
| 0:45 | Bob opens `frontend/login.tsx` | Bob |
| 0:50 | Carol runs `curl` command, sees error | Carol |
| ~1:20 | BLOCKER_DETECTED nudge fires | Hermes auto |
| 1:50 | Alice starts her server (`node server.js`) | Alice |
| ~2:00 | DEPENDENCY_READY nudge fires | Hermes auto |
| 2:20 | Optional: show session 2 ownership warm-start | Presenter |
| 2:45 | Close | Presenter |
| Time | Action | Who |
| ----- | ----------------------------------------------- | ----------- |
| 0:00 | All three join `demo-pod` | All |
| 0:05 | PodMan greets by voice | Hermes auto |
| 0:20 | Alice opens `auth/middleware.ts`, starts typing | Alice |
| 0:45 | Bob opens `frontend/login.tsx` | Bob |
| 0:50 | Carol runs `curl` command, sees error | Carol |
| ~1:20 | BLOCKER_DETECTED nudge fires | Hermes auto |
| 1:50 | Alice starts her server (`node server.js`) | Alice |
| ~2:00 | DEPENDENCY_READY nudge fires | Hermes auto |
| 2:20 | Optional: show session 2 ownership warm-start | Presenter |
| 2:45 | Close | Presenter |
---
docs/digitalocean.md
+1
View File
@@ -114,6 +114,7 @@ static_sites:
## Fallback plan (if App Platform deploy fails on stage)
Run Hermes locally:
```bash
cd backend && pnpm dev
```
docs/gemini.md
+3
View File
@@ -104,6 +104,7 @@ Respond with the message text only.
```
**Example output:**
> "Carol — Alice just got the auth endpoint running. You're clear to integrate."
---
@@ -115,6 +116,7 @@ Respond with the message text only.
**Integration:** LiveKit Agents framework — Hermes runs as a LiveKit Agent with Gemini Live 2.5 as the voice provider
**Flow:**
1. Nudge message text generated (step 3)
2. Hermes passes text to Gemini Live via LiveKit Agents
3. Gemini Live streams audio back in real-time
@@ -122,6 +124,7 @@ Respond with the message text only.
5. All participants hear it through their audio output
**Why Gemini Live (not plain TTS):**
- Streams audio directly — no intermediate WAV file conversion
- Latency ~300–500ms from text to first audio packet
- Natural-sounding voice
docs/idea.md
+5 -5
View File
@@ -86,8 +86,8 @@ Each engineer opens a browser PWA on their laptop. The PWA captures a screen fra
## Prize alignment
| Prize | How PodMan earns it |
|---|---|
| Best Gemini 3.5 / 2.5 | Gemini Vision for screen understanding + Gemini Live 2.5 for voice output |
| Best LiveKit | LiveKit is the real-time backbone for room presence and voice delivery — load-bearing, not decorative |
| Best DigitalOcean | Hermes deployed on DigitalOcean App Platform; MongoDB Atlas on DO-adjacent infrastructure |
| Prize | How PodMan earns it |
| --------------------- | ----------------------------------------------------------------------------------------------------- |
| Best Gemini 3.5 / 2.5 | Gemini Vision for screen understanding + Gemini Live 2.5 for voice output |
| Best LiveKit | LiveKit is the real-time backbone for room presence and voice delivery — load-bearing, not decorative |
| Best DigitalOcean | Hermes deployed on DigitalOcean App Platform; MongoDB Atlas on DO-adjacent infrastructure |
docs/livekit.md
+7
View File
@@ -16,17 +16,20 @@ LiveKit is the real-time backbone for PodMan. It handles room presence and voice
## Engineer side (PWA)
**Joining:**
1. PWA calls `POST /pods/:podId/token` → receives `{ token, url }`
2. LiveKit client connects to the room with the token
3. PWA publishes screen track via `getDisplayMedia` (used client-side for frame capture — Hermes does NOT subscribe to this track)
4. PWA sets mic enabled for ambient presence
**Receiving:**
- LiveKit client automatically receives Hermes audio track
- No special subscription needed — LiveKit delivers audio to all participants
- PWA also listens for data channel messages from Hermes for UI card updates
**Data channel listener (PWA):**
```ts
room.on(RoomEvent.DataReceived, (payload, participant) => {
if (participant?.identity !== 'podman-hermes') return;
@@ -43,17 +46,20 @@ room.on(RoomEvent.DataReceived, (payload, participant) => {
**Framework:** LiveKit Agents (Node.js)
**Startup:**
1. Hermes mints its own token via the same `createPodToken` function with `identity: 'podman-hermes'`
2. Connects to the room on pod creation / first engineer joining
3. Registers as a LiveKit Agent with Gemini Live 2.5 as voice provider
**Voice delivery:**
1. Nudge message text is ready (from Gemini text generation)
2. Hermes passes text to Gemini Live 2.5 via LiveKit Agents voice pipeline
3. Audio streams into the room in real-time
4. All participants hear it
**Data channel message (sent alongside audio):**
```ts
const nudge = {
type: 'DEPENDENCY_READY' | 'BLOCKER_DETECTED' | 'DUPLICATE_WORK',
@@ -75,6 +81,7 @@ room.localParticipant.publishData(
Already implemented at `POST /pods/:podId/token`.
Hermes uses the same endpoint. Grants:
- `roomJoin: true`
- `canPublish: true` (for audio track)
- `canPublishData: true` (for data channel)
docs/mongodb.md
+1
View File
@@ -63,6 +63,7 @@ Tracks who works on which files. Built up over the session. **Persists across se
**Index:** `{ podId: 1, file: 1 }` (unique)
**Upsert logic:**
- On each context update where `currentFile` is non-null:
- Increment `observationCount`
- Update `primaryOwner` to the engineer with the most recent `lastSeenAt` on this file
docs/superpowers/specs/2026-06-27-podman-design.md
+17 -11
View File
@@ -46,6 +46,7 @@ PodMan is a real-time AI team coordination agent for software teams. Engineers j
## Components
### PWA (local agent)
- Joins LiveKit room via existing `joinPod` flow
- Captures frame every 30s via `getDisplayMedia`, compresses to JPEG (1280×720, quality 0.7)
- POSTs `{ engineerId, podId, screenshotBase64, capturedAt }` to `POST /ingest`
@@ -54,6 +55,7 @@ PodMan is a real-time AI team coordination agent for software teams. Engineers j
- Two screens: join screen (built), active session screen (to build)
### Hermes (orchestrator)
- Express server + LiveKit Agent on DigitalOcean
- `POST /ingest`: receives frame, queues for vision
- Vision pipeline: Gemini 2.0 Flash → `EngineerContext`
@@ -66,18 +68,21 @@ PodMan is a real-time AI team coordination agent for software teams. Engineers j
- Cooldown: 3 min between nudges per pod
### Gemini usage
- **Vision:** `gemini-2.0-flash` — screen → `{ currentFile, inferredTask, terminalVisible, recentTerminalOutput, confidence }`
- **Event detection:** `gemini-2.0-flash` — all engineer states → `{ event, involvedEngineers, file, reason }`
- **Nudge generation:** `gemini-2.0-flash` — event → spoken message text
- **Voice:** `gemini-live-2.5-flash` via LiveKit Agents — text → streaming audio
### MongoDB Atlas (4 collections)
- `engineer_states`: latest context per engineer, upserted each ingest
- `ownership_map`: file → primaryOwner + contributors, persists across sessions (continual learning)
- `events`: all detected coordination events
- `nudges`: all voice nudges sent + cooldown history
### LiveKit
- One room per pod
- Engineers publish screen track (used client-side for capture — Hermes does not subscribe)
- Hermes joins as `podman-hermes`, publishes audio + data channel messages
@@ -87,17 +92,18 @@ PodMan is a real-time AI team coordination agent for software teams. Engineers j
## Event types
| Event | Trigger | Example nudge |
|---|---|---|
| Event | Trigger | Example nudge |
| ------------------ | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| `BLOCKER_DETECTED` | Engineer stuck (error in terminal, same file N frames) + teammate can help | "Carol, looks like you're waiting on auth. Alice is actively building it — hang tight." |
| `DEPENDENCY_READY` | Engineer A completes work that Engineer B was waiting on | "Carol, Bob — Alice just got the auth endpoint running. You're clear to integrate." |
| `DUPLICATE_WORK` | 2+ engineers on same file simultaneously | "Alice and Bob — you're both in login.tsx. Coordinate before pushing." |
| `DEPENDENCY_READY` | Engineer A completes work that Engineer B was waiting on | "Carol, Bob — Alice just got the auth endpoint running. You're clear to integrate." |
| `DUPLICATE_WORK` | 2+ engineers on same file simultaneously | "Alice and Bob — you're both in login.tsx. Coordinate before pushing." |
---
## Continual learning story
The `ownership_map` collection persists across sessions. On Hermes startup:
1. Load ownership map for this pod from Atlas
2. Build in-memory cache: `Map<file, { primaryOwner, contributors }>`
3. Event detection uses priors immediately — no ramp-up phase
@@ -120,10 +126,10 @@ The `ownership_map` collection persists across sessions. On Hermes startup:
## Key risks
| Risk | Mitigation |
|---|---|
| Gemini Vision accuracy | Large font, single editor window, confidence gate |
| Gemini Live 2.5 + LiveKit Agents integration | Build together hour 5–7, have TTS fallback |
| Frame POST latency | JPEG compression, target < 500ms |
| Event false positives | 3-min cooldown, pre-staged demo |
| DO deploy failure | Hermes runs local, PWA defaults to localhost:8787 |
| Risk | Mitigation |
| -------------------------------------------- | ------------------------------------------------- |
| Gemini Vision accuracy | Large font, single editor window, confidence gate |
| Gemini Live 2.5 + LiveKit Agents integration | Build together hour 5–7, have TTS fallback |
| Frame POST latency | JPEG compression, target < 500ms |
| Event false positives | 3-min cooldown, pre-staged demo |
| DO deploy failure | Hermes runs local, PWA defaults to localhost:8787 |