docs: fix mangled README title
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
This commit is contained in:
Ramis
@@ -1,9 +1,9 @@
|
||||
# PodMan — A Pair Programmer for Engineering Teams
|
||||
|
||||
[](https://livekit.io/)
|
||||
[](https://www.mongodb.com/)
|
||||
[](https://ai.google.dev/)
|
||||
[](https://www.digitalocean.com/)
|
||||
[LiveKit](https://livekit.io/)
|
||||
[MongoDB](https://www.mongodb.com/)
|
||||
[Gemini](https://ai.google.dev/)
|
||||
[DigitalOcean](https://www.digitalocean.com/)
|
||||
|
||||
**2026 AI Engineer World's Fair Hackathon — Theme: Continual Learning**
|
||||
|
||||
@@ -47,6 +47,8 @@ time.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## How it learns
|
||||
|
||||
The learning loop is the product, not a side feature. It runs with almost no
|
||||
@@ -58,8 +60,9 @@ observe → detect → RECALL prior outcomes → policy gate → act → record
|
||||
└──────────────────────────── feeds next recall ───────────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
| Stage | What happens | Code |
|
||||
| --- | --- | --- |
|
||||
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
|
||||
| **Observe** | Gemini Vision turns each screen frame into structured work context (file, symbol, activity, unpushed hints) | `backend/src/vision/gemini.ts` |
|
||||
| **Detect** | Same file touched by 2+ engineers with unpushed work → a coordination event | `backend/src/collision/detector.ts` |
|
||||
| **Recall (memory)** | Embed the event, query MongoDB Atlas `$vectorSearch` for similar past events, attach their prior intervention + outcome | `backend/src/memory/vectors.ts` |
|
||||
@@ -67,15 +70,16 @@ observe → detect → RECALL prior outcomes → policy gate → act → record
|
||||
| **Act (least intrusive)** | Reuse the action kind that was accepted before; default to a card, escalate to a Hermes message, voice only when urgent | `backend/src/action/hermes.ts` |
|
||||
| **Record (feedback)** | Accept/dismiss + "was it real?" is written back to memory, closing the loop for next time | `backend/src/memory/store.ts` |
|
||||
|
||||
|
||||
A few things make this real learning rather than a static prompt:
|
||||
|
||||
- It adapts from real teammate behavior during a real session, not an offline
|
||||
dataset.
|
||||
dataset.
|
||||
- It gets more useful as the `outcomes` collection grows — better recall, fewer
|
||||
false alarms.
|
||||
false alarms.
|
||||
- It needs one tap. No labeling, no config, no retraining.
|
||||
- The mechanism is memory: Atlas vector recall plus an outcome-conditioned
|
||||
policy, with an exact-signature fallback when vector search isn't available.
|
||||
policy, with an exact-signature fallback when vector search isn't available.
|
||||
|
||||
In practice: a false alarm gets dismissed once, and the same pattern stays quiet
|
||||
next time. A real conflict gets accepted once, and when it recurs PodMan recalls
|
||||
@@ -83,6 +87,8 @@ it and escalates straight to a spoken "seen before" cue.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
A browser PWA, an HTTP API service, LiveKit agent workers, and a
|
||||
@@ -138,10 +144,15 @@ flowchart LR
|
||||
API --> Mongo
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Runtime shape
|
||||
|
||||
|
||||
| Layer | Runtime | Responsibility |
|
||||
| --- | --- | --- |
|
||||
| ----------------------- | ----------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| Frontend PWA | React + Vite | Join pods, publish screen share, render interventions, play audio |
|
||||
| Backend API | Express | Mint LiveKit tokens, manage pods, record outcomes, expose memory stats, create sync PRs |
|
||||
| Vision agent | `@livekit/rtc-node` | Subscribe to screen-share tracks, sample frames, publish intervention data |
|
||||
@@ -152,6 +163,9 @@ flowchart LR
|
||||
| Action layer | Hermes | Cards, teammate messages, Gemini TTS urgent voice, Lyria background score |
|
||||
| Deployment | DigitalOcean | Static frontend, API service, agent workers |
|
||||
|
||||
|
||||
|
||||
|
||||
### Data flow
|
||||
|
||||
```mermaid
|
||||
@@ -181,22 +195,28 @@ sequenceDiagram
|
||||
API->>Mongo: Store learning signal
|
||||
```
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Built with
|
||||
|
||||
Everything below maps to code in this repo.
|
||||
|
||||
**Gemini** does the perception, the voice, and the memory:
|
||||
|
||||
|
||||
| Use | Model | Where |
|
||||
| --- | --- | --- |
|
||||
| --------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------ |
|
||||
| Real-time voice agent (talk to PodMan, answered with repo/git/memory tools) | `gemini-3.1-flash-live-preview` | `agents/podman-live-conversation/agent.py` |
|
||||
| Spoken urgent alerts over LiveKit | `gemini-3.1-flash-tts-preview` | `backend/src/voice/live.ts` |
|
||||
| Screen understanding → structured work context | `gemini-2.0-flash` | `backend/src/vision/gemini.ts` |
|
||||
| Per-pod background music (Interactions API) | `lyria-3-clip-preview` | `backend/src/voice/music.ts` |
|
||||
| Embeddings for memory recall | `gemini-embedding-001` | `backend/src/memory/vectors.ts` |
|
||||
|
||||
|
||||
**LiveKit** is the real-time layer: screen-share tracks are the input, a typed
|
||||
data channel (`podman.intervention`) carries cards and messages, audio tracks
|
||||
carry the spoken alerts and music, and a Python LiveKit Agents worker runs the
|
||||
@@ -212,6 +232,8 @@ fail loudly rather than degrade silently.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## How it works
|
||||
|
||||
1. Engineers open the PWA and join a pod room.
|
||||
@@ -236,10 +258,13 @@ in the repo — answered with real tool calls, not guesses.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Public interfaces
|
||||
|
||||
|
||||
| Interface | Purpose |
|
||||
| --- | --- |
|
||||
| ----------------------------------------------------------- | ------------------------------------------------- |
|
||||
| `GET /health` | API health check |
|
||||
| `POST /api/token` | Mint LiveKit room tokens |
|
||||
| `POST /api/sync-pr` | Create a visible sync PR artifact |
|
||||
@@ -250,12 +275,16 @@ in the repo — answered with real tool calls, not guesses.
|
||||
| LiveKit topic `podman.intervention` | Intervention data channel |
|
||||
| Wire messages `COLLISION`, `ACK`, `GIT_REPORT`, `VOICE_CUE` | Agent/PWA contract |
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Monorepo layout
|
||||
|
||||
|
||||
| Folder | What |
|
||||
| --- | --- |
|
||||
| ----------- | -------------------------------------------------------------------------- |
|
||||
| `frontend/` | React + Vite PWA — pods, LiveKit room UI, screen share, intervention cards |
|
||||
| `backend/` | Express API plus the LiveKit vision agent worker |
|
||||
| `agents/` | Python LiveKit Agents worker for the Gemini Live conversation agent |
|
||||
@@ -265,8 +294,11 @@ in the repo — answered with real tool calls, not guesses.
|
||||
| `scripts/` | Local git watcher + deploy/verify tooling |
|
||||
| `docs/` | Integration specs and the demo script |
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
@@ -283,6 +315,8 @@ The live conversation agent (Gemini Live API) runs from `agents/podman-live-conv
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Git watcher — run on every demo laptop
|
||||
|
||||
Each engineer runs this before the demo. It polls the local git working tree
|
||||
|
||||
Reference in New Issue
Block a user