22 Commits

Author SHA1 Message Date
sb-iam c0fe424780 docs: add visual HTML demo runbook (ascii architecture + RSI-loop diagrams)
Self-contained one-pager: how-it-works ASCII diagrams (data flow + continual-
learning loop), the 3-min demo timeline, sponsor coverage, last-hour caveats
(suppression disabled, don't redeploy prod), and recovery. Demo-branch only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 13:11:27 -05:00
sb-iam 1a592abb39 docs: add final demo runbook (demo-branch only; not for main)
Reflects the current build: suppression disabled (Ramis), so the demo uses the
'Seen before.' escalate beat + learned_from edge, not the dead 'stays quiet' beat.
Kept on the demo branch so main stays clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 13:04:35 -05:00
Yahya Alhinai 454fdab13b chore(box): snapshot in-progress auth + user-learning WIP before deploy
Captures uncommitted work present on the live droplet so origin/main can be integrated and the new control toolbar deployed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 17:51:21 +00:00
Kartikeya 63a411490c feat(ui): compact, tooltip-driven pod control toolbar
Collapse the five full-width control buttons (audio, mic, background
music, test voice, share) into a compact icon pill with hover/focus
tooltips, plus a labelled Share primary CTA. Row now flex-wraps instead
of a fixed 2-col grid, so it stays on one line on desktop and wraps
cleanly on narrow screens. State is encoded visually (filled = engaged,
ghost = idle, primary = needs action). Background music gets a distinct
Music2 icon instead of reusing the audio Volume2 icon. Handlers are
unchanged — presentation only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 10:44:07 -07:00
Kartikeya faa9769470 fix(intervention): clear stale voice cue/message on new card and dismiss
voiceCue and hermes were never reset, so a new intervention card could show the previous one's spoken line. Clear both when a COLLISION arrives and on respond(); safe against the reliable publish order COLLISION -> HERMES_MESSAGE -> VOICE_CUE.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 10:34:31 -07:00
Kartikeya 79620bccaa fix(collision): treat case/whitespace-variant identities as one person
Vision reports a display name ("Karti") while the git watcher reports a handle ("karti"); the detectors compared engineerId case-sensitively, so the same human collided with themselves. Canonicalize identity at the decision point in both the file and research detectors, keeping the display-cased name for the card.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 10:34:31 -07:00
Ramis d8eaf8d2e2 feat(voice): speak every intervention, not just critical escalations
Voice was gated to severity === 'critical', so most conflict cards were
silent. Now every intervention is spoken: priority is derived from
severity in publishHermesIntervention — critical jumps the queue,
the rest serialize so concurrent alerts don't garble each other.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 10:31:35 -07:00
Ramis e6e7a42d90 fix(agent): re-alert persistent + new-pair conflicts (README always surfaces)
The edge-trigger single-shot keyed conflicts by file basename only and
re-armed solely when the conflict left a detection cycle. README is dirty
for someone in the pod the whole session, so its key never re-armed: it
alerted once (ram+karti at restart) and stayed muted even as the colliding
pair changed to karti+Ramis Hasanli. Transient files (research, new files)
churned out and back, so only those kept firing.

- conflictKey now includes the sorted engineer pair, so a different pair on
  the same file re-alerts immediately.
- activeConflicts is now Map<key, lastAlertedAt>; a persistent conflict
  re-alerts every CONFLICT_REALERT_MS (default 20s) instead of going silent
  forever, without spamming once-per-frame.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 10:25:10 -07:00
sb-iam 9658d700fa fix(graph): harden suppressed-repeat activity (Codex deep review) (#44)
Four demo-safety fixes on top of #43, none touching demo-pod data:

- De-dup at source: the suppression path returned before activeConflicts.add(),
  so the same unresolved dismissed collision wrote a SuppressionDoc every frame
  (~194 spam rows observed on prod). Now mark the conflict handled first, so it
  records ONCE per recurrence and re-arms via the onScreenFrame resolution sweep.
- Await the write: recordSuppression is the visible learning proof, so await it
  (like recordCollision/recordIntervention) instead of void ...catch().
- Display de-dup: collapse suppressed beats to one per file (most recent) with a
  stable per-file id, so any pre-fix duplicate rows never render as spam.
- Live-graph gate: suppression beats now satisfy the "has activity" check in
  materializePodGraph, so a clean pod with preserved suppressions (but no
  collision/file nodes) no longer falls back to the demo graph and hides the proof.
- Ops: /api/memory/stats counts `suppressions`; docs/mongodb.md documents the
  collection and flags it "preserve in DB cleanup" (visible learning evidence).

Verified end-to-end on a throwaway pod (not demo-pod): suppression-only pod
materializes; 2 dupe rows render as 1 beat. backend+frontend typecheck + eslint pass.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 10:18:10 -07:00
Ramis 06098f3c4e fix(policy): disable intervention suppression for demo
Dismissing one collision permanently silenced all future ones: the
dismissal gate (priorOutcome.accepted === false) plus loose file/vector
recall meant one README discard muted every later README clash. The
per-pod cooldown additionally hid consecutive conflicts for 3 min.

shouldIntervene now only filters `info` severity — Accept/Dismiss still
record outcomes but no longer gate whether a collision surfaces. The
same-collision single-shot dedupe (activeConflicts in PodmanAgent.handle)
still prevents repeat spam, so removing the cooldown is safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 10:12:51 -07:00
sb-iam 65764f0a1c feat(graph): record + surface suppressed-repeat activity at repeat time (Feature A) (#43)
Codex review fix. The earlier approach synthesized a 'suppressed' beat from every
dismissed OUTCOME, which is wrong: a dismissal is not a later suppressed repeat,
and the old dismissal timestamp sank below the 12-row activity cap (invisible).

Now the negative-feedback loop is recorded WHEN IT HAPPENS. When shouldIntervene()
returns false specifically because a signature was dismissed before
(agent/podman.ts), the agent writes a durable SuppressionDoc to a new
`suppressions` collection, timestamped at the repeat. graph/live.ts materializes
those into kind:'suppressed' activity (recent -> surfaces at the top). The
suppressed collision is never written to `collisions` (agent returns before
recordCollision), so this is its own record.

Verified end-to-end: a transient demo-pod suppression renders at the TOP of the
activity stream; cleaned up after. shared build + backend/frontend typecheck +
eslint pass.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 09:35:56 -07:00
Ramis b6447bb12c docs: surface Gemini in vision agent node
Make it explicit in the infra diagram that the vision agent worker uses
Gemini to read each screen frame.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 08:18:53 -07:00
Ramis f7521fb5df docs: group engineer-browser edges in infra diagram
Fold the cards/alerts return path into the bidirectional Dev<->LiveKit
edge so the browser shows two grouped routes (app+token, combined live
channel) instead of three.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 08:18:09 -07:00
Ramis 91403abe9f docs: simplify infra diagram to value-focused paths
Combine Caddy+API and Hermes+detector into single nodes, collapse the
LiveKit room internals, and relabel every edge with the value it carries
(pre-commit signal, nudge before clash, learn from outcomes) instead of
listing every transport hop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 08:16:15 -07:00
Ramis f4ad3bec63 docs: split architecture graph into infra + Gemini diagrams
Replace the single all-in-one architecture flowchart with two focused
diagrams: (1) how LiveKit, Hermes, MongoDB, and DigitalOcean wire
together, and (2) Gemini's five surfaces with their triggers, models,
and call sites.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 08:11:43 -07:00
Ramis 9fb252ec6a feat: add coordination-ROI band to member Work History
Surfaces "rework saved" at the top of the teammate Work History dialog:
a transparent heuristic over collisions Hermes caught early (eligible =
gitOverlap or critical, with an intervention), credit split across
involved engineers. Shows hard counts (clashes caught, conflict-free
files) plus an info-tooltip breakdown of the estimate.

- shared: optional MemberWorkHistoryRoi field (back-compat, self-zeroes)
- backend: query collisions + interventions in getMemberWorkHistory,
  computeRoi helper
- frontend: RoiBand + RoiTooltip components, hidden when no clashes

Implements docs/plans/work-history-roi.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 07:51:40 -07:00
Ramis 8d17663734 docs: add Work History coordination-ROI build plan
Build-ready spec for adding a "rework saved" ROI band to the member
Work History dialog: transparent heuristic over caught collisions,
additive shared field + backend queries + presentational component.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 07:46:47 -07:00
Yahya Alhinai 3cb66e3f27 Simplify frontend room surfaces 2026-06-28 14:24:04 +00:00
Yahya Alhinai dec6557cad Add research overlap nudges 2026-06-28 14:21:29 +00:00
Ramis 50cc4a2900 docs: add research-overlap build spec; switch git policy to push-to-main
- docs/plans/research-overlap.md: self-contained build spec for code-edit
  vs research cross-channel overlap nudge (vision classifies editing vs
  research, semantic embedding match, collaboration nudge). Ready for an
  implementing agent.
- CLAUDE.md: replace branch-first habit with push-directly-to-main policy
  plus mandatory pull --rebase before push for concurrent teammates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 07:06:24 -07:00
Ramis H. d1570a44d6 Remove hackathon theme from README
Removed the hackathon theme section from the README.
2026-06-28 06:43:32 -07:00
Ramis 9459203ffc Merge docs/cleanup-and-positioning: reposition + spec sync + doc cleanup
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuV8W8oNYRsDWKoqK8Mkqc
2026-06-28 06:41:44 -07:00
50 changed files with 6205 additions and 681 deletions
+3
View File
@@ -26,13 +26,16 @@ VOYAGE_EMBEDDING_MODEL=voyage-4-lite
# --- Backend server ---
PORT=8787
POD_ROOM=demo-pod
CLERK_SECRET_KEY=
# --- Nudge cooldown (ms) — set to 0 during demo if needed ---
NUDGE_COOLDOWN_MS=180000
RESEARCH_OVERLAP_THRESHOLD=0.6
# --- Frontend (Vite — must be VITE_ prefixed to reach the client) ---
VITE_LIVEKIT_URL=wss://your-project.livekit.cloud
VITE_BACKEND_URL=http://localhost:8787
VITE_CLERK_PUBLISHABLE_KEY=
# Keep off by default so users hear Gemini audio delivered through LiveKit.
VITE_ENABLE_BROWSER_TTS_FALLBACK=false
+1
View File
@@ -41,4 +41,5 @@ __pycache__/
# Ramis
.remember/
.claude/
.hermes/
.playwright-mcp/
+14
View File
@@ -365,6 +365,20 @@ This repo is actively used by **4 engineers at the same time**. Claude sessions
- **Prefer additive changes** — new files, new functions — over modifying existing ones. This minimizes merge conflicts in a concurrent team.
- **When proposing new files**, verify they match the file names and paths described in the relevant spec in `docs/`. Do not invent new paths.
### Git workflow — push directly to `main`, stay in sync
This repo **does not use feature branches**. Commit straight to `main` and push.
There is no branch-first step. Because several people push concurrently:
- **Always sync before pushing:** `git pull --rebase origin main` immediately
before `git push`. Never force-push `main`.
- **Keep commits small and additive** so rebases stay clean — prefer new files
and new functions over editing shared hot files (`backend/src/agent/podman.ts`,
`backend/src/server.ts`, `frontend/src/App.tsx`).
- **If a rebase conflicts**, resolve it locally and re-run the pull-rebase before
pushing; do not overwrite a teammate's commit.
- Commit/push only when the user asks (overrides any default branch-first habit).
---
## Production deployment & ops — READ BEFORE TOUCHING THE SERVER
+480 -265
View File
@@ -1,172 +1,263 @@
# PodMan — A Pair Programmer for Engineering Teams
# PodMan
[LiveKit](https://livekit.io/)
[MongoDB](https://www.mongodb.com/)
[Gemini](https://ai.google.dev/)
[DigitalOcean](https://www.digitalocean.com/)
**An ambient pair programmer for engineering teams.**
**2026 AI Engineer World's Fair Hackathon — Theme: Continual Learning**
PodMan watches the work happening inside a shared LiveKit room, understands what
each engineer is doing, remembers which interventions helped, and nudges the
team before duplicated work, merge collisions, or missed handoffs slow everyone
down.
## The bottleneck moved
[LiveKit](https://livekit.io/) · [MongoDB](https://www.mongodb.com/) ·
[Gemini](https://ai.google.dev/) · [Hermes](https://hermes-agent.nousresearch.com/) ·
[vLLM](https://vllm.ai/) · [DigitalOcean](https://www.digitalocean.com/)
Models keep getting better, and more people can build software than ever before.
Writing the code is no longer the hard part — engineering ability is not the
ceiling anymore.
## Read This First
What slows teams down now is everything *around* the code: manually checking each
other's work, re-planning when two people drift into the same change, and
constantly asking "what are you working on?" just to stay in sync. People
naturally want to build together — so as more people start coding, there will be
thousands of teams bottlenecked not by skill, but by the **speed of human
coordination.**
**PodMan is a pair programmer for the whole team.** It watches what every member
is doing in real time, learns your team's decisions and working dynamics, and
gives everyone a live picture of where the others are — without anyone having to
stop and ask.
### The five-minute meeting that isn't
> **What people assume:** "Quick question, five minutes."
> **What actually happens:** the interrupted developer loses their place and needs
> 1525 minutes to climb back into deep focus. That five-minute ask quietly costs
> half an hour — for *two* people.
Multiply that by every teammate, every day, and coordination overhead — not
engineering skill — becomes the real ceiling on how fast a team ships.
PodMan removes the reason to interrupt. Because it already knows who is touching
which file, what's still unpushed, and what each person is in the middle of, any
teammate can see another's status instantly — no tap on the shoulder, no standup,
no recovery tax. And it gets better as it goes: every accept or dismiss teaches
it what your team actually cares about, so it nudges less and helps more over
time.
> GitHub sees pushed work. PodMan sees work while it is still happening — and
> remembers what helped.
| Need | Use this |
| ----------------------- | --------------------------------------------------------------------------------------------- |
| Open the product | `https://podman.live` |
| Check the app | `curl https://podman.live/health` |
| Check pods and memory | `curl https://podman.live/api/pods && curl https://podman.live/api/memory/stats` |
| Use the LLM externally | Base URL `https://llm.alhinai.dev/v1`, model `gemma-4-31B-it` |
| Test Hermes | `hermes -z 'Reply with exactly: working' --provider gemma4-31b-vllm --model gemma-4-31B-it` |
| Start local development | API, vision agent, and frontend commands are in [Local Development](#local-development) |
| Debug production | Public checks first, then systemd services in [Production Operations](#production-operations) |
---
## Current Live System
| Surface | Running now | Purpose |
| ------------- | ---------------------------- | -------------------------------------- |
| Product | `https://podman.live` | Team room, screen context, cards |
| API | `https://podman.live/api/*` | Pods, tokens, outcomes, memory |
| Health | `https://podman.live/health` | Backend readiness |
| Local API | `127.0.0.1:8787` | Express service behind Caddy |
| Reasoning LLM | `https://llm.alhinai.dev/v1` | OpenAI-compatible Gemma/vLLM endpoint |
| API key | `not-needed` | Placeholder key for OpenAI clients |
| Hermes model | `gemma-4-31B-it` | 262K-context tool-using agent |
| Hermes config | `gemma4-31b-vllm` | Custom provider used by Hermes locally |
## How it learns
```mermaid
flowchart TB
Browser["Engineer Browser<br/>React + Vite PWA"]
Room["LiveKit Room<br/>screen share + audio + data messages"]
The learning loop is the product, not a side feature. It runs with almost no
extra work from anyone — the only human signal is a single accept/dismiss tap on
a card.
subgraph Droplet["DigitalOcean PodMan Droplet"]
Caddy["Caddy<br/>static app + /api proxy"]
API["Express API<br/>127.0.0.1:8787"]
Vision["Vision Agent<br/>screen frame observer"]
Voice["Live Conversation Agent<br/>Python + Gemini Live"]
Ops["Hermes Ops Timers<br/>watchdog + sync deploy"]
end
```
observe → detect → RECALL prior outcomes → policy gate → act → record outcome
└──────────────────────────── feeds next recall ───────────────────────────┘
subgraph Memory["MongoDB Atlas"]
Observations["observations"]
State["engineer_states"]
Outcomes["interventions + outcomes"]
end
subgraph Google["Google Gemini APIs"]
GeminiVision["Vision"]
GeminiVoice["Live voice + TTS"]
GeminiEmbed["Embeddings fallback"]
Lyria["Music"]
end
subgraph Reasoning["External Reasoning Endpoint"]
Tunnel["Cloudflare Tunnel<br/>llm.alhinai.dev"]
VLLM["vLLM OpenAI Server<br/>gemma-4-31B-it<br/>262144 context"]
Hermes["Hermes Agent<br/>provider: gemma4-31b-vllm"]
end
Browser --> Caddy --> API
Browser <-->|screen, audio, cards| Room
API --> Room
API --> Observations
API --> State
API --> Outcomes
Vision <-->|screen tracks| Room
Vision --> GeminiVision
Vision --> Observations
Vision --> Outcomes
Voice <-->|conversation| Room
Voice --> GeminiVoice
Vision --> GeminiEmbed
Voice --> GeminiEmbed
Ops --> Hermes --> Tunnel --> VLLM
API --> Hermes
Lyria --> Voice
classDef user fill:#e8f1ff,stroke:#3366cc,color:#0b1f44;
classDef app fill:#eef8ee,stroke:#2f8a3a,color:#123915;
classDef data fill:#fff6df,stroke:#c47f00,color:#3d2b00;
classDef ai fill:#f4edff,stroke:#805ad5,color:#2d1857;
class Browser,Room user;
class Caddy,API,Vision,Voice,Ops app;
class Observations,State,Outcomes data;
class GeminiVision,GeminiVoice,GeminiEmbed,Lyria,Tunnel,VLLM,Hermes ai;
```
| Stage | What happens | Code |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| **Observe** | Gemini Vision turns each screen frame into structured work context (file, symbol, activity, unpushed hints) | `backend/src/vision/gemini.ts` |
| **Detect** | Same file touched by 2+ engineers with unpushed work → a coordination event | `backend/src/collision/detector.ts` |
| **Recall (memory)** | Embed the event, query MongoDB Atlas `$vectorSearch` for similar past events, attach their prior intervention + outcome | `backend/src/memory/vectors.ts` |
| **Policy gate (adapt)** | A dismissed false alarm stays silent; a confirmed real catch escalates to critical; a per-pod cooldown prevents nagging | `backend/src/memory/policy.ts` |
| **Act (least intrusive)** | Reuse the action kind that was accepted before; default to a card, escalate to a Hermes message, voice only when urgent | `backend/src/action/hermes.ts` |
| **Record (feedback)** | Accept/dismiss + "was it real?" is written back to memory, closing the loop for next time | `backend/src/memory/store.ts` |
A few things make this real learning rather than a static prompt:
- It adapts from real teammate behavior during a real session, not an offline
dataset.
- It gets more useful as the `outcomes` collection grows — better recall, fewer
false alarms.
- It needs one tap. No labeling, no config, no retraining.
- The mechanism is memory: Atlas vector recall plus an outcome-conditioned
policy, with an exact-signature fallback when vector search isn't available.
In practice: a false alarm gets dismissed once, and the same pattern stays quiet
next time. A real conflict gets accepted once, and when it recurs PodMan recalls
it and escalates straight to a spoken "seen before" cue.
---
## Architecture
A browser PWA, an HTTP API service, LiveKit agent workers, and a
memory/action layer. The screen signal flows through LiveKit, never a manual
screenshot upload.
## What It Does
```mermaid
flowchart LR
subgraph Laptop["Engineer laptop"]
PWA["React PWA"]
Screen["Screen share track"]
Git["Git watcher<br/>scripts/podman-agent.mjs"]
end
subgraph Realtime["LiveKit room"]
Room["Pod room"]
Data["Data topic<br/>podman.intervention"]
Audio["Audio tracks<br/>Gemini TTS + Lyria"]
end
subgraph Backend["PodMan backend"]
API["API service<br/>/api/token /api/pods /api/outcome"]
Agent["Vision agent worker<br/>@livekit/rtc-node"]
Convo["Live conversation agent<br/>Gemini Live API (Python)"]
Vision["Gemini Vision<br/>structured JSON"]
Detector["Coordination detector<br/>collisions, blockers"]
end
subgraph Memory["Memory and actions"]
Mongo["MongoDB Atlas<br/>observations, outcomes, vectors"]
GitHub["GitHub<br/>repo state + sync PR artifact"]
Hermes["Hermes action layer<br/>cards, messages, urgent voice"]
end
PWA -->|"POST /api/token"| API
API -->|"LiveKit JWT"| PWA
PWA --> Screen
Screen --> Room
Room --> Agent
Agent --> Vision
Vision --> Detector
Git --> Mongo
Detector --> Mongo
Detector --> GitHub
Detector --> Hermes
Hermes --> Data
Hermes --> Audio
Convo --> Audio
Convo --> Mongo
Data --> PWA
Audio --> PWA
PWA -->|"POST /api/outcome"| API
API --> Mongo
A["Engineer shares screen"] --> B["Gemini extracts work context"]
B --> C["PodMan detects overlap<br/>files, symbols, research, unpushed work"]
C --> D["MongoDB recalls<br/>similar prior events"]
D --> E{"Policy gate"}
E -->|"seen false alarm"| F["stay quiet"]
E -->|"seen real collision"| G["raise urgency"]
E -->|"new useful signal"| H["show card or message"]
G --> I["Hermes / voice escalation"]
H --> J["teammate accepts or dismisses"]
I --> J
J --> K["outcome becomes future memory"]
K --> D
```
PodMan removes the reason to interrupt. It gives the team a live picture of work
in progress, then improves from every accepted or dismissed intervention.
> GitHub sees pushed work. PodMan sees work while it is still happening.
---
## The Model Stack
### Runtime shape
PodMan uses multiple AI surfaces. They are intentionally split by job.
```mermaid
flowchart LR
Work["Screen + room activity"] --> VisionRoute["Perception route"]
VoiceInput["Engineer speech"] --> ConversationRoute["Conversation route"]
OpsNeed["Ops / autonomous task"] --> HermesRoute["Reasoning route"]
MemoryNeed["Similarity search"] --> EmbedRoute["Memory route"]
Ambient["Session atmosphere"] --> MusicRoute["Audio route"]
| Layer | Runtime | Responsibility |
| ----------------------- | ----------------------------------------- | --------------------------------------------------------------------------------------- |
| Frontend PWA | React + Vite | Join pods, publish screen share, render interventions, play audio |
| Backend API | Express | Mint LiveKit tokens, manage pods, record outcomes, expose memory stats, create sync PRs |
| Vision agent | `@livekit/rtc-node` | Subscribe to screen-share tracks, sample frames, publish intervention data |
| Live conversation agent | LiveKit Agents (Python) + Gemini Live API | Real-time voice Q&A with function tools over repo, git, and memory |
| Perception | Gemini Vision (`gemini-2.0-flash`) | Sampled IDE frames → structured work context |
| Team memory | MongoDB Atlas | Observations, collisions, interventions, outcomes, vector embeddings |
| Git watcher | Node script | Report each laptop's dirty/unpushed state as ground truth |
| Action layer | Hermes | Cards, teammate messages, Gemini TTS urgent voice, Lyria background score |
| Deployment | DigitalOcean | Static frontend, API service, agent workers |
VisionRoute --> Gemini20["gemini-2.0-flash<br/>screen understanding"]
ConversationRoute --> GeminiLive["gemini-3.1-flash-live-preview<br/>live Q&A"]
ConversationRoute --> GeminiTTS["gemini-3.1-flash-tts-preview<br/>urgent speech"]
HermesRoute --> Gemma["gemma-4-31B-it<br/>Hermes + vLLM + tools"]
EmbedRoute --> Voyage["voyage-4-lite<br/>primary embeddings"]
EmbedRoute --> GeminiEmbed["gemini-embedding-001<br/>fallback embeddings"]
MusicRoute --> Lyria["lyria-3-clip-preview<br/>background music"]
classDef signal fill:#e8f1ff,stroke:#3366cc,color:#0b1f44;
classDef route fill:#eef8ee,stroke:#2f8a3a,color:#123915;
classDef model fill:#f4edff,stroke:#805ad5,color:#2d1857;
class Work,VoiceInput,OpsNeed,MemoryNeed,Ambient signal;
class VisionRoute,ConversationRoute,HermesRoute,EmbedRoute,MusicRoute route;
class Gemini20,GeminiLive,GeminiTTS,Gemma,Voyage,GeminiEmbed,Lyria model;
```
### Gemma 4 31B via vLLM
Hermes uses the external OpenAI-compatible endpoint:
### Data flow
```text
Base URL: https://llm.alhinai.dev/v1
API key: not-needed
Model: gemma-4-31B-it
Context: 262144 tokens
```
The vLLM server is configured for Hermes-style tool use:
```text
--max-model-len 262144
--max-num-seqs 1
--max-num-batched-tokens 16384
--gpu-memory-utilization 0.85
--kv-cache-dtype fp8
--enable-auto-tool-choice
--tool-call-parser gemma4
--enable-chunked-prefill
```
Hermes should point at that endpoint with this provider shape:
```yaml
model:
default: gemma-4-31B-it
provider: gemma4-31b-vllm
providers:
gemma4-31b-vllm:
name: Gemma 4 31B vLLM (256K)
api: https://llm.alhinai.dev/v1
api_key: not-needed
transport: chat_completions
default_model: gemma-4-31B-it
discover_models: true
models:
gemma-4-31B-it:
context_length: 262144
agent:
tool_use_enforcement: auto
```
Why this matters: Hermes sends OpenAI tool schemas and `tool_choice: "auto"`.
Without `--enable-auto-tool-choice` and `--tool-call-parser gemma4`, vLLM returns
HTTP 400 before Hermes can initialize an agent.
### Gemini Surfaces
Gemini remains the realtime perception and voice layer inside PodMan.
| Use | Model | Code |
| -------------------------- | ------------------------------- | ------------------------------------------ |
| Screen understanding | `gemini-2.0-flash` | `backend/src/vision/gemini.ts` |
| Urgent spoken alerts | `gemini-3.1-flash-tts-preview` | `backend/src/voice/live.ts` |
| Live room conversation | `gemini-3.1-flash-live-preview` | `agents/podman-live-conversation/agent.py` |
| Memory embeddings fallback | `gemini-embedding-001` | `backend/src/memory/vectors.ts` |
| Ambient background music | `lyria-3-clip-preview` | `backend/src/voice/music.ts` |
Voyage embeddings can be used first when `VOYAGE_API_KEY` is set. Gemini
embeddings remain the fallback. If no embedding provider is available, PodMan
falls back to exact-signature matching.
---
## How The Learning Loop Works
The learning loop is the product. A teammate only has to accept or dismiss an
intervention; the rest is captured automatically.
```text
observe -> detect -> recall prior outcomes -> policy gate -> act -> record outcome
^ |
+-------------------------- next recall -----------------------------+
```
| Stage | What happens | Code |
| ------- | --------------------------------------------------------------------------- | ----------------------------------- |
| Observe | Gemini Vision turns sampled screen frames into structured work context. | `backend/src/vision/gemini.ts` |
| Detect | PodMan detects overlapping files, symbols, research, and unpushed work. | `backend/src/collision/detector.ts` |
| Recall | MongoDB Atlas recalls similar prior events and outcomes. | `backend/src/memory/vectors.ts` |
| Gate | Policy suppresses dismissed false alarms and escalates recurring real ones. | `backend/src/memory/policy.ts` |
| Act | PodMan publishes a card, Hermes message, or urgent voice cue. | `backend/src/action/hermes.ts` |
| Learn | Accept/dismiss feedback is written back to memory. | `backend/src/memory/store.ts` |
---
## Runtime Components
| Layer | Runtime | Responsibility |
| ------------------ | ----------------------------------- | -------------------------------------------------------------------- |
| Frontend | React + Vite | Pod rooms, screen share, cards, voice controls, member state |
| Backend API | Express on `:8787` | LiveKit tokens, pod CRUD, outcomes, memory stats, sync PRs |
| Vision agent | Node + `@livekit/rtc-node` | Subscribes to screen tracks, samples frames, publishes interventions |
| Live voice agent | Python LiveKit Agents + Gemini Live | Real-time voice Q&A in a pod room |
| Memory | MongoDB Atlas | Observations, engineer state, collisions, interventions, outcomes |
| Reasoning agent | Hermes + Gemma vLLM | Tool-using autonomous assistant and ops layer |
| Realtime transport | LiveKit Cloud | Screen tracks, audio tracks, data messages |
| Hosting | DigitalOcean + Caddy + systemd | Static frontend, API, workers, watchdog timers |
---
## Data Flow
```mermaid
sequenceDiagram
@@ -174,161 +265,106 @@ sequenceDiagram
participant Dev as Engineer PWA
participant API as Backend API
participant LK as LiveKit Room
participant Agent as PodMan Agent
participant Gemini as Gemini Vision
participant Mongo as MongoDB Memory
participant Hermes as Hermes / Action Layer
participant Agent as Vision Agent
participant Gemini as Gemini APIs
participant Mongo as MongoDB Atlas
participant Hermes as Hermes/Gemma
Dev->>API: POST /api/token
API-->>Dev: LiveKit URL + JWT
Dev->>LK: Join pod room + publish screen share
Agent->>LK: Subscribe to screen-share video
Agent->>Gemini: Sampled JPEG frame
Agent->>Gemini: Sampled frame
Gemini-->>Agent: Structured work context
Agent->>Mongo: Record observation
Agent->>Mongo: Recall prior patterns (vector search)
Mongo-->>Agent: Prior intervention + outcome
Agent->>Hermes: Create intervention (gated by policy)
Hermes->>LK: Publish card / message / Gemini TTS voice
LK-->>Dev: Render intervention
Dev->>API: POST /api/outcome (accept / dismiss)
API->>Mongo: Store learning signal
Agent->>Mongo: Store observation
Agent->>Mongo: Recall similar prior events
Mongo-->>Agent: Prior outcome + policy hints
Agent->>Hermes: Escalate when autonomous help is useful
Agent->>LK: Publish card / message / voice cue
Dev->>API: POST /api/outcome
API->>Mongo: Store accept/dismiss feedback
```
---
## Public Interfaces
| Interface | Purpose |
| ----------------------------------------------------------- | ---------------------------------------------- |
| `GET /health` | API health check |
| `POST /api/token` | Mint LiveKit room tokens |
| `GET /api/pods` | List pods |
| `GET/POST/PATCH/DELETE /api/pods` | Pod CRUD |
| `POST/DELETE /api/pods/:id/members` | Pod membership |
| `GET /api/pods/:id/members/:name/history` | Recent member work history |
| `POST /api/outcome` | Store accepted/dismissed intervention outcomes |
| `GET /api/memory/stats` | Live memory collection counts |
| `POST /api/sync-pr` | Create a visible sync PR artifact |
| LiveKit topic `podman.intervention` | Intervention data channel |
| Wire messages `COLLISION`, `ACK`, `GIT_REPORT`, `VOICE_CUE` | Agent/PWA contract |
---
## Monorepo Layout
## Built with
Everything below maps to code in this repo.
**Gemini** does the perception, the voice, and the memory:
| Use | Model | Where |
| --------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------ |
| Real-time voice agent (talk to PodMan, answered with repo/git/memory tools) | `gemini-3.1-flash-live-preview` | `agents/podman-live-conversation/agent.py` |
| Spoken urgent alerts over LiveKit | `gemini-3.1-flash-tts-preview` | `backend/src/voice/live.ts` |
| Screen understanding → structured work context | `gemini-2.0-flash` | `backend/src/vision/gemini.ts` |
| Per-pod background music (Interactions API) | `lyria-3-clip-preview` | `backend/src/voice/music.ts` |
| Embeddings for memory recall | `gemini-embedding-001` | `backend/src/memory/vectors.ts` |
**LiveKit** is the real-time layer: screen-share tracks are the input, a typed
data channel (`podman.intervention`) carries cards and messages, audio tracks
carry the spoken alerts and music, and a Python LiveKit Agents worker runs the
live conversation agent in the room.
**MongoDB Atlas** is the memory: `$vectorSearch` recalls similar past events
(exact-signature fallback when needed), and the `outcomes` collection drives the
policy that decides whether and how to act.
**DigitalOcean** hosts it: a static frontend, the API service, and the agent
workers, supervised by systemd. Mongo connectivity is required at boot — services
fail loudly rather than degrade silently.
| Folder | Purpose |
| ----------- | ----------------------------------------------------------------------- |
| `frontend/` | React + Vite PWA |
| `backend/` | Express API, vision agent, memory, collision detection, Hermes job APIs |
| `agents/` | Python LiveKit conversation agent |
| `shared/` | Shared TypeScript contracts |
| `database/` | MongoDB setup and seed utilities |
| `infra/` | Caddy, Docker, DigitalOcean, systemd units |
| `scripts/` | Git watcher, deploy doctor, watchdog, verification tooling |
| `docs/` | Demo, deployment, learning, graph, and architecture notes |
---
## Local Development
Run the core app in three terminals:
## How it works
```mermaid
flowchart LR
Env["1. Configure .env"] --> Install["2. pnpm install"]
Install --> API["Terminal A<br/>backend API :8787"]
Install --> Agent["Terminal B<br/>vision agent"]
Install --> UI["Terminal C<br/>frontend :5173"]
Install --> Voice["Optional<br/>conversation agent"]
API --> Browser["Open local PWA"]
Agent --> Browser
UI --> Browser
Voice --> Browser
1. Engineers open the PWA and join a pod room.
2. The API mints a LiveKit token via `POST /api/token`.
3. The PWA publishes screen share into the room on "Share my screen".
4. The PodMan vision agent subscribes to the screen-share tracks.
5. The agent samples frames, sends them to Gemini Vision, records structured
observations in MongoDB.
6. Each engineer runs the git watcher so PodMan has deterministic
dirty/unpushed truth.
7. The detector fuses live screen context, git truth, GitHub state, and recalled
memory.
8. The policy gate decides whether and how to act — card, Hermes message, or
urgent voice — reusing what worked before.
9. Urgent escalations are spoken via Gemini TTS over a LiveKit audio track.
10. The teammate's accept/dismiss is saved as an outcome, closing the
continual-learning loop.
Separately, any teammate can start a **live voice conversation** with PodMan
(Gemini Live API) to ask about current work, git state, or where something lives
in the repo — answered with real tool calls, not guesses.
---
## Public interfaces
| Interface | Purpose |
| ----------------------------------------------------------- | ------------------------------------------------- |
| `GET /health` | API health check |
| `POST /api/token` | Mint LiveKit room tokens |
| `POST /api/sync-pr` | Create a visible sync PR artifact |
| `POST /api/outcome` | Store accepted/dismissed intervention outcomes |
| `GET /api/memory/stats` | Memory collection counts (live learning evidence) |
| `GET/POST/PATCH/DELETE /api/pods` | Pod CRUD |
| `POST/DELETE /api/pods/:id/members` | Pod membership |
| LiveKit topic `podman.intervention` | Intervention data channel |
| Wire messages `COLLISION`, `ACK`, `GIT_REPORT`, `VOICE_CUE` | Agent/PWA contract |
---
## Monorepo layout
| Folder | What |
| ----------- | -------------------------------------------------------------------------- |
| `frontend/` | React + Vite PWA — pods, LiveKit room UI, screen share, intervention cards |
| `backend/` | Express API plus the LiveKit vision agent worker |
| `agents/` | Python LiveKit Agents worker for the Gemini Live conversation agent |
| `shared/` | Shared TypeScript types and LiveKit data message contracts |
| `database/` | MongoDB setup and seed utilities |
| `infra/` | DigitalOcean specs, Caddyfile, systemd units |
| `scripts/` | Local git watcher + deploy/verify tooling |
| `docs/` | Integration specs and the demo script |
---
## Quick start
classDef step fill:#eef8ee,stroke:#2f8a3a,color:#123915;
classDef run fill:#e8f1ff,stroke:#3366cc,color:#0b1f44;
class Env,Install step;
class API,Agent,UI,Voice,Browser run;
```
```bash
cp .env.example .env
# fill in LIVEKIT_*, GEMINI_*, GITHUB_*, MONGODB_URI
# Fill LIVEKIT_*, GEMINI_*, GITHUB_*, MONGODB_URI.
pnpm install
pnpm --filter @podman/backend dev # API on :8787
pnpm --filter @podman/backend dev:agent # PodMan LiveKit vision agent
pnpm --filter @podman/backend dev:agent # LiveKit vision agent
pnpm --filter @podman/frontend dev # PWA on :5173
```
The live conversation agent (Gemini Live API) runs from `agents/podman-live-conversation/`.
---
## Git watcher — run on every demo laptop
Each engineer runs this before the demo. It polls the local git working tree
every 15 seconds and writes git state to MongoDB so PodMan has deterministic
dirty/unpushed truth that vision alone cannot reliably infer.
Run the Python live conversation agent:
```bash
# from the repo root
node scripts/podman-agent.mjs --name <yourname> --pod <podId>
pnpm livekit:conversation:agent
```
**Demo setup:**
Run the local git watcher on each demo laptop:
```bash
node scripts/podman-agent.mjs --name <engineer-name> --pod <pod-id>
```
Demo identities:
```bash
node scripts/podman-agent.mjs --name alice --pod demo-pod
@@ -336,5 +372,184 @@ node scripts/podman-agent.mjs --name bob --pod demo-pod
node scripts/podman-agent.mjs --name carol --pod demo-pod
```
**Requirements:** `MONGODB_URI` exported (or in `backend/.env`), `pnpm install`
run first, launched from the repo root.
---
## Production Operations
The production droplet is systemd-supervised. Caddy serves the built frontend
and proxies `/api/*` to the backend on `127.0.0.1:8787`.
```mermaid
flowchart LR
Public["https://podman.live"] --> Caddy["caddy.service"]
Caddy --> Static["/var/www/podman"]
Caddy --> API["podman-platform-api.service<br/>:8787"]
API --> Agent["podman-platform-agent.service"]
API --> Voice["podman-live-conversation-agent.service"]
Watchdog["podman-hermes-watchdog.timer"] --> Public
Sync["podman-hermes-sync-deploy.timer"] --> API
classDef public fill:#e8f1ff,stroke:#3366cc,color:#0b1f44;
classDef service fill:#eef8ee,stroke:#2f8a3a,color:#123915;
classDef timer fill:#fff6df,stroke:#c47f00,color:#3d2b00;
class Public public;
class Caddy,Static,API,Agent,Voice service;
class Watchdog,Sync timer;
```
| Service / timer | Purpose |
| ---------------------------------------- | -------------------------------------------------------------- |
| `podman-platform-api.service` | Built backend API on port `8787` |
| `podman-platform-agent.service` | Node LiveKit vision agent |
| `podman-live-conversation-agent.service` | Python Gemini Live conversation agent |
| `podman-hermes-watchdog.timer` | Periodic public health and remediation |
| `podman-hermes-sync-deploy.timer` | Clean-tree fast-forward deploy loop |
| `caddy.service` | Serves `/var/www/podman`, proxies `/api/*` to `127.0.0.1:8787` |
Check the app from the outside first:
```bash
curl https://podman.live/
curl https://podman.live/health
curl https://podman.live/api/pods
curl https://podman.live/api/presence
curl https://podman.live/api/memory/stats
```
Then check the droplet services:
```bash
systemctl is-active podman-platform-api podman-platform-agent
systemctl is-active podman-live-conversation-agent
systemctl is-active podman-hermes-watchdog.timer podman-hermes-sync-deploy.timer
```
Hermes operations scripts:
```bash
pnpm hermes:watchdog
pnpm hermes:watchdog:strict
pnpm hermes:sync-deploy
pnpm deploy:doctor:strict
```
Gemma/vLLM endpoint checks:
```bash
curl https://llm.alhinai.dev/v1/models \
-H "Authorization: Bearer not-needed"
curl https://llm.alhinai.dev/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer not-needed" \
-d '{
"model": "gemma-4-31B-it",
"messages": [{"role": "user", "content": "Reply with exactly: working"}],
"temperature": 0,
"max_tokens": 512
}'
```
---
## Required Environment
```bash
LIVEKIT_URL=wss://your-livekit-server.livekit.cloud
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...
LIVEKIT_CONVERSATION_AGENT_NAME=podman-live-conversation
GEMINI_API_KEY=...
GEMINI_VISION_MODEL=gemini-2.0-flash
GEMINI_LIVE_MODEL=gemini-3.1-flash-tts-preview
GEMINI_CONVERSATION_MODEL=gemini-3.1-flash-live-preview
GEMINI_EMBEDDING_MODEL=gemini-embedding-001
GEMINI_TTS_VOICE=Charon
GITHUB_TOKEN=...
GITHUB_REPO=karti-ai/podman
MONGODB_URI=mongodb+srv://...
VOYAGE_API_KEY=...
VOYAGE_EMBEDDING_MODEL=voyage-4-lite
PORT=8787
POD_ROOM=demo-pod
```
Gemma/Hermes provider values live in Hermes config, not PodMan `.env`:
```text
provider: gemma4-31b-vllm
model: gemma-4-31B-it
base_url: https://llm.alhinai.dev/v1
api_key: not-needed
```
---
## Verification
Before calling a deployment healthy:
```bash
pnpm verify
pnpm verify:infra
pnpm deploy:doctor:strict
pnpm hermes:watchdog:strict
```
For the public site:
```bash
curl https://podman.live/
curl https://podman.live/health
curl https://podman.live/api/pods
curl https://podman.live/api/presence
curl https://podman.live/api/memory/stats
```
For Hermes/Gemma:
```bash
hermes -z 'Reply with exactly: working' \
--provider gemma4-31b-vllm \
--model gemma-4-31B-it
```
Expected output:
```text
working
```
---
## Troubleshooting
| Symptom | Likely cause | Check |
| ---------------------------------------- | ---------------------------------------------- | ------------------------------------------------------------ |
| Frontend loads but API fails | Backend or Caddy proxy issue | `systemctl status podman-platform-api caddy` |
| `/api/*` returns 502 | API not listening on `8787` | `ss -ltnp`, `curl http://127.0.0.1:8787/health` |
| No screen observations | Vision agent not in LiveKit room | `journalctl -u podman-platform-agent -n 80` |
| Live voice missing | Python conversation agent down | `journalctl -u podman-live-conversation-agent -n 80` |
| Memory empty | MongoDB unavailable or env missing | `curl /api/memory/stats`, backend logs |
| Hermes says tool auto-choice is disabled | vLLM missing tool flags | verify `--enable-auto-tool-choice --tool-call-parser gemma4` |
| `llm.alhinai.dev` returns 502 | Gemma vLLM still loading or tunnel target down | `curl /v1/models`, vLLM logs on Gemma host |
---
## Why It Gets Better
PodMan is not a static alert system. It remembers what actually helped.
- A dismissed false alarm lowers future urgency.
- An accepted real collision raises future urgency for similar work.
- Exact-signature recall catches repeats even without vector search.
- MongoDB outcomes become the policy signal for the next session.
- Hermes and Gemma give the system a tool-using agent when the coordination
problem needs active investigation instead of a passive card.
The goal is simple: fewer interruptions, fewer duplicate branches, and a team
that can move fast without constantly asking what everyone else is doing.
+16
View File
@@ -52,6 +52,8 @@ You are in a private 1:1 voice conversation with one developer.
Use PodMan tools before making claims about current work, git state, collisions, blockers,
team memory, or recent decisions. Keep spoken answers short. Prefer one useful next step.
For questions about a person's style, goals, past work, collaboration habits, personal context,
or what they know across pods/sessions, call get_user_learning_profile before answering.
If a critical collision event arrives, stop the current turn and state the alert immediately.
To find code, files, symbols, or how something is implemented in the repository, call search_repo.
For git commit history, authorship, recent changes, or which commit introduced something, call
@@ -130,6 +132,20 @@ class PodManLiveAgent(Agent):
)
return json.dumps(data, ensure_ascii=True)[:12000]
@function_tool()
async def get_user_learning_profile(self, context: RunContext) -> str:
"""Get persistent cross-pod, cross-session knowledge about this developer:
collaboration style, goals, known work, recent activity, and Hermes history.
"""
data = await asyncio.to_thread(
request_json,
f"/api/internal/pods/{self.pod_id}/live-context?identity={self.identity}",
)
profile = data.get("userLearningProfile")
if not profile:
return "No persistent user learning profile has been built for this developer yet."
return json.dumps(profile, ensure_ascii=True)[:10000]
@function_tool()
async def record_conversation_note(self, context: RunContext, note: str, kind: str = "summary") -> str:
"""Store a useful decision, outcome, or preference learned during this conversation."""
+1
View File
@@ -17,6 +17,7 @@
"typecheck": "tsc -p tsconfig.json --noEmit"
},
"dependencies": {
"@clerk/express": "^2.1.32",
"@google/genai": "^2.10.0",
"@livekit/rtc-node": "^0.13.29",
"@podman/shared": "workspace:*",
+4 -1
View File
@@ -58,7 +58,10 @@ export async function publishHermesIntervention(
void notifyCriticalLiveConversations(collision, intervention, voiceLine).catch((err) =>
console.warn(`[live-conversation] critical notify failed: ${(err as Error).message}`),
);
if (voiceLine) await speak(room, voiceLine, { priority: 'critical' });
if (voiceLine)
await speak(room, voiceLine, {
priority: collision.severity === 'critical' ? 'critical' : 'normal',
});
}
async function hermesToken(roomName: string): Promise<string> {
+72 -2
View File
@@ -1,4 +1,10 @@
import type { EngineerContext, MemberWorkHistory, MemberWorkHistoryFile } from '@podman/shared';
import type {
Collision,
EngineerContext,
MemberWorkHistory,
MemberWorkHistoryFile,
MemberWorkHistoryRoi,
} from '@podman/shared';
import { getDb } from '../memory/db.js';
import { parseGitStatusPath } from '../graph/live.js';
@@ -80,7 +86,7 @@ export async function getMemberWorkHistory(
const limit = Math.min(Math.max(options.limit ?? 80, 10), 200);
const since = new Date(Date.now() - windowHours * 60 * 60 * 1000).toISOString();
const [observations, gitState] = await Promise.all([
const [observations, gitState, collisions, interventions] = await Promise.all([
db
.collection<EngineerContext>('observations')
.find({ podId, observedAt: { $gte: since } }, { projection: { _id: 0 } })
@@ -91,6 +97,16 @@ export async function getMemberWorkHistory(
podId,
name: { $regex: `^${member.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, $options: 'i' },
}),
db
.collection<Collision>('collisions')
.find({ podId, detectedAt: { $gte: since } }, { projection: { _id: 0 } })
.sort({ detectedAt: -1 })
.limit(200)
.toArray(),
db
.collection<{ collisionId: string }>('interventions')
.find({ podId }, { projection: { collisionId: 1, _id: 0 } })
.toArray(),
]);
const files = new Map<string, FileAccumulator>();
@@ -158,6 +174,9 @@ export async function getMemberWorkHistory(
timeline.sort((a, b) => Date.parse(b.at) - Date.parse(a.at));
const interventionIds = new Set(interventions.map((i) => i.collisionId));
const roi = computeRoi(member, collisions, interventionIds, gitState?.changedFiles?.length ?? 0);
return {
podId,
member,
@@ -170,5 +189,56 @@ export async function getMemberWorkHistory(
},
files: fileRows,
timeline: timeline.slice(0, limit),
roi,
};
}
function computeRoi(
member: string,
collisions: Collision[],
interventionCollisionIds: Set<string>,
changedFileCount: number,
): MemberWorkHistoryRoi {
const involved = (c: Collision) =>
c.engineers?.some((e) => sameMember(e, member)) ||
sameMember(c.researcher, member) ||
sameMember(c.editor, member);
const eligible = collisions.filter(
(c) =>
involved(c) &&
interventionCollisionIds.has(c.id) &&
(c.gitOverlap === true || c.severity === 'critical'),
);
const weightOf = (c: Collision): { label: string; minutes: number } => {
if (c.overlapKind === 'research') return { label: 'research overlap', minutes: 10 };
if (c.severity === 'critical') return { label: 'critical same-file', minutes: 45 };
if (c.severity === 'warn') return { label: 'warn same-file', minutes: 20 };
return { label: 'info same-file', minutes: 10 };
};
let savedMinutes = 0;
const groups = new Map<string, { count: number; minutesEach: number }>();
for (const c of eligible) {
const { label, minutes } = weightOf(c);
savedMinutes += minutes / Math.max(1, c.engineers?.length ?? 1);
const g = groups.get(label) ?? { count: 0, minutesEach: minutes };
g.count += 1;
groups.set(label, g);
}
const filesDeconflicted = new Set(eligible.map((c) => c.file)).size;
return {
savedMinutes: Math.round(savedMinutes),
clashesCaught: eligible.length,
filesDeconflicted,
conflictFreeFiles: Math.max(0, changedFileCount - filesDeconflicted),
totalFiles: changedFileCount,
breakdown: [...groups.entries()].map(([label, g]) => ({
label,
count: g.count,
minutesEach: g.minutesEach,
})),
};
}
+88 -20
View File
@@ -2,11 +2,13 @@ import { RoomEvent, type Room } from '@livekit/rtc-node';
import type { EngineerContext, Collision, Intervention, DataMessage } from '@podman/shared';
import { analyzeFrame } from '../vision/gemini.js';
import { detectCollisions } from '../collision/detector.js';
import { detectResearchOverlaps } from '../collision/research.js';
import { getGithubState } from '../github/client.js';
import {
recordObservation,
recordCollision,
recordIntervention,
recordSuppression,
updateInterventionStatus,
} from '../memory/store.js';
import { getGitStates, type GitState } from '../memory/db.js';
@@ -33,6 +35,12 @@ function canonicalName(raw?: string): string {
return (raw ?? '').trim().toLowerCase();
}
/** How long a still-present conflict stays muted after we voice it, before it
* re-alerts. Edge-trigger alone permanently muted files that stay dirty for
* the whole session (e.g. README everyone tests on). This bounds that: voice
* once, then re-alert at most every interval while it persists. */
const CONFLICT_REALERT_MS = Number(process.env.CONFLICT_REALERT_MS ?? '20000');
/** Git ground truth: do ALL involved engineers currently have the collided file
* in their changedFiles? Computed at detection time while git state is fresh. */
function engineersOverlapOnFile(collision: Collision, gitStates: Map<string, GitState>): boolean {
@@ -48,13 +56,15 @@ function engineersOverlapOnFile(collision: Collision, gitStates: Map<string, Git
export class PodMan {
private contexts = new Map<string, EngineerContext>();
/**
* Conflicts we have already voiced and that are still unresolved, keyed by
* file (see conflictKey). Edge-triggered alerting: speak once when a conflict
* appears, stay quiet while it persists. A conflict is re-armed (deleted
* here) by onScreenFrame as soon as a detection cycle no longer sees it, so a
* resolved-then-recurring conflict alerts again.
* Conflicts we have already voiced, keyed by file + engineer pair (see
* conflictKey), mapped to the time we last voiced them. Edge-triggered:
* speak once when a conflict appears. Re-armed (deleted here) by
* onScreenFrame as soon as a detection cycle no longer sees it, so a
* resolved-then-recurring conflict alerts again. Additionally, a conflict
* that *persists* re-alerts every CONFLICT_REALERT_MS so a perpetually-dirty
* file (README) doesn't go silent forever after the first alert.
*/
private activeConflicts = new Set<string>();
private activeConflicts = new Map<string, number>();
constructor(
private room: Room,
@@ -97,19 +107,24 @@ export class PodMan {
}
const github = await getGithubState(); // cached
const collisions = detectCollisions([...this.contexts.values()], github, gitStates);
const contexts = [...this.contexts.values()];
const fileCollisions = detectCollisions(contexts, github, gitStates);
const researchCollisions = await detectResearchOverlaps(contexts, gitStates);
const collisions = [...fileCollisions, ...researchCollisions];
// Re-arm: any conflict we previously voiced that is no longer present has
// resolved, so allow it to alert again if it recurs.
const current = new Set(collisions.map((c) => this.conflictKey(c)));
for (const key of this.activeConflicts) {
for (const key of this.activeConflicts.keys()) {
if (!current.has(key)) this.activeConflicts.delete(key);
}
// Capture git ground-truth overlap now, while engineer_states are fresh, so
// the outcome-time verifier never depends on a stale sidecar or a late click.
for (const collision of collisions) {
collision.gitOverlap = engineersOverlapOnFile(collision, gitStates);
if (collision.overlapKind !== 'research') {
collision.gitOverlap = engineersOverlapOnFile(collision, gitStates);
}
}
for (const collision of collisions) await this.handle(collision);
@@ -122,12 +137,16 @@ export class PodMan {
* basename.
*/
private conflictKey(collision: Collision): string {
return comparableBasename(collision.file);
const who = [...collision.engineers].map(canonicalName).sort().join('+');
return `${collision.overlapKind ?? 'file'}:${comparableBasename(collision.file)}:${who}`;
}
private async handle(collision: Collision): Promise<void> {
const key = this.conflictKey(collision);
if (this.activeConflicts.has(key)) return; // single-shot: already voiced, still unresolved
const lastAlerted = this.activeConflicts.get(key);
// Edge-trigger + time-based re-arm: stay quiet right after voicing, but
// re-alert a persistent conflict once CONFLICT_REALERT_MS has elapsed.
if (lastAlerted !== undefined && Date.now() - lastAlerted < CONFLICT_REALERT_MS) return;
const prior = await recallSimilar(collision); // Loop A: exact/vector recall raises confidence
// Only escalate to critical (which triggers the spoken alert) when the
@@ -138,13 +157,64 @@ export class PodMan {
if (prior?.priorOutcome?.accepted && prior?.priorOutcome?.wasRealCollision) {
collision.severity = 'critical';
}
if (!shouldIntervene(collision, prior)) return; // Loop B: policy gate
if (!shouldIntervene(collision, prior)) {
// Feature A — make the negative-feedback loop VISIBLE. If we stayed quiet
// *specifically* because this signature was DISMISSED before, record a
// durable suppressed-repeat event (timestamped now, at the repeat) so the
// activity stream shows the learning instead of nothing.
if (prior?.priorOutcome && !prior.priorOutcome.accepted) {
// Mark handled first — like the alert path below — so we record ONE
// suppressed-repeat per recurrence, not once per frame; it re-arms via
// the resolution sweep in onScreenFrame. Awaited like recordCollision so
// the durable learning proof is reliably written.
this.activeConflicts.set(key, Date.now());
await recordSuppression(
collision,
prior.priorOutcome.interventionId,
prior.priorOutcome.recordedAt,
);
}
return; // Loop B: policy gate
}
this.activeConflicts.add(key); // claim now we're alerting; re-armed in onScreenFrame on resolution
this.activeConflicts.set(key, Date.now()); // claim + timestamp; re-armed on resolution or after CONFLICT_REALERT_MS
await recordCollision(collision);
const action = preferredAction(collision, prior);
const names = collision.engineers.join(' + ');
const shortFile = collision.file.split('/').pop() ?? collision.file;
const isResearchOverlap = collision.overlapKind === 'research';
if (isResearchOverlap) {
const researcher = collision.researcher ?? collision.engineers[1] ?? 'A teammate';
const editor = collision.editor ?? collision.engineers[0] ?? 'a teammate';
const topic = collision.researchTopic ?? 'the same area';
const source = collision.researchSource ? ` (${collision.researchSource})` : '';
const message = `🤝 ${researcher} is researching ${topic}${source} while ${editor} edits ${shortFile} — sync up before duplicating effort.`;
const voiceLine = `${researcher} is researching ${topic} while ${editor} works on ${shortFile}. Worth a quick sync.`;
const intervention: Intervention = {
id: `int_${Date.now()}`,
collisionId: collision.id,
podId: this.podId,
kind: 'card',
message,
suggestedAction: {
kind: 'ping_teammate',
params: {
file: collision.file,
summary: message,
engineers: collision.engineers,
researchTopic: collision.researchTopic,
researchSource: collision.researchSource,
},
},
status: 'pending',
createdAt: new Date().toISOString(),
};
await recordIntervention(intervention);
await publishHermesIntervention(this.room, collision, intervention, voiceLine);
return;
}
const names = collision.engineers.join(' + ');
// Terse, demo-centered alert — short and direct, not chatty AI prose.
const message =
@@ -174,11 +244,9 @@ export class PodMan {
};
await recordIntervention(intervention);
await publishHermesIntervention(
this.room,
collision,
intervention,
collision.severity === 'critical' ? voiceLine : undefined,
);
// Voice every intervention, not just critical escalations. Priority is set
// by severity inside publishHermesIntervention: critical jumps the queue,
// the rest play sequentially so concurrent alerts don't garble each other.
await publishHermesIntervention(this.room, collision, intervention, voiceLine);
}
}
+30
View File
@@ -0,0 +1,30 @@
import { clerkMiddleware, getAuth } from '@clerk/express';
import type { Request, RequestHandler, Response } from 'express';
import { env } from './env.js';
export interface RequestUserContext {
clerkUserId: string;
}
export const clerkAuthMiddleware: RequestHandler = env.CLERK_SECRET_KEY
? clerkMiddleware()
: (_req, _res, next) => next();
export function requestUser(req: Request): RequestUserContext | null {
if (!env.CLERK_SECRET_KEY) return null;
try {
const auth = getAuth(req);
return auth.isAuthenticated && auth.userId ? { clerkUserId: auth.userId } : null;
} catch {
return null;
}
}
export function requireRequestUser(req: Request, res: Response): RequestUserContext | null {
const user = requestUser(req);
if (!user) {
res.status(401).json({ error: 'sign in required' });
return null;
}
return user;
}
+18 -2
View File
@@ -19,6 +19,15 @@ function fileKey(raw?: string): string | undefined {
return base.toLowerCase();
}
/**
* Canonicalize an engineer identity for case/whitespace-insensitive matching.
* Vision reports a display name ("Karti") while the git watcher reports a handle
* ("karti"); without this the same human collides with themselves.
*/
function canonicalName(raw: string): string {
return raw.trim().toLowerCase();
}
interface Touch {
engineerId: string;
unpushed: boolean;
@@ -64,8 +73,15 @@ export function detectCollisions(
const out: Collision[] = [];
for (const [, touches] of byFile) {
const engineers = [...new Set(touches.map((t) => t.engineerId))];
if (engineers.length < 2) continue; // need two distinct people on one file
// Distinct PEOPLE, case/whitespace-insensitive — one display name per person.
// Vision "Karti" and git "karti" are the same human, not a collision.
const byPerson = new Map<string, string>(); // canonical id -> display name
for (const t of touches) {
const id = canonicalName(t.engineerId);
if (id && !byPerson.has(id)) byPerson.set(id, t.engineerId);
}
if (byPerson.size < 2) continue; // need two distinct people on one file
const engineers = [...byPerson.values()];
const anyUnpushed = touches.some((t) => t.unpushed) || github.unpushed === true;
if (!anyUnpushed) continue; // the crux GitHub alone cannot answer
+147
View File
@@ -0,0 +1,147 @@
import type { Collision, EngineerContext } from '@podman/shared';
import { env } from '../env.js';
import type { GitState } from '../memory/db.js';
import { semanticSimilarity } from '../memory/vectors.js';
export interface ResearchOpts {
similarity?: (a: string, b: string) => Promise<number | null>;
threshold?: number;
}
interface EditorFile {
engineerId: string;
file: string;
symbol?: string;
activity?: string;
}
interface Candidate {
collision: Collision;
score: number;
}
function stripGitPrefix(raw: string): string {
return raw.trim().replace(/^(\?\?|[MADRCU!]{1,2})\s+/, '');
}
/** Case/whitespace-insensitive identity so vision "Karti" and git "karti" are
* recognized as the same person and never flagged researching-vs-editing self. */
function canonicalName(raw: string): string {
return raw.trim().toLowerCase();
}
function fileStem(raw: string): string {
const base = stripGitPrefix(raw).split(/[\\/]/).pop()?.trim().toLowerCase() ?? '';
return base.replace(/\.[^.]+$/, '');
}
function words(raw: string): string[] {
return raw
.toLowerCase()
.replace(/[^a-z0-9]+/g, ' ')
.split(/\s+/)
.filter((word) => word.length >= 3);
}
function uniqueTokens(raw: string): Set<string> {
const tokens = new Set(words(raw));
for (const word of [...tokens]) {
if (word.endsWith('kit')) tokens.add(word.replace(/kit$/, ''));
if (word.endsWith('s')) tokens.add(word.slice(0, -1));
}
return tokens;
}
function fallbackMatches(researchText: string, fileText: string): boolean {
const research = uniqueTokens(researchText);
const file = uniqueTokens(fileText);
for (const token of file) {
if (research.has(token)) return true;
}
return false;
}
function collectEditorFiles(
contexts: EngineerContext[],
gitStates: Map<string, GitState> | undefined,
): EditorFile[] {
const files = new Map<string, EditorFile>();
const add = (editor: EditorFile): void => {
const stem = fileStem(editor.file);
if (!stem) return;
files.set(`${editor.engineerId}:${stripGitPrefix(editor.file)}`, editor);
};
for (const [engineerId, git] of gitStates ?? []) {
for (const changed of git.changedFiles) {
add({ engineerId, file: changed });
}
}
for (const context of contexts) {
if (context.mode !== 'research' && context.currentFile) {
add({
engineerId: context.engineerId,
file: context.currentFile,
symbol: context.currentSymbol,
activity: context.activity,
});
}
}
return [...files.values()];
}
export async function detectResearchOverlaps(
contexts: EngineerContext[],
gitStates: Map<string, GitState> | undefined,
opts: ResearchOpts = {},
): Promise<Collision[]> {
const similarity = opts.similarity ?? semanticSimilarity;
const threshold = opts.threshold ?? env.RESEARCH_OVERLAP_THRESHOLD;
const researchers = contexts.filter((c) => c.mode === 'research' && c.researchTopic);
const editorFiles = collectEditorFiles(contexts, gitStates);
const bestByResearcher = new Map<string, Candidate>();
for (const researcher of researchers) {
const topic = researcher.researchTopic?.trim();
if (!topic) continue;
const source = researcher.researchSource?.trim();
const researchText = [topic, source].filter(Boolean).join(' ');
for (const editor of editorFiles) {
if (canonicalName(editor.engineerId) === canonicalName(researcher.engineerId)) continue;
const stem = fileStem(editor.file);
if (!stem) continue;
const fileText = [stem, editor.symbol, editor.activity].filter(Boolean).join(' ');
const score = await similarity(researchText, fileText);
const matched = score === null ? fallbackMatches(researchText, fileText) : score >= threshold;
if (!matched) continue;
const rank = score ?? 1;
const existing = bestByResearcher.get(researcher.engineerId);
if (existing && existing.score >= rank) continue;
bestByResearcher.set(researcher.engineerId, {
score: rank,
collision: {
id: `col_research_${stem}_${Date.now()}`,
podId: researcher.podId,
file: editor.file,
symbol: editor.symbol,
engineers: [editor.engineerId, researcher.engineerId],
severity: 'warn',
overlapKind: 'research',
researchTopic: topic,
...(source ? { researchSource: source } : {}),
researcher: researcher.engineerId,
editor: editor.engineerId,
detectedAt: new Date().toISOString(),
},
});
}
}
return [...bestByResearcher.values()].map((candidate) => candidate.collision);
}
+9 -2
View File
@@ -1,4 +1,6 @@
import 'dotenv/config';
import { config } from 'dotenv';
config({ path: ['.env.local', '../.env.local', '.env', '../.env'] });
function req(name: string): string {
const v = process.env[name];
@@ -22,7 +24,10 @@ export const env = {
LIVEKIT_API_KEY: req('LIVEKIT_API_KEY'),
LIVEKIT_API_SECRET: req('LIVEKIT_API_SECRET'),
LIVEKIT_AGENT_NAME: opt('LIVEKIT_AGENT_NAME'),
LIVEKIT_CONVERSATION_AGENT_NAME: opt('LIVEKIT_CONVERSATION_AGENT_NAME', 'podman-live-conversation'),
LIVEKIT_CONVERSATION_AGENT_NAME: opt(
'LIVEKIT_CONVERSATION_AGENT_NAME',
'podman-live-conversation',
),
// Gemini
GEMINI_API_KEY: reqAny('GEMINI_API_KEY', ['GOOGLE_API_KEY', 'GOOGLE_GENERATIVE_AI_API_KEY']),
GEMINI_VISION_MODEL: opt('GEMINI_VISION_MODEL', 'gemini-2.0-flash'),
@@ -39,7 +44,9 @@ export const env = {
VOYAGE_EMBEDDING_MODEL: opt('VOYAGE_EMBEDDING_MODEL', 'voyage-4-lite'),
// Server
PORT: Number(opt('PORT', '8787')),
CLERK_SECRET_KEY: opt('CLERK_SECRET_KEY'),
NUDGE_COOLDOWN_MS: Number(opt('NUDGE_COOLDOWN_MS', '180000')),
RESEARCH_OVERLAP_THRESHOLD: Number(opt('RESEARCH_OVERLAP_THRESHOLD', '0.6')),
INTERNAL_AGENT_TOKEN: opt('INTERNAL_AGENT_TOKEN'),
} as const;
+37 -1
View File
@@ -457,6 +457,38 @@ export async function materializePodGraph(podId: string): Promise<PodGraph | nul
);
}
// 7. Suppressed repeats: PodMan stayed quiet on a recurring collision because
// the signature was dismissed before — the negative-feedback loop made visible
// (Feature A). Stamped at repeat time, so it sorts as recent activity.
const suppressionDocs = await c.suppressions
.find({ podId })
.sort({ suppressedAt: -1 })
.limit(50)
.toArray();
// Collapse to one beat per file (keep the most recent — docs are sorted desc)
// so pre-fix duplicate rows never render as spam. The stable per-file id also
// dedupes through pushActivity's `seen` set.
const seenSuppressedFiles = new Set<string>();
for (const s of suppressionDocs) {
const sFile = normalizeFile(s.file);
if (!isFilePath(sFile)) continue;
const fileKey = sFile.toLowerCase();
if (seenSuppressedFiles.has(fileKey)) continue;
seenSuppressedFiles.add(fileKey);
const sEngs = (s.engineers ?? []).join(' + ') || 'teammates';
pushActivity(
activity,
{
id: `suppressed:${fileKey}`,
at: s.suppressedAt,
kind: 'suppressed',
title: `Suppressed — ${shortLabel(sFile)} repeat silenced`,
detail: `${sEngs} on ${sFile} recurred, but it was dismissed before — PodMan stayed quiet.`,
},
activityIds,
);
}
// Prune test-artifact engineers, then anything left orphaned by that.
const dropNode = (id: string) => {
b.nodes.delete(id);
@@ -495,7 +527,11 @@ export async function materializePodGraph(podId: string): Promise<PodGraph | nul
const nodes = [...b.nodes.values()];
// No real activity beyond the bare roster -> let the caller fall back to demo.
const hasActivity = nodes.some((n) => n.kind !== 'engineer');
// Suppression beats are real negative-feedback proof even when they add no
// nodes/edges, so they satisfy the gate too — a clean pod with preserved
// suppressions must not fall back to the demo graph and hide the proof.
const hasSuppressed = activity.some((a) => a.kind === 'suppressed');
const hasActivity = hasSuppressed || nodes.some((n) => n.kind !== 'engineer');
if (!hasActivity) return null;
layout(nodes);
+16 -1
View File
@@ -1,5 +1,9 @@
import { getDb } from '../memory/db.js';
import { getMemberWorkHistory } from '../activity/member-history.js';
import {
buildUserLearningProfile,
getUserLearningProfileByIdentity,
} from '../memory/user-learning.js';
const DEFAULT_LIMIT = 8;
@@ -10,7 +14,8 @@ function sinceIso(hours: number): string {
export async function getLiveConversationContext(podId: string, identity: string) {
const db = await getDb();
const since = sinceIso(12);
const [pod, history, gitState, collisions, interventions, outcomes] = await Promise.all([
const [pod, history, gitState, collisions, interventions, outcomes, userLearningProfile] =
await Promise.all([
db.collection('pods').findOne({ id: podId }, { projection: { _id: 0 } }),
getMemberWorkHistory(podId, identity, { hours: 24, limit: 30 }).catch(() => null),
db.collection('engineer_states').findOne(
@@ -44,6 +49,7 @@ export async function getLiveConversationContext(podId: string, identity: string
.sort({ recordedAt: -1 })
.limit(DEFAULT_LIMIT)
.toArray(),
getUserLearningProfileByIdentity(identity).catch(() => null),
]);
return {
@@ -51,6 +57,7 @@ export async function getLiveConversationContext(podId: string, identity: string
identity,
generatedAt: new Date().toISOString(),
currentGitState: gitState,
userLearningProfile,
memberHistory: history,
recentCollisions: collisions,
recentInterventions: interventions,
@@ -76,5 +83,13 @@ export async function recordLiveConversationNote(input: {
createdAt: new Date().toISOString(),
};
await (await getDb()).collection('conversation_notes').insertOne(doc);
if (input.identity) {
const profile = await getUserLearningProfileByIdentity(input.identity);
if (profile) {
void buildUserLearningProfile(profile.clerkUserId).catch((err) =>
console.warn(`[memory] note user learning refresh failed: ${(err as Error).message}`),
);
}
}
return { ...doc, _id: undefined };
}
+50
View File
@@ -31,12 +31,40 @@ export async function closeMemory(): Promise<void> {
await client.close();
}
/** A durable record that PodMan stayed quiet on a recurring collision because
* its signature was previously dismissed — the negative-feedback loop made
* auditable. Timestamped at the repeat; materialized as `suppressed` activity
* in graph/live.ts. The suppressed collision is never written to `collisions`
* (the agent returns before recordCollision), so this is its own record. */
export interface SuppressionDoc {
id: string;
podId: string;
collisionId: string;
file: string;
engineers: string[];
priorInterventionId?: string;
priorDismissedAt?: string;
suppressedAt: string;
}
export interface PodCollections {
pods: Collection<Pod>;
observations: Collection<EngineerContext>;
collisions: Collection<Collision>;
interventions: Collection<Intervention>;
outcomes: Collection<InterventionOutcome>;
suppressions: Collection<SuppressionDoc>;
}
export interface UserPodContextDoc {
id: string;
clerkUserId: string;
podId: string;
memberName?: string;
action: string;
source: 'clerk';
observedAt: string;
metadata?: Record<string, string | number | boolean | null>;
}
export async function collections(): Promise<PodCollections> {
@@ -47,6 +75,7 @@ export async function collections(): Promise<PodCollections> {
collisions: db.collection<Collision>('collisions'),
interventions: db.collection<Intervention>('interventions'),
outcomes: db.collection<InterventionOutcome>('outcomes'),
suppressions: db.collection<SuppressionDoc>('suppressions'),
};
}
@@ -114,6 +143,7 @@ export async function initMemory(): Promise<void> {
['collisions.file', () => c.collisions.createIndex({ podId: 1, file: 1, detectedAt: -1 })],
['interventions.collisionId', () => c.interventions.createIndex({ collisionId: 1 })],
['outcomes.interventionId', () => c.outcomes.createIndex({ interventionId: 1 })],
['suppressions.podId', () => c.suppressions.createIndex({ podId: 1, suppressedAt: -1 })],
['hermes_jobs.id', () => db.collection('hermes_jobs').createIndex({ id: 1 }, { unique: true })],
[
'hermes_jobs.session',
@@ -123,6 +153,26 @@ export async function initMemory(): Promise<void> {
'hermes_job_events.job',
() => db.collection('hermes_job_events').createIndex({ jobId: 1, createdAt: 1 }),
],
[
'user_pod_context.user',
() => db.collection('user_pod_context').createIndex({ clerkUserId: 1, observedAt: -1 }),
],
[
'user_pod_context.pod',
() => db.collection('user_pod_context').createIndex({ podId: 1, observedAt: -1 }),
],
[
'user_learning_profiles.user',
() => db.collection('user_learning_profiles').createIndex({ clerkUserId: 1 }, { unique: true }),
],
[
'user_learning_profiles.updated',
() => db.collection('user_learning_profiles').createIndex({ updatedAt: -1 }),
],
[
'conversation_notes.identity',
() => db.collection('conversation_notes').createIndex({ identity: 1, createdAt: -1 }),
],
];
for (const [name, make] of indexes) {
try {
+16 -26
View File
@@ -1,32 +1,22 @@
import type { Collision, SuggestedActionKind } from '@podman/shared';
import type { RecalledCollision } from './vectors.js';
const lastNudgeByPod = new Map<string, number>();
function cooldownMs(): number {
return Number(process.env.NUDGE_COOLDOWN_MS ?? '180000');
}
/** Policy gate: combines severity, exact recall outcomes, and per-pod cooldown. */
export function shouldIntervene(collision: Collision, prior: RecalledCollision | null): boolean {
if (collision.severity === 'info') return false;
const priorOutcome = prior?.priorOutcome;
// Suppress when the identical prior was dismissed (accepted === false). The
// former `&& !priorOutcome.wasRealCollision` term was dead code: outcomes are
// recorded with wasRealCollision hardcoded true, so the gate never fired and
// the 85 real dismissals in Atlas were ignored. Dismissals are the negative
// signal per continual-learning/policy.md:41 + spec.md:163. (RSI Step 1)
if (priorOutcome && !priorOutcome.accepted) return false;
const cooldown = cooldownMs();
const last = lastNudgeByPod.get(collision.podId) ?? 0;
if (cooldown > 0 && Date.now() - last < cooldown) {
return false;
}
lastNudgeByPod.set(collision.podId, Date.now());
return true;
/**
* Policy gate — suppression fully disabled for the demo.
*
* Every real collision surfaces an intervention. Accept/Dismiss are still
* recorded as outcomes but carry NO gating meaning: dismissing one collision
* never silences future ones, and there is no per-pod cooldown throttling
* consecutive conflicts. The only filter is severity: `info` collisions are
* informational, not actionable, so they don't nudge. The same-collision
* single-shot dedupe lives in `PodmanAgent.handle` (activeConflicts), so
* removing the cooldown does not cause repeat spam of an unresolved collision.
*
* (Prior dismissal-based suppression over-generalized: one README discard
* permanently muted all README clashes via loose file/vector recall.)
*/
export function shouldIntervene(collision: Collision, _prior: RecalledCollision | null): boolean {
return collision.severity !== 'info';
}
/** Preferred action selection based on collision severity and prior accepted actions. */
+81 -9
View File
@@ -5,16 +5,19 @@ import type {
InterventionOutcome,
InterventionStatus,
} from '@podman/shared';
import { collections, getGitStates } from './db.js';
import { collections, getDb, getGitStates, type UserPodContextDoc } from './db.js';
import { enrichCollisionMemory } from './vectors.js';
import { buildUserLearningProfile } from './user-learning.js';
function comparableFile(raw?: string): string {
return (raw ?? '')
.trim()
.replace(/^(\?\?|[MADRCU!]{1,2})\s+/, '')
.split(/[\\/]/)
.pop()
?.toLowerCase() ?? '';
return (
(raw ?? '')
.trim()
.replace(/^(\?\?|[MADRCU!]{1,2})\s+/, '')
.split(/[\\/]/)
.pop()
?.toLowerCase() ?? ''
);
}
/**
@@ -38,6 +41,30 @@ export async function recordObservation(ctx: EngineerContext): Promise<void> {
);
}
export async function recordUserPodContext(input: {
clerkUserId: string;
podId: string;
memberName?: string;
action: string;
metadata?: UserPodContextDoc['metadata'];
}): Promise<void> {
await persist('user pod context', async () =>
(await getDb()).collection<UserPodContextDoc>('user_pod_context').insertOne({
id: `upc_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`,
clerkUserId: input.clerkUserId,
podId: input.podId,
memberName: input.memberName,
action: input.action,
source: 'clerk',
observedAt: new Date().toISOString(),
metadata: input.metadata,
}),
);
void buildUserLearningProfile(input.clerkUserId).catch((err) =>
console.warn(`[memory] user learning refresh failed: ${(err as Error).message}`),
);
}
export async function recordCollision(collision: Collision): Promise<void> {
await persist('collision', async () =>
(await collections()).collisions.insertOne(await enrichCollisionMemory(collision)),
@@ -50,6 +77,31 @@ export async function recordIntervention(intervention: Intervention): Promise<vo
);
}
/**
* Feature A — record that PodMan stayed quiet on a recurring collision because
* its signature was previously dismissed. Timestamped at the repeat (now), so
* the negative-feedback beat surfaces as recent `suppressed` activity rather
* than being re-synthesized from the old dismissal row.
*/
export async function recordSuppression(
collision: Collision,
priorInterventionId?: string,
priorDismissedAt?: string,
): Promise<void> {
await persist('suppression', async () =>
(await collections()).suppressions.insertOne({
id: `supp_${Date.now()}`,
podId: collision.podId,
collisionId: collision.id,
file: collision.file,
engineers: collision.engineers,
priorInterventionId,
priorDismissedAt,
suppressedAt: new Date().toISOString(),
}),
);
}
export async function hasRecentInterventionForCollision(
collision: Collision,
windowMs = Number(process.env.NUDGE_COOLDOWN_MS ?? '180000'),
@@ -138,11 +190,31 @@ export async function recordOutcome(outcome: InterventionOutcome): Promise<void>
/** Document counts per collection — used by the /api/memory/stats endpoint. */
export async function memoryStats(): Promise<Record<string, number>> {
const c = await collections();
const [observations, collisions, interventions, outcomes] = await Promise.all([
const db = await getDb();
const [
observations,
collisions,
interventions,
outcomes,
suppressions,
userPodContext,
userLearningProfiles,
] = await Promise.all([
c.observations.estimatedDocumentCount(),
c.collisions.estimatedDocumentCount(),
c.interventions.estimatedDocumentCount(),
c.outcomes.estimatedDocumentCount(),
c.suppressions.estimatedDocumentCount(),
db.collection<UserPodContextDoc>('user_pod_context').estimatedDocumentCount(),
db.collection('user_learning_profiles').estimatedDocumentCount(),
]);
return { observations, collisions, interventions, outcomes };
return {
observations,
collisions,
interventions,
outcomes,
suppressions,
userPodContext,
userLearningProfiles,
};
}
+282
View File
@@ -0,0 +1,282 @@
import type { Collision, EngineerContext, HermesJob, Pod, UserLearningProfile } from '@podman/shared';
import { getDb, type UserPodContextDoc } from './db.js';
interface EngineerStateDoc {
podId: string;
name: string;
changedFiles?: string[];
branch?: string | null;
recentCommit?: string | null;
gitUpdatedAt?: Date | string;
}
interface ConversationNoteDoc {
podId: string;
identity?: string;
kind?: string;
note?: string;
createdAt?: string;
}
function clean(value: unknown): string | undefined {
return typeof value === 'string' && value.trim() ? value.trim() : undefined;
}
function toIso(value: unknown): string {
if (value instanceof Date) return value.toISOString();
if (typeof value === 'string' && !Number.isNaN(Date.parse(value))) return new Date(value).toISOString();
return new Date().toISOString();
}
function uniq(values: Array<string | undefined>): string[] {
const seen = new Set<string>();
const out: string[] = [];
for (const value of values) {
const item = value?.trim();
if (!item) continue;
const key = item.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
out.push(item);
}
return out;
}
function top(values: string[], limit: number): string[] {
const counts = new Map<string, number>();
for (const value of values) {
const item = value.trim();
if (!item) continue;
counts.set(item, (counts.get(item) ?? 0) + 1);
}
return [...counts.entries()]
.sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
.slice(0, limit)
.map(([value]) => value);
}
function lowerRegex(values: string[]): RegExp[] {
return values.map((value) => new RegExp(`^${value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, 'i'));
}
function noteGoals(notes: ConversationNoteDoc[]): string[] {
const goalNotes = notes
.filter((note) => /goal|plan|todo|next|preference|prefers|likes|wants|needs/i.test(`${note.kind} ${note.note}`))
.map((note) => clean(note.note)?.slice(0, 180))
.filter(Boolean) as string[];
return uniq(goalNotes).slice(0, 8);
}
function inferCollaborationStyle(input: {
contexts: UserPodContextDoc[];
observations: EngineerContext[];
collisions: Collision[];
outcomes: Array<{ accepted?: boolean; wasRealCollision?: boolean }>;
notes: ConversationNoteDoc[];
jobs: HermesJob[];
}): string[] {
const style: string[] = [];
const actions = input.contexts.map((ctx) => ctx.action);
const pods = new Set(input.contexts.map((ctx) => ctx.podId));
if (pods.size > 1) style.push(`Works across ${pods.size} pods and carries context between rooms.`);
if (actions.filter((action) => action === 'joined_pod').length >= 3)
style.push('Frequently joins live rooms and collaborates synchronously.');
if (input.collisions.length > 0)
style.push(`Has been involved in ${input.collisions.length} coordination risk signals.`);
const accepted = input.outcomes.filter((outcome) => outcome.accepted).length;
if (accepted > 0) style.push(`Accepts Hermes help when useful (${accepted} accepted outcomes).`);
if (input.jobs.length > 0) style.push(`Delegates complex work to Hermes (${input.jobs.length} jobs).`);
if (input.notes.some((note) => /concise|short|brief/i.test(note.note ?? '')))
style.push('Prefers concise coordination.');
return style.slice(0, 8);
}
function inferWorkingStyle(input: {
observations: EngineerContext[];
gitStates: EngineerStateDoc[];
}): string[] {
const style: string[] = [];
const modes = top(input.observations.map((obs) => obs.mode ?? '').filter(Boolean), 2);
const activities = top(input.observations.map((obs) => obs.activity ?? '').filter(Boolean), 5);
const files = top(
[
...input.observations.map((obs) => obs.currentFile ?? ''),
...input.gitStates.flatMap((state) => state.changedFiles ?? []),
].filter(Boolean),
6,
);
if (modes.length) style.push(`Usually seen in ${modes.join(' and ')} mode.`);
if (activities.length) style.push(`Common work patterns: ${activities.join(', ')}.`);
if (files.length) style.push(`Frequently touches ${files.join(', ')}.`);
return style.slice(0, 8);
}
function inferKnowledge(input: {
observations: EngineerContext[];
gitStates: EngineerStateDoc[];
notes: ConversationNoteDoc[];
}): string[] {
const topics = top(
[
...input.observations.map((obs) => obs.researchTopic ?? ''),
...input.observations.map((obs) => obs.researchSource ?? ''),
...input.observations.map((obs) => obs.currentSymbol ?? ''),
...input.gitStates.flatMap((state) => state.changedFiles ?? []),
...input.notes
.filter((note) => /learn|knows|expert|worked on|decision/i.test(note.note ?? ''))
.map((note) => note.note?.slice(0, 120) ?? ''),
].filter(Boolean),
10,
);
return topics;
}
export async function buildUserLearningProfile(clerkUserId: string): Promise<UserLearningProfile | null> {
const db = await getDb();
const contexts = await db
.collection<UserPodContextDoc>('user_pod_context')
.find({ clerkUserId }, { projection: { _id: 0 } })
.sort({ observedAt: -1 })
.limit(500)
.toArray();
if (!contexts.length) return null;
const latest = contexts[0];
const identities = uniq([
...contexts.map((ctx) => ctx.memberName),
...contexts.map((ctx) => clean(ctx.metadata?.identity)),
...contexts.map((ctx) => clean(ctx.metadata?.email)),
]);
const email = clean(latest?.metadata?.email) ?? identities.find((item) => item.includes('@'));
const displayName = latest?.memberName ?? identities.find((item) => !item.includes('@')) ?? email;
const imageUrl = clean(latest?.metadata?.imageUrl);
const identityRegex = lowerRegex(identities);
const podIds = uniq(contexts.map((ctx) => ctx.podId));
const [pods, observations, gitStates, collisions, outcomes, notes, jobs] = await Promise.all([
db
.collection<Pod>('pods')
.find({ id: { $in: podIds } }, { projection: { _id: 0 } })
.toArray(),
db
.collection<EngineerContext>('observations')
.find({ engineerId: { $in: identities } }, { projection: { _id: 0, screenshotDataUrl: 0 } })
.sort({ observedAt: -1 })
.limit(500)
.toArray(),
db
.collection<EngineerStateDoc>('engineer_states')
.find({ name: { $in: identityRegex } }, { projection: { _id: 0 } })
.limit(100)
.toArray(),
db
.collection<Collision>('collisions')
.find({ engineers: { $in: identities } }, { projection: { _id: 0, embedding: 0 } })
.sort({ detectedAt: -1 })
.limit(100)
.toArray(),
db
.collection<{ podId: string; accepted?: boolean; wasRealCollision?: boolean }>('outcomes')
.find({ podId: { $in: podIds } }, { projection: { _id: 0 } })
.sort({ recordedAt: -1 })
.limit(100)
.toArray(),
db
.collection<ConversationNoteDoc>('conversation_notes')
.find({ identity: { $in: identities } }, { projection: { _id: 0 } })
.sort({ createdAt: -1 })
.limit(100)
.toArray(),
db
.collection<HermesJob>('hermes_jobs')
.find({ identity: { $in: identities } }, { projection: { _id: 0 } })
.sort({ createdAt: -1 })
.limit(100)
.toArray(),
]);
const podById = new Map(pods.map((pod) => [pod.id, pod]));
const podsSummary = podIds.map((podId) => {
const rows = contexts.filter((ctx) => ctx.podId === podId);
const sorted = [...rows].sort((a, b) => Date.parse(a.observedAt) - Date.parse(b.observedAt));
return {
podId,
podName: podById.get(podId)?.name,
visits: rows.filter((row) => row.action === 'joined_pod').length,
actions: top(rows.map((row) => row.action), 6),
firstSeenAt: toIso(sorted[0]?.observedAt),
lastSeenAt: toIso(sorted.at(-1)?.observedAt),
};
});
const profile: UserLearningProfile = {
clerkUserId,
displayName,
email,
imageUrl,
identities,
pods: podsSummary,
recentWork: observations.slice(0, 20).map((obs) => ({
podId: obs.podId,
file: obs.currentFile,
activity: obs.activity ?? obs.researchTopic,
at: obs.observedAt,
})),
collaborationStyle: inferCollaborationStyle({ contexts, observations, collisions, outcomes, notes, jobs }),
workingStyle: inferWorkingStyle({ observations, gitStates }),
goals: noteGoals(notes),
knowledge: inferKnowledge({ observations, gitStates, notes }),
counts: {
podActions: contexts.length,
observations: observations.length,
gitStates: gitStates.length,
collisionsInvolved: collisions.length,
outcomes: outcomes.length,
conversationNotes: notes.length,
hermesJobs: jobs.length,
},
updatedAt: new Date().toISOString(),
};
await db
.collection<UserLearningProfile>('user_learning_profiles')
.updateOne({ clerkUserId }, { $set: profile }, { upsert: true });
return profile;
}
export async function refreshUserLearningProfiles(): Promise<UserLearningProfile[]> {
const db = await getDb();
const ids = await db.collection<UserPodContextDoc>('user_pod_context').distinct('clerkUserId');
const profiles = await Promise.all(ids.map((id) => buildUserLearningProfile(String(id))));
return profiles.filter(Boolean) as UserLearningProfile[];
}
export async function getUserLearningProfileByIdentity(identity: string): Promise<UserLearningProfile | null> {
const db = await getDb();
const direct = await db
.collection<UserLearningProfile>('user_learning_profiles')
.findOne({ identities: { $regex: `^${identity.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, $options: 'i' } }, { projection: { _id: 0 } });
if (direct) return direct;
const context = await db
.collection<UserPodContextDoc>('user_pod_context')
.findOne(
{
$or: [
{ memberName: { $regex: `^${identity.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, $options: 'i' } },
{ 'metadata.identity': { $regex: `^${identity.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, $options: 'i' } },
],
},
{ sort: { observedAt: -1 }, projection: { _id: 0 } },
);
return context ? buildUserLearningProfile(context.clerkUserId) : null;
}
export async function listUserLearningProfiles(): Promise<UserLearningProfile[]> {
const db = await getDb();
return db
.collection<UserLearningProfile>('user_learning_profiles')
.find({}, { projection: { _id: 0 } })
.sort({ updatedAt: -1 })
.toArray();
}
+7 -1
View File
@@ -49,7 +49,7 @@ function memoryText(collision: Collision): string {
.join('\n');
}
function cosine(a: number[], b: number[]): number {
export function cosine(a: number[], b: number[]): number {
const n = Math.min(a.length, b.length);
let dot = 0;
let aNorm = 0;
@@ -69,6 +69,12 @@ async function embed(text: string, inputType: 'document' | 'query'): Promise<num
return (await embedWithVoyage(text, inputType)) ?? embedWithGemini(text, inputType);
}
export async function semanticSimilarity(a: string, b: string): Promise<number | null> {
const [va, vb] = await Promise.all([embed(a, 'query'), embed(b, 'document')]);
if (!va || !vb) return null;
return cosine(va, vb);
}
async function embedWithVoyage(
text: string,
inputType: 'document' | 'query',
+59 -6
View File
@@ -1,5 +1,5 @@
import type { Pod, PodInput } from '@podman/shared';
import { collections } from '../memory/db.js';
import { collections, getDb, type UserPodContextDoc } from '../memory/db.js';
const NO_ID = { projection: { _id: 0 } } as const;
@@ -59,14 +59,67 @@ function now(): string {
return new Date().toISOString();
}
function regexExact(value: string): RegExp {
return new RegExp(`^${value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`, 'i');
}
function profileString(value: unknown): string | undefined {
return typeof value === 'string' && value.trim() ? value.trim() : undefined;
}
function matchesMember(doc: UserPodContextDoc, member: string): boolean {
const key = member.trim().toLowerCase();
return [doc.memberName, doc.metadata?.identity, doc.metadata?.email]
.filter(Boolean)
.some((value) => String(value).trim().toLowerCase() === key);
}
async function hydrateMemberProfiles(pod: Pod): Promise<Pod> {
if (!pod.members.length) return pod;
const db = await getDb();
const matchers = pod.members.map(regexExact);
const docs = await db
.collection<UserPodContextDoc>('user_pod_context')
.find(
{
$or: [
{ memberName: { $in: matchers } },
{ 'metadata.identity': { $in: matchers } },
{ 'metadata.email': { $in: matchers } },
],
},
{ projection: { _id: 0 } },
)
.sort({ observedAt: -1 })
.limit(500)
.toArray();
const memberProfiles: NonNullable<Pod['memberProfiles']> = {};
for (const member of pod.members) {
const doc = docs.find((row) => matchesMember(row, member));
const imageUrl = profileString(doc?.metadata?.imageUrl);
if (!doc || !imageUrl) continue;
memberProfiles[member] = {
displayName: doc.memberName ?? member,
email: profileString(doc.metadata?.email),
imageUrl,
};
}
return Object.keys(memberProfiles).length ? { ...pod, memberProfiles } : pod;
}
async function hydratePods(pods: Pod[]): Promise<Pod[]> {
return Promise.all(pods.map((pod) => hydrateMemberProfiles(pod)));
}
export async function listPods(): Promise<Pod[]> {
const c = await collections();
return c.pods.find({}, NO_ID).sort({ createdAt: 1 }).toArray();
return hydratePods(await c.pods.find({}, NO_ID).sort({ createdAt: 1 }).toArray());
}
export async function getPod(id: string): Promise<Pod | null> {
const c = await collections();
return c.pods.findOne({ id }, NO_ID);
const pod = await c.pods.findOne({ id }, NO_ID);
return pod ? hydrateMemberProfiles(pod) : null;
}
export async function createPod(input: PodInput): Promise<Pod> {
@@ -92,7 +145,7 @@ export async function createPod(input: PodInput): Promise<Pod> {
};
try {
await c.pods.insertOne({ ...pod });
return pod;
return hydrateMemberProfiles(pod);
} catch (err) {
if (isDuplicateKey(err)) continue;
throw err;
@@ -119,7 +172,7 @@ export async function updatePod(id: string, patch: PodInput): Promise<Pod | null
{ $set: set },
{ returnDocument: 'after', projection: { _id: 0 } },
);
return updated ?? null;
return updated ? hydrateMemberProfiles(updated) : null;
}
export async function deletePod(id: string): Promise<boolean> {
@@ -135,7 +188,7 @@ async function setMembers(id: string, members: string[]): Promise<Pod | null> {
{ $set: { members, updatedAt: now() } },
{ returnDocument: 'after', projection: { _id: 0 } },
);
return updated ?? null;
return updated ? hydrateMemberProfiles(updated) : null;
}
export async function addMember(id: string, rawName: unknown): Promise<Pod | null> {
+91 -7
View File
@@ -12,7 +12,9 @@ import {
recordOutcome,
hasRecentInterventionForCollision,
memoryStats,
recordUserPodContext,
} from './memory/store.js';
import { clerkAuthMiddleware, requestUser } from './auth.js';
import { closeMemory, initMemory } from './memory/db.js';
import {
listPods,
@@ -40,6 +42,10 @@ import {
getLiveConversationContext,
recordLiveConversationNote,
} from './live-conversation/context.js';
import {
listUserLearningProfiles,
refreshUserLearningProfiles,
} from './memory/user-learning.js';
import {
abortHermesJob,
appendHermesJobEvent,
@@ -60,6 +66,7 @@ import type {
const app = express();
app.use(cors());
app.use(express.json());
app.use(clerkAuthMiddleware);
app.get('/health', (_req, res) => res.json({ ok: true }));
function stringArray(value: unknown): string[] {
@@ -72,6 +79,10 @@ function suggestedAction(value: unknown): SuggestedActionKind {
: 'ping_teammate';
}
function stringMeta(value: unknown): string | undefined {
return typeof value === 'string' && value.trim() ? value.trim() : undefined;
}
function hermesJobEventType(value: unknown): HermesJobEventType | null {
return value === 'accepted' ||
value === 'heartbeat' ||
@@ -90,11 +101,16 @@ function hermesJobEventType(value: unknown): HermesJobEventType | null {
app.post('/api/token', async (req, res) => {
const { room, identity, name, githubLogin } = req.body ?? {};
if (!room || !identity) return res.status(400).json({ error: 'room+identity required' });
const user = requestUser(req);
const at = new AccessToken(env.LIVEKIT_API_KEY, env.LIVEKIT_API_SECRET, {
identity,
name,
ttl: '4h',
metadata: JSON.stringify({ githubLogin: githubLogin ?? name }),
metadata: JSON.stringify({
githubLogin: githubLogin ?? name,
email: stringMeta(req.body?.profile?.email),
imageUrl: stringMeta(req.body?.profile?.imageUrl),
}),
});
at.addGrant({ roomJoin: true, room, canPublish: true, canSubscribe: true, canPublishData: true });
const agents = env.LIVEKIT_AGENT_NAME
@@ -113,6 +129,19 @@ app.post('/api/token', async (req, res) => {
departureTimeout: 20,
agents,
});
if (user) {
await recordUserPodContext({
clerkUserId: user.clerkUserId,
podId: String(room),
memberName: typeof name === 'string' ? name : String(identity),
action: 'joined_pod',
metadata: {
identity: String(identity),
email: stringMeta(req.body?.profile?.email) ?? null,
imageUrl: stringMeta(req.body?.profile?.imageUrl) ?? null,
},
});
}
res.json({ token: await at.toJwt(), url: env.LIVEKIT_URL });
});
@@ -150,6 +179,22 @@ app.get('/api/memory/stats', async (_req, res) => {
}
});
app.get('/api/memory/users', async (_req, res) => {
try {
res.json(await listUserLearningProfiles());
} catch (e) {
res.status(500).json({ error: (e as Error).message });
}
});
app.post('/api/memory/users/refresh', async (_req, res) => {
try {
res.json({ profiles: await refreshUserLearningProfiles() });
} catch (e) {
res.status(500).json({ error: (e as Error).message });
}
});
// Live presence: who is currently connected in each pod's LiveKit room.
app.get('/api/presence', async (_req, res) => {
try {
@@ -170,7 +215,24 @@ app.get('/api/pods', async (_req, res) => {
app.post('/api/pods', async (req, res) => {
try {
res.status(201).json(await createPod(req.body ?? {}));
const pod = await createPod(req.body ?? {});
const user = requestUser(req);
if (user) {
await recordUserPodContext({
clerkUserId: user.clerkUserId,
podId: pod.id,
memberName: stringMeta(req.body?.profile?.displayName),
action: 'created_pod',
metadata: {
podName: pod.name,
repo: pod.repo,
identity: stringMeta(req.body?.profile?.displayName) ?? null,
email: stringMeta(req.body?.profile?.email) ?? null,
imageUrl: stringMeta(req.body?.profile?.imageUrl) ?? null,
},
});
}
res.status(201).json(await getPod(pod.id));
} catch (e) {
res.status(400).json({ error: (e as Error).message });
}
@@ -186,6 +248,15 @@ app.patch('/api/pods/:id', async (req, res) => {
try {
const pod = await updatePod(req.params.id, req.body ?? {});
if (!pod) return res.status(404).json({ error: 'pod not found' });
const user = requestUser(req);
if (user) {
await recordUserPodContext({
clerkUserId: user.clerkUserId,
podId: pod.id,
action: 'updated_pod',
metadata: { podName: pod.name, repo: pod.repo },
});
}
res.json(pod);
} catch (e) {
res.status(400).json({ error: (e as Error).message });
@@ -203,7 +274,21 @@ app.post('/api/pods/:id/members', async (req, res) => {
try {
const pod = await addMember(req.params.id, req.body?.name ?? '');
if (!pod) return res.status(404).json({ error: 'pod not found' });
res.json(pod);
const user = requestUser(req);
if (user) {
await recordUserPodContext({
clerkUserId: user.clerkUserId,
podId: pod.id,
memberName: typeof req.body?.name === 'string' ? req.body.name.trim() : undefined,
action: 'added_member',
metadata: {
identity: typeof req.body?.name === 'string' ? req.body.name.trim() : null,
email: stringMeta(req.body?.profile?.email) ?? null,
imageUrl: stringMeta(req.body?.profile?.imageUrl) ?? null,
},
});
}
res.json(await getPod(pod.id));
} catch (e) {
res.status(400).json({ error: (e as Error).message });
}
@@ -306,15 +391,14 @@ app.post('/api/internal/pods/:id/live-conversation/:sessionId/note', async (req,
const note = typeof req.body?.note === 'string' ? req.body.note : '';
const identity = typeof req.body?.identity === 'string' ? req.body.identity : undefined;
const kind = typeof req.body?.kind === 'string' ? req.body.kind : undefined;
res.status(201).json(
await recordLiveConversationNote({
const saved = await recordLiveConversationNote({
podId: req.params.id,
sessionId: req.params.sessionId,
identity,
kind,
note,
}),
);
});
res.status(201).json(saved);
} catch (e) {
res.status(400).json({ error: (e as Error).message });
}
+23 -1
View File
@@ -7,6 +7,11 @@ const ai = new GoogleGenAI({ apiKey: env.GEMINI_API_KEY });
const SCHEMA = {
type: Type.OBJECT,
properties: {
mode: {
type: Type.STRING,
description:
'editing when an IDE/editor/terminal is primary; research for browser docs/SDK pages',
},
currentFile: {
type: Type.STRING,
description: 'open file path if visible, e.g. src/auth/session.ts',
@@ -20,13 +25,24 @@ const SCHEMA = {
type: Type.BOOLEAN,
description: 'dirty git gutter / modified markers visible',
},
researchTopic: {
type: Type.STRING,
description: 'topic being researched when mode is research, e.g. LiveKit agents setup',
},
researchSource: {
type: Type.STRING,
description: 'source domain when mode is research, e.g. docs.livekit.io',
},
confidence: { type: Type.NUMBER, description: '0..1 confidence in this read' },
},
propertyOrdering: [
'mode',
'currentFile',
'currentSymbol',
'activity',
'hasUnpushedChanges',
'researchTopic',
'researchSource',
'confidence',
],
} as const;
@@ -43,7 +59,10 @@ export async function analyzeFrame(
role: 'user',
parts: [
{
text: "You are PodMan watching an engineer's screen. Identify what file/symbol they are working on and whether there are uncommitted edits. JSON only.",
text:
"You are PodMan watching an engineer's screen. Return JSON only. " +
"If the primary window is an IDE/editor/terminal, set mode='editing' and identify the file, symbol, activity, and whether uncommitted edits are visible. " +
"If the primary window is a browser/docs/SDK/reference page, set mode='research', leave currentFile empty unless a file path is clearly visible, and extract researchTopic plus researchSource as the source domain.",
},
{ inlineData: { mimeType: 'image/jpeg', data: jpeg.toString('base64') } },
],
@@ -63,6 +82,9 @@ export async function analyzeFrame(
currentFile: parsed.currentFile,
currentSymbol: parsed.currentSymbol,
activity: parsed.activity,
mode: parsed.mode,
researchTopic: parsed.researchTopic,
researchSource: parsed.researchSource,
hasUnpushedChanges: parsed.hasUnpushedChanges,
confidence: parsed.confidence ?? 0.5,
observedAt: new Date().toISOString(),
+1
View File
@@ -98,6 +98,7 @@ Additive routes in `backend/src/server.ts` (shared file — additive only).
| `collisions` | **collision** nodes; `collides` (eng→col) + `touches` (file→col) |
| `interventions` | **intervention** nodes; `warns` (col→intervention) |
| `outcomes` | `learned_from` (intervention→owner) on accepted; flips nodes to `learned` |
| `suppressions` | `suppressed` activity beats — a dismissed signature recurred and PodMan stayed quiet (negative-feedback made visible; written at repeat time by the agent) |
Metrics (learned owners / open risk paths / accept rate) are live counts.
+56
View File
@@ -0,0 +1,56 @@
# PodMan — Demo Runbook (final, read-off)
> One page. Reflects the **current live state** (`podman.live`) as of the final hour.
> Companion to `docs/demo.md` (full 4-min script) — this is the de-risked version.
## ⛔ Read first — last-hour changes that affect the demo
1. **Suppression is DISABLED for the demo** (`shouldIntervene` only filters `info`).
Every real collision now surfaces — good (no accidental silencing). **But the
`docs/demo.md` 2:10 beat "it learned to stay QUIET / no card fires" will NOT
work.** Do the **escalate** half only (below). Do not promise silence on stage.
2. **Do NOT redeploy prod.** The live build has demo code not on `main`
(`userPodContext`); redeploying `main` would remove it *and* surface stale
"SUPPRESSED" beats. If the auto-deploy timer is on, stop it before the run:
`systemctl stop podman-hermes-sync-deploy.timer`.
3. **Never debug on stage.** Narrate, fall back to the backup recording, keep moving.
## ✅ Pre-flight (do once, before you start)
- [ ] `curl https://podman.live/health``{"ok":true}`
- [ ] Deploy timer stopped (above); confirm exactly one agent: `systemctl status podman-platform-agent`
- [ ] Browser open on the pod; **click "Enable audio"** in-room (TTS needs it)
- [ ] Backup recording cued on a second device
- [ ] Two screen-shares live (the two "engineers")
## 🎬 The 3-minute path (the coherent, working story)
| Time | Do | Say (land the value) |
|---|---|---|
| **0:00** | Pod view, two live screen tiles | "Coordination is the bottleneck now, not coding. PodMan watches everyone's work live — no one has to ask 'what are you on?'" |
| **0:30** | Point at tiles; show activity stream filling | "Real screen-shares over **LiveKit**; each frame → **Gemini Vision** returns file/symbol/activity — a perception layer, not a chatbot." |
| **1:05** | Two engineers edit the **same file**, unpushed | "GitHub can't see this — nothing's pushed. We fuse live screen context with **local git truth**." → **collision card** appears. |
| **1:40** | **Accept** the card | Real **GitHub sync-PR** is created + outcome recorded. "One tap closes the loop." |
| **2:10** | **Trigger the same collision again** | Card now says **"Seen before."** and escalates straight to the **Gemini TTS voice** cue. "It recalled the prior event from **MongoDB Atlas vector search** and escalated — that's the learning." |
| **2:35** | Cut to **GraphView** | "This graph is materialized **live from MongoDB right now** — and here's the **`learned_from` edge**: PodMan learned who owns this file from the accepted outcome. Not a mock." |
| **3:00** | Stack + close | "All on **DigitalOcean**, systemd-supervised; ambient score is **Gemini Lyria**. Coordination awareness, collisions caught before they cost an afternoon, memory that sharpens each session." |
## 🗣️ Sponsor coverage (say each ≥ once)
- **Gemini** — Vision (0:30), TTS voice (2:10), Lyria (3:00) — add Live conversation if time allows
- **LiveKit** — "real screen-shares over LiveKit" (0:30), TTS over LiveKit (2:10)
- **MongoDB** — "Atlas vector-search recall" + "graph materialized live from MongoDB" (2:10, 2:35)
- **DigitalOcean** — "all on DigitalOcean, systemd-supervised" (3:00)
## 🚑 If it breaks
| Failure | Recovery |
|---|---|
| Voice doesn't fire | Cut to the card, say the line aloud — cards are the default path |
| Collision won't trigger | Use the backup recording for that beat; keep narrating |
| Graph looks empty/odd | Reload once; if still off, narrate from the recording |
| Agent silent | Suspect a duplicate `podman-hermes` process, not the UI — but **don't fix on stage** |
## Optional (only if ahead of time): the five-minute-meeting killer
Open the live voice conversation; ask *"PodMan, what is everyone working on, and where is the collision detector?"* → answers via real tool calls (**Gemini Live API**). Skip entirely if tight on time or if it drops.
---
**The spine:** live awareness → collision caught pre-push → accept → **"Seen before." + voice + `learned_from` edge**. That's the win. Suppression/"stays quiet" is intentionally out this run.
+50 -35
View File
@@ -2,8 +2,8 @@
**Theme:** Continual Learning. **Hard limit:** 4:00. Practice to land at 3:45.
**The one-line story:** writing code isn't the bottleneck anymore — *coordinating
who's writing what* is. PodMan is a pair programmer for the whole team: it watches
**The one-line story:** writing code isn't the bottleneck anymore — _coordinating
who's writing what_ is. PodMan is a pair programmer for the whole team: it watches
every member's work in real time, gives everyone live status without anyone having
to interrupt anyone, and learns your team's dynamics so it nudges less and helps
more over time.
@@ -25,7 +25,7 @@ recovery time across every teammate, every day, and that is the value.
> everyone's work live, so anyone can see another's status without interrupting
> them — and it learns your team as it goes."
*On screen:* the pod view, two teammates joined, screen-share tiles live.
_On screen:_ the pod view, two teammates joined, screen-share tiles live.
### 0:301:05 — Real-time team awareness (LiveKit + Gemini Vision)
@@ -35,27 +35,41 @@ recovery time across every teammate, every day, and that is the value.
symbol, activity — not a chatbot, a perception layer."
- Show the live activity stream filling in (Signals vs Reasoning sections).
- Land the value: "This is the part that replaces 'what are you working on?' —
every teammate's current work is just *visible*, in real time. Nobody had to
every teammate's current work is just _visible_, in real time. Nobody had to
ask."
*Built-by-us callout:* `backend/src/vision/gemini.ts`, the LiveKit agent worker.
_Built-by-us callout:_ `backend/src/vision/gemini.ts`, the LiveKit agent worker.
### 1:051:50 — The catch (detection + first intervention)
### 1:051:40 — The catch (detection + first intervention)
- Have alice and bob both edit the **same file** with unpushed changes.
- "Normally nobody notices until merge time. GitHub can't see this — nothing's
pushed. Our detector fuses live screen context with **local git truth** from a
watcher on each laptop."
- A collision card appears: *"alice + bob both on detector.ts (unpushed)."*
- Let the **Gemini TTS** urgent voice fire once over LiveKit: *"alice and bob are
both editing detector.ts. Please sync before pushing."*
- A collision card appears: _"alice + bob both on detector.ts (unpushed)."_
- Let the **Gemini TTS** urgent voice fire once over LiveKit: _"alice and bob are
both editing detector.ts. Please sync before pushing."_
- Land the value: "That's a merge conflict and a wasted afternoon caught before it
happened — and neither of them had to be tracking the other."
*Built-by-us callout:* `collision/detector.ts`, `action/hermes.ts`,
_Built-by-us callout:_ `collision/detector.ts`, `action/hermes.ts`,
`voice/live.ts`.
### 1:502:50 — Continual learning (the theme — the money shot)
### 1:402:10 — Cross-channel overlap (research + code)
- Keep alice editing `livekit.py`.
- Have bob share a browser tab on LiveKit docs/SDK pages.
- A collaboration nudge appears: _"🤝 bob is researching LiveKit agents
(docs.livekit.io) while alice edits livekit.py — sync up before duplicating
effort."_
- Land the value: "This is not a merge conflict. PodMan caught duplicated effort
across channels — code on one screen, research on another — and nudged the team
before two people solved the same problem twice."
_Built-by-us callout:_ `vision/gemini.ts`, `collision/research.ts`,
`memory/vectors.ts`.
### 2:102:50 — Continual learning (the theme — the money shot)
This is the differentiator. Two beats, both from pre-seeded memory:
@@ -71,15 +85,15 @@ This is the differentiator. Two beats, both from pre-seeded memory:
that adapts on the recalled outcome."
- Optional: show `/api/memory/stats` counts climbing — accumulated experience.
*Built-by-us callout:* `memory/vectors.ts` ($vectorSearch), `memory/policy.ts`
_Built-by-us callout:_ `memory/vectors.ts` ($vectorSearch), `memory/policy.ts`
(outcome-conditioned gate), `memory/store.ts`.
### 2:503:30 — The five-minute meeting, killed (Gemini Live API)
- Frame it: "Instead of breaking a teammate's focus to ask what they're up to,
you ask PodMan."
- Open the live voice conversation. Ask out loud: *"PodMan, what is everyone
working on, and where is the collision detector implemented?"*
- Open the live voice conversation. Ask out loud: _"PodMan, what is everyone
working on, and where is the collision detector implemented?"_
- It answers with **real tool calls**`search_repo`, git history, current
collisions — not guesses.
- "This is the **Gemini Live API**, streaming speech-to-speech over LiveKit, with
@@ -87,7 +101,7 @@ This is the differentiator. Two beats, both from pre-seeded memory:
and live state. That's the status sync, answered in seconds, with zero recovery
tax on anyone else."
*Built-by-us callout:* `agents/podman-live-conversation/agent.py`.
_Built-by-us callout:_ `agents/podman-live-conversation/agent.py`.
### 3:303:50 — Stack + close
@@ -106,23 +120,23 @@ This is the differentiator. Two beats, both from pre-seeded memory:
## Sponsor-prize coverage (say each at least once)
| Prize | Spoken moment | Segment |
| --- | --- | --- |
| **Gemini** | Vision perception, Live API agent w/ tools, TTS voice, Lyria score | 0:30, 1:05, 2:50, 3:30 |
| **LiveKit** | "real screen shares over LiveKit", agent subscribes, TTS audio track, live voice | 0:30, 1:05, 3:30 |
| **MongoDB** | "Atlas vector search recalling past events" | 1:50 |
| **DigitalOcean** | "all on DigitalOcean, systemd-supervised workers" | 3:30 |
| Prize | Spoken moment | Segment |
| ---------------- | -------------------------------------------------------------------------------- | ---------------------- |
| **Gemini** | Vision perception, Live API agent w/ tools, TTS voice, Lyria score | 0:30, 1:05, 2:50, 3:30 |
| **LiveKit** | "real screen shares over LiveKit", agent subscribes, TTS audio track, live voice | 0:30, 1:05, 3:30 |
| **MongoDB** | "Atlas vector search recalling past events" | 1:50 |
| **DigitalOcean** | "all on DigitalOcean, systemd-supervised workers" | 3:30 |
---
## If something breaks (live recovery)
| Failure | Recovery |
| --- | --- |
| Voice doesn't fire | Cut to the card; say the line aloud; cards are the default path anyway. |
| Live conversation drops | Skip 2:503:30; lean longer on the learning beat. |
| Collision won't trigger | Use the backup recording for that beat; keep narrating. |
| Agent flapping | Pre-checked — but if so, `systemctl restart podman-platform-agent`. |
| Failure | Recovery |
| ----------------------- | ----------------------------------------------------------------------- |
| Voice doesn't fire | Cut to the card; say the line aloud; cards are the default path anyway. |
| Live conversation drops | Skip 2:503:30; lean longer on the learning beat. |
| Collision won't trigger | Use the backup recording for that beat; keep narrating. |
| Agent flapping | Pre-checked — but if so, `systemctl restart podman-platform-agent`. |
**Rule:** never debug on stage. Narrate, fall back to recording, keep moving.
@@ -130,12 +144,13 @@ This is the differentiator. Two beats, both from pre-seeded memory:
## Tight timing summary
| Time | Beat |
| --- | --- |
| 0:00 | Problem (coordination cost) + hook + original-work line |
| 0:30 | Real-time team awareness — LiveKit + Gemini Vision |
| 1:05 | The catch — collision caught before merge |
| 1:50 | **Continual learning — quiet + escalate** |
| Time | Beat |
| ---- | ---------------------------------------------------------- |
| 0:00 | Problem (coordination cost) + hook + original-work line |
| 0:30 | Real-time team awareness — LiveKit + Gemini Vision |
| 1:05 | The catch — collision caught before merge |
| 1:40 | Cross-channel overlap — research + code nudge |
| 2:10 | **Continual learning — quiet + escalate** |
| 2:50 | The five-minute meeting, killed — Gemini Live conversation |
| 3:30 | DigitalOcean + Lyria + close |
| 3:50 | Buffer |
| 3:30 | DigitalOcean + Lyria + close |
| 3:50 | Buffer |
+189
View File
@@ -0,0 +1,189 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PodMan — Demo Runbook</title>
<style>
:root{
--bg:#0b0f17; --panel:#121826; --panel2:#0e1421; --ink:#e6edf6; --muted:#9fb0c3;
--line:#23304a; --blue:#5b9bff; --violet:#a78bfa; --red:#ff6b6b; --amber:#f4b942;
--green:#43d39e; --slate:#8aa0b8;
}
*{box-sizing:border-box}
body{margin:0;background:radial-gradient(1200px 600px at 70% -10%,#15203a 0,var(--bg) 55%);
color:var(--ink);font:15px/1.55 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;}
.wrap{max-width:960px;margin:0 auto;padding:32px 22px 80px}
header{border-bottom:1px solid var(--line);padding-bottom:18px;margin-bottom:26px}
h1{font-size:30px;margin:0 0 6px;letter-spacing:-.3px}
h1 .dot{color:var(--green)}
.sub{color:var(--muted);font-size:15px;max-width:680px}
h2{font-size:13px;text-transform:uppercase;letter-spacing:.14em;color:var(--blue);
margin:34px 0 12px;border-left:3px solid var(--blue);padding-left:10px}
pre{background:var(--panel2);border:1px solid var(--line);border-radius:10px;
padding:16px 18px;overflow:auto;color:#cfe0f5;font:12.5px/1.45 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
.grid{display:grid;gap:14px}
.card{background:linear-gradient(180deg,var(--panel),var(--panel2));border:1px solid var(--line);
border-radius:12px;padding:16px 18px}
table{width:100%;border-collapse:collapse;font-size:14px}
th,td{text-align:left;padding:9px 10px;border-bottom:1px solid var(--line);vertical-align:top}
th{color:var(--muted);font-weight:600;font-size:12px;text-transform:uppercase;letter-spacing:.06em}
td.t{white-space:nowrap;color:var(--violet);font-variant-numeric:tabular-nums;font-weight:600}
.say{color:var(--green)}
.pill{display:inline-block;padding:1px 8px;border-radius:999px;font-size:11px;font-weight:700;letter-spacing:.04em}
.p-blue{background:rgba(91,155,255,.16);color:var(--blue)} .p-violet{background:rgba(167,139,250,.16);color:var(--violet)}
.p-green{background:rgba(67,211,158,.16);color:var(--green)} .p-amber{background:rgba(244,185,66,.16);color:var(--amber)}
.warn{background:rgba(255,107,107,.08);border:1px solid rgba(255,107,107,.4);border-radius:12px;padding:14px 18px}
.warn h3{margin:0 0 8px;color:var(--red);font-size:14px;letter-spacing:.02em}
ol{margin:0;padding-left:20px} ol li{margin:6px 0}
code{background:#0a1220;border:1px solid var(--line);border-radius:5px;padding:1px 6px;color:#bcd4f2;font-size:12.5px}
.muted{color:var(--muted)} .spine{color:var(--green);font-weight:600}
footer{margin-top:40px;color:var(--muted);font-size:12.5px;border-top:1px solid var(--line);padding-top:14px}
</style>
</head>
<body>
<div class="wrap">
<header>
<h1>PodMan <span class="dot"></span> Demo Runbook</h1>
<div class="sub">A pair programmer for the whole team. It watches everyone's work live, catches
collisions <em>before the push</em> (when GitHub still can't see them), and learns the team's
dynamics so it nudges less and helps more over time.</div>
</header>
<div class="card">
<strong>The 10-second story:</strong> <span class="muted">Coding isn't the bottleneck anymore —
<em>coordination</em> is. PodMan gives a team real-time awareness with zero interruptions, catches
the pre-push collision that costs an afternoon, and the Team-Memory Graph proves it learned —
materialized live from MongoDB, not a mock.</span>
</div>
<h2>How it works — architecture</h2>
<pre>
░ PodMan data flow ░
┌──────────────┐ screen-share track ┌────────────────────────────┐
│ Engineers │ ──────────────────────▶ │ LiveKit Cloud │
│ (browsers) │ ◀───────────────────────│ room = "demo-pod" │
└──────┬───────┘ cards · voice · cues └─────────────┬──────────────┘
│ (data channel) │ frames @ ~1 fps
│ live graph (poll 5s) │ + local git truth
│ ▼
│ ┌────────────────────────────────────┐
│ │ HERMES AGENT (Node, podman-hermes)│
│ │ frame ─▶ GEMINI VISION (file/ │
│ │ symbol/activity) │
│ │ fuse with git watcher (unpushed!) │
│ │ ─▶ detect same-file collision │
│ │ ─▶ Gemini TTS voice over LiveKit │
│ └──────────────────┬───────────────────┘
│ │ read / write
┌──────┴────────────┐ materialize live ┌───────────▼──────────────────┐
│ TEAM-MEMORY │ ◀──────────────────────│ MongoDB Atlas │
│ GRAPH (SVG) │ graph/live.ts │ observations · collisions │
│ owns/edits/ │ │ interventions · outcomes │
│ collides/ │ │ team_model · graph_nodes/edges│
│ learned_from │ │ + $vectorSearch recall │
└───────────────────┘ └───────────────────────────────┘
Infra: DigitalOcean droplet · systemd-supervised API + agent · Caddy static frontend
Models: Gemini (Vision · TTS · Live voice · embeddings · Lyria ambient score)
</pre>
<h2>How it works — the continual-learning loop</h2>
<pre>
observe ──▶ store ──▶ predict ──▶ outcome ──▶ adapt
│ │ │ │ │
Gemini MongoDB same-file accept / learned_from
Vision + records overlap dismiss edge + owner
git truth detected (1 tap) persists
▲ │
└───────────────── recall ($vectorSearch) ◀─────────┘
"Seen before." → escalates on a repeat
The proof: one accept/dismiss tap — no retraining, no labels — changes the
NEXT run. Run 2 of the same collision says "Seen before." and a violet
learned_from edge appears on the graph. That edge is read straight from Atlas.
</pre>
<h2>The 3-minute demo (read off this)</h2>
<table>
<thead><tr><th>Time</th><th>Do</th><th>Say — land the value</th></tr></thead>
<tbody>
<tr><td class="t">0:00</td><td>Pod view, two live screen tiles</td>
<td class="say">"Coordination is the bottleneck now, not coding. PodMan watches everyone's work live — nobody has to ask 'what are you on?'"</td></tr>
<tr><td class="t">0:30</td><td>Point at tiles; activity stream fills</td>
<td class="say">"Real screen-shares over <b>LiveKit</b>; each frame → <b>Gemini Vision</b> → file/symbol/activity. A perception layer, not a chatbot."</td></tr>
<tr><td class="t">1:05</td><td>Two engineers edit the <b>same file</b>, unpushed</td>
<td class="say">"GitHub can't see this — nothing's pushed. We fuse live screen context with <b>local git truth</b>." → collision card appears.</td></tr>
<tr><td class="t">1:40</td><td><b>Accept</b> the card</td>
<td class="say">Real <b>GitHub sync-PR</b> created + outcome recorded. "One tap closes the loop."</td></tr>
<tr><td class="t">2:10</td><td><b>Trigger the same collision again</b></td>
<td class="say">Card now says <b>"Seen before."</b> and escalates to the <b>Gemini TTS voice</b>. "It recalled the prior event from <b>MongoDB Atlas vector search</b> and escalated — that's the learning."</td></tr>
<tr><td class="t">2:35</td><td>Cut to <b>GraphView</b></td>
<td class="say">"This graph is materialized <b>live from MongoDB right now</b> — and here's the <b>learned_from edge</b>: PodMan learned who owns this file. Not a mock."</td></tr>
<tr><td class="t">3:00</td><td>Stack + close</td>
<td class="say">"All on <b>DigitalOcean</b>, systemd-supervised; ambient score is <b>Gemini Lyria</b>. Awareness, collisions caught pre-push, memory that sharpens each session."</td></tr>
</tbody>
</table>
<h2>Sponsor coverage — say each ≥ once</h2>
<table>
<thead><tr><th>Prize</th><th>Spoken moment</th><th>Beat</th></tr></thead>
<tbody>
<tr><td><span class="pill p-blue">Gemini</span></td><td>Vision perception · TTS voice · Lyria score (Live conversation if time)</td><td class="muted">0:30 · 2:10 · 3:00</td></tr>
<tr><td><span class="pill p-violet">LiveKit</span></td><td>"real screen-shares over LiveKit"; TTS audio over LiveKit</td><td class="muted">0:30 · 2:10</td></tr>
<tr><td><span class="pill p-green">MongoDB</span></td><td>"Atlas vector-search recall" · "graph materialized live from MongoDB"</td><td class="muted">2:10 · 2:35</td></tr>
<tr><td><span class="pill p-amber">DigitalOcean</span></td><td>"all on DigitalOcean, systemd-supervised workers"</td><td class="muted">3:00</td></tr>
</tbody>
</table>
<h2 style="color:var(--red);border-color:var(--red)">⛔ Last-hour caveats — read before you run</h2>
<div class="warn">
<h3>1 · Suppression is DISABLED for the demo</h3>
<div class="muted">Every collision now surfaces (good — no accidental silencing). The old
<em>"it learned to stay QUIET / no card fires"</em> beat will <b>not</b> work. Use the
<b>escalate</b> beat instead: trigger the same collision again → <b>"Seen before." + voice +
learned_from edge</b>. Don't promise silence on stage.</div>
</div>
<div class="warn" style="margin-top:12px">
<h3>2 · Do NOT redeploy prod</h3>
<div class="muted">The live build has demo code not on <code>main</code> (e.g. <code>userPodContext</code>);
redeploying would remove it <em>and</em> surface stale "SUPPRESSED" beats. If the auto-deploy timer is on,
stop it: <code>systemctl stop podman-hermes-sync-deploy.timer</code>.</div>
</div>
<div class="warn" style="margin-top:12px">
<h3>3 · Never debug on stage</h3>
<div class="muted">Narrate, fall back to the backup recording, keep moving.</div>
</div>
<h2>If it breaks — live recovery</h2>
<table>
<thead><tr><th>Failure</th><th>Recovery</th></tr></thead>
<tbody>
<tr><td>Voice doesn't fire</td><td>Cut to the card, say the line aloud — cards are the default path. (Click "Enable audio" in-room first!)</td></tr>
<tr><td>Collision won't trigger</td><td>Use the backup recording for that beat; keep narrating.</td></tr>
<tr><td>Graph looks empty / odd</td><td>Reload once; if still off, narrate from the recording.</td></tr>
<tr><td>Agent silent</td><td>Suspect a duplicate <code>podman-hermes</code> process, not the UI — but don't fix on stage.</td></tr>
</tbody>
</table>
<h2>Pre-flight checklist</h2>
<div class="card">
<ol>
<li><code>curl https://podman.live/health</code><code>{"ok":true}</code></li>
<li>Deploy timer stopped; exactly one agent — <code>systemctl status podman-platform-agent</code></li>
<li>Browser on the pod; <b>click "Enable audio"</b> (TTS needs it)</li>
<li>Backup recording cued on a second device</li>
<li>Two screen-shares live (the two "engineers")</li>
</ol>
</div>
<footer>
<div class="spine">The spine: live awareness → collision caught pre-push → accept → "Seen before." + voice + learned_from edge → "this graph is live from MongoDB."</div>
<div style="margin-top:6px">Runs on the <code>demo</code> branch (ahead of <code>main</code>). Suppression intentionally off this run. · Generated for the final demo.</div>
</footer>
</div>
</body>
</html>
+10
View File
@@ -33,14 +33,24 @@ screen-share track (not an HTTP upload — frames arrive over LiveKit).
```ts
{
mode: 'editing' | 'research', // browser/docs/SDK research vs editor work
currentFile: string, // open file path, e.g. src/auth/session.ts
currentSymbol: string, // function/class under the cursor
activity: string, // editing | reading | debugging | terminal | PR review
hasUnpushedChanges: boolean, // dirty git gutter / modified markers visible
researchTopic: string, // e.g. "LiveKit agents setup", for research mode
researchSource: string, // source domain, e.g. "docs.livekit.io"
confidence: number // 0..1
}
```
When a frame shows a browser/docs/SDK page instead of an editor, Gemini Vision
classifies it as `mode: "research"` and extracts the topic/source. That feeds the
cross-channel overlap detector: one teammate researching LiveKit docs while
another edits `livekit.py` becomes a collaboration nudge, not a merge-conflict
alert. Editor/IDE frames remain `mode: "editing"` and use the existing file,
symbol, activity, and dirty-change fields.
**Latency/cost levers (in code):**
- `thinkingConfig: { thinkingBudget: 0 }` — minimal thinking for the ambient loop.
+7
View File
@@ -26,6 +26,10 @@ intervention that fits:
(`publishHermesIntervention` / `publishHermesMessage`). Default path.
- **Urgent voice** — only for `critical` collisions. `speak()` generates Gemini
TTS audio and publishes it as a LiveKit audio track.
- **Research overlap nudge** — a collaboration card when one engineer is editing
a file while another is researching the same topic in docs/browser context.
This uses `suggestedAction.kind = "ping_teammate"` and is spoken once for the
demo beat, but it is explicitly **not** a merge conflict.
Intervention text is short and deterministic (template, not an LLM call):
`Conflict: alice + bob both on detector.ts (unpushed). Seen before.` The spoken
@@ -33,6 +37,9 @@ line is phrased for natural TTS prosody. Each intervention is persisted to the
`interventions` collection; the teammate's accept/dismiss returns via
`POST /api/outcome`.
Research-overlap text is also deterministic:
`🤝 bob is researching LiveKit agents (docs.livekit.io) while alice edits livekit.py — sync up before duplicating effort.`
A per-pod cooldown (`NUDGE_COOLDOWN_MS`, default 3 min) and a single-shot
"active conflict" guard prevent repeat nagging; a conflict re-arms once it
resolves.
+26
View File
@@ -69,6 +69,10 @@ Key fields:
- `symbol`
- `engineers`
- `severity`
- `overlapKind` — optional; `file`/undefined for same-file collisions,
`research` for code-edit ↔ research overlaps.
- `researchTopic`, `researchSource`, `researcher`, `editor` — optional fields
present only for research overlaps.
- `memorySignature`
- `githubState`
- `detectedAt`
@@ -117,6 +121,28 @@ Key fields:
Primary use: accepted and dismissed outcomes drive exact recall, suppression,
and learned graph paths.
### `suppressions`
Durable negative-feedback proof: one record per *suppressed repeat* — a
previously-dismissed collision signature recurred and PodMan stayed quiet.
Written at repeat time by `backend/src/agent/podman.ts` (once per recurrence,
re-armed on resolution), materialized as `suppressed` activity by
`backend/src/graph/live.ts`.
Key fields:
- `id`
- `podId`
- `collisionId`
- `file`
- `engineers`
- `priorInterventionId` — the dismissed intervention this repeat matched
- `priorDismissedAt`
- `suppressedAt` — the repeat time (drives recency in the activity stream)
Index `{ podId: 1, suppressedAt: -1 }`. Counted in `/api/memory/stats`.
**Preserve in any DB cleanup** — this is visible learning evidence, not noise.
### `team_model`
Durable per-pod summary memory.
+166
View File
@@ -0,0 +1,166 @@
# Build spec: cross-channel overlap (code-edit ↔ research nudge)
> Self-contained spec for an implementing agent. Everything needed to build is
> here: current-state anchors, exact edits, code sketches, verification. Read
> the referenced files before editing.
## Context / why
Today PodMan only fires when **two engineers touch the same file** (basename
match) with unpushed changes — `backend/src/collision/detector.ts:36`. That makes
the system feel like a one-file trick. The wow we want: detect a **code↔research
overlap** — one teammate is *editing* `livekit.py` while another is *researching*
the same topic in a browser (LiveKit docs/SDK). PodMan nudges:
> 🤝 bob is deep in LiveKit docs while you edit livekit.py — sync up before duplicating effort.
This reframes PodMan from merge-conflict detector to a **team-coordination agent
that catches duplicated effort / knowledge overlap** — stronger continual-learning
story, distinct demo beat.
### Locked decisions
- **Signal capture:** Gemini vision on the existing LiveKit screenshare. When a
teammate shares a browser/docs window, vision classifies it `research` and
extracts `{researchTopic, researchSource}`. No browser extension, no new client
surface. Reuses `backend/src/agent.ts` + `backend/src/vision/gemini.ts`.
- **Matching:** semantic embeddings (reuse `embed()` + `cosine()` in
`backend/src/memory/vectors.ts`). Deterministic stem/keyword fallback fires when
an embed call returns `null`, so the demo path never depends on a live vector call.
- **Framing:** collaboration nudge (`ping_teammate`), spoken once for the beat.
## Current-state anchors (read these first)
- `backend/src/collision/detector.ts:14-20``fileKey()` stem/basename logic to mirror.
- `backend/src/collision/detector.ts:36-89``detectCollisions()` (same-file path; leave unchanged).
- `backend/src/agent/podman.ts:85-116``onScreenFrame()` orchestration; `:124-126` `conflictKey()`; `:128-183` `handle()`.
- `backend/src/agent/podman.ts:38-46``engineersOverlapOnFile()` (git ground-truth; research must skip it).
- `backend/src/vision/gemini.ts:7-70``SCHEMA`, prompt, and `EngineerContext` mapping.
- `backend/src/memory/vectors.ts:52-66` `cosine()`, `:68-70` private `embed()`.
- `backend/src/action/hermes.ts:46-62``publishHermesIntervention()` (speaks only if `voiceLine` passed).
- `frontend/src/livekit/useInterventions.ts:36-48` — data-channel handler (renders `intervention.message` verbatim).
- `shared/src/engineer.ts:6-23` `EngineerContext`; `shared/src/collision.ts:7-28` `Collision`.
**Off-spec gate:** nothing in `docs/` covers cross-channel overlap. Update specs
in step 1 **before** code (repo documentation-first rule).
## Concurrency / safety (several people build at once)
- Shared-contract edits are **additive optional fields only** — no signature
changes. Safe.
- `backend/src/agent/podman.ts` is hot: two in-place edits (`conflictKey`,
`handle` branch). `git pull --rebase` before pushing.
- **No required frontend change** — nudge text rides in `intervention.message`,
already rendered by `frontend/src/components/PodView.tsx`.
## Implementation steps
### 1. Specs first
- `docs/gemini.md` — vision classifies `editing` vs `research`; extracts `researchTopic`/`researchSource`.
- `docs/hermes.md` — new intervention type **research overlap** (collaboration nudge, `ping_teammate`, spoken once); explicitly NOT a merge conflict.
- `docs/mongodb.md` — new optional `Collision` fields.
- `docs/demo.md` — insert ~30s beat after the same-file collision.
### 2. Shared contract (additive, optional)
`shared/src/engineer.ts` — add to `EngineerContext`:
```ts
mode?: 'editing' | 'research';
researchTopic?: string;
researchSource?: string; // domain, e.g. "docs.livekit.io"
```
`shared/src/collision.ts` — add to `Collision`:
```ts
overlapKind?: 'file' | 'research'; // undefined = file (preserves current behavior)
researchTopic?: string;
researchSource?: string;
researcher?: string; // engineer doing research
editor?: string; // engineer editing the file
```
### 3. Vision: classify editing vs research
`backend/src/vision/gemini.ts` — extend `SCHEMA` with `mode`, `researchTopic`,
`researchSource` (+ propertyOrdering). Update the prompt so a browser/docs/SDK
frame returns `mode:'research'` + topic + source domain, else `mode:'editing'`
with existing IDE fields. Map new fields into the returned `EngineerContext`.
Keep `thinkingConfig.thinkingBudget:0` and `MEDIA_RESOLUTION_LOW`.
### 4. Semantic matcher helper (reuse embed/cosine)
`backend/src/memory/vectors.ts` — add and export:
```ts
export async function semanticSimilarity(a: string, b: string): Promise<number | null> {
const [va, vb] = await Promise.all([embed(a, 'query'), embed(b, 'document')]);
if (!va || !vb) return null;
return cosine(va, vb);
}
```
### 5. New detector (additive file)
`backend/src/env.ts` — add `RESEARCH_OVERLAP_THRESHOLD` (default `0.6`).
`backend/src/collision/research.ts` (new):
```ts
export interface ResearchOpts { similarity?: (a: string, b: string) => Promise<number | null>; threshold?: number; }
export async function detectResearchOverlaps(
contexts: EngineerContext[],
gitStates: Map<string, GitState> | undefined,
opts: ResearchOpts = {},
): Promise<Collision[]>;
```
Logic:
- **researchers** = contexts with `mode==='research'` && `researchTopic`.
- **editor files** = ground truth from `gitStates[*].changedFiles` (reliable) plus
any `mode==='editing'` `currentFile`; each tagged with its engineer.
- For each distinct (researcher, editor) pair on a file, score
`similarity("${topic} ${source}", "<file stem words> <symbol> <activity>")`
(default `similarity = semanticSimilarity`). Fire when `score >= threshold`
(default `RESEARCH_OVERLAP_THRESHOLD`).
- **Fallback:** if `score === null`, deterministic stem/token overlap (mirror
`fileKey()` stemming) — guarantees `livekit``livekit.py` fires offline.
- Dedupe to best-scoring file per researcher; require distinct engineers.
- Emit `Collision`: `id: col_research_<stem>_<Date.now()>`, `engineers:[editor, researcher]`,
`file:<editor file>`, `severity:'warn'`, `overlapKind:'research'`, plus
`researcher`, `editor`, `researchTopic`, `researchSource`.
### 6. Wire into orchestrator — `backend/src/agent/podman.ts`
- In `onScreenFrame`, after `detectCollisions(...)` (line ~100):
```ts
const research = await detectResearchOverlaps([...this.contexts.values()], gitStates);
const collisions = [...fileCollisions, ...research];
```
(concat **before** the re-arm + handle loops at `:104` / `:111` / `:115`).
- `conflictKey()` (`:124`) — namespace by overlap kind so research and file
overlaps on the same file don't share an edge-trigger key:
```ts
return `${collision.overlapKind ?? 'file'}:${comparableBasename(collision.file)}`;
```
- `gitOverlap` loop (`:111-113`) — guard: only call `engineersOverlapOnFile` when
`collision.overlapKind !== 'research'` (researcher won't have the file dirty;
leave `gitOverlap` undefined for research).
- `handle()` (`:128`) — branch on `overlapKind === 'research'`:
- `message`: `` `🤝 ${researcher} is researching ${researchTopic}` + (researchSource ? ` (${researchSource})` : '') + ` while ${editor} edits ${shortFile} — sync up before duplicating effort.` ``
- `voiceLine`: `` `${researcher} is researching ${researchTopic} while ${editor} works on ${shortFile}. Worth a quick sync.` ``
- `suggestedAction.kind = 'ping_teammate'`.
- Pass `voiceLine` to `publishHermesIntervention` **regardless of severity** so
the beat is spoken once. (For file collisions keep existing
`severity === 'critical' ? voiceLine : undefined`.)
- Existing `recallSimilar` / `shouldIntervene` gate stays unchanged.
### 7. Frontend (nice-to-have, cut if behind)
`frontend/src/livekit/useInterventions.ts` — capture `msg.collision.overlapKind`;
show a 🤝 badge on the card in `PodView.tsx`. Core path needs nothing.
## Verification
1. **Build:** `pnpm -r build` (shared builds first — order matters).
2. **Detector (offline, deterministic):** harness calling `detectResearchOverlaps`
with stubbed `opts.similarity`:
- researcher `{mode:'research', researchTopic:'LiveKit agent init', researchSource:'docs.livekit.io'}`
+ gitState with `livekit.py` dirty for another engineer → exactly one
`overlapKind:'research'` collision naming both.
- single engineer both sides → no overlap.
- `similarity` returns `null` → keyword fallback still fires on `livekit`.
3. **Live:** run the agent (systemd on the box, or local); one participant shares a
browser tab on LiveKit docs, another keeps `livekit.py` dirty (git sidecar
running) → 🤝 nudge card + one-time spoken cue. Fires once (edge-triggered),
re-arms after the browser closes.
4. **Regression:** same-file collision still fires, unaffected (separate
`conflictKey` namespace).
+299
View File
@@ -0,0 +1,299 @@
# Build spec: Work History "Coordination ROI" band
> Self-contained spec for an implementing agent. Everything needed to build is
> here: current-state anchors, exact edits, code sketches, verification. Read
> the referenced files before editing. Changes are **additive** — one optional
> field on a shared type, two new backend queries, one new presentational
> component. No route signature changes, no DB writes, no new deps.
## Context / why
The Work History dialog (opens when you click a teammate's "History" button)
today shows three raw counters (files / screen logs / git changes), a Recent
files bar list, and a Timeline. It is pure **activity volume** — it summarizes
nothing and shows none of PodMan's actual value.
We add a **Coordination ROI band** at the top of the dialog that answers "what
did PodMan save this person?": estimated rework-hours saved by clashes Hermes
caught early, plus hard defensible counts. This is a *summary* layer — it must
NOT re-stream the pod activity feed ("Team Memory"). Existing Recent files +
Timeline sections stay exactly as they are, below the new band.
### Locked decisions
- **Primary story = saved time / ROI.** One number leads: `~Xh Ym rework saved`.
- **Transparent heuristic.** The hours are a labeled estimate (`~`, `(est.)`)
with an info tooltip showing the per-clash breakdown. Technical judges can see
the model; we never claim precision.
- **Backend extension required** (collisions/interventions are not in the
current `MemberWorkHistory` payload). Field is optional → old frontends and
pods with no collisions degrade gracefully (band hidden).
- **Cut:** editing/research focus donut. ROI is the single story; a donut
dilutes it.
- **Credit split** across involved engineers so a 2-person clash does not
double-count the pod total.
## Heuristic (this is the contract — implement exactly)
For each `collisions` doc in the window where the member is involved AND an
`interventions` doc exists for it (Hermes actually surfaced it):
1. **Eligibility (must be "real"):** count the collision only if
`gitOverlap === true` OR `severity === 'critical'`. Otherwise skip.
2. **Weight by kind/severity:**
| condition | minutesEach |
|---|---|
| `overlapKind === 'research'` | 10 |
| `severity === 'critical'` (same-file) | 45 |
| `severity === 'warn'` (same-file) | 20 |
| `severity === 'info'` (same-file) | 10 |
(Evaluate research first; a research overlap is always 10 regardless of severity.)
3. **Member share:** `minutesEach / max(1, engineers.length)`. Sum all shares →
`savedMinutes` (round to nearest integer).
4. **breakdown[]:** group eligible collisions by label
(`"critical same-file"`, `"warn same-file"`, `"research overlap"`, etc.),
each entry `{ label, count, minutesEach }` — for the tooltip.
Member "involved" = `engineers` array contains the member (case-insensitive),
OR `researcher`/`editor` equals the member (case-insensitive). Reuse the
existing `sameMember()` helper logic.
Hard counts (no estimation):
- `clashesCaught` = number of eligible collisions (the integer behind the hours).
- `filesDeconflicted` = distinct `collision.file` values across eligible collisions.
- `conflictFreeCommits` / `totalCommits`: see "commits" note below.
### Commits note (keep simple, defensible)
There is no per-commit log in the window. Use the git ground-truth we have:
`totalCommits` = count of distinct `changedFiles` for the member from
`engineer_states` (already loaded as `gitState.changedFiles.length`).
`conflictFreeCommits` = `totalCommits - filesDeconflicted` (clamped ≥ 0). This
reads as "files in flight that never hit a clash". Label it in the UI as
**"conflict-free files"**, not commits, so the wording matches the data. (The
band copy below already says "files".)
## Current-state anchors (read these first)
- `shared/src/member-history.ts:24-36``MemberWorkHistory` interface (add `roi?`).
- `backend/src/activity/member-history.ts:73-174``getMemberWorkHistory()`.
- `:83-94` — the `Promise.all` that loads `observations` + `engineer_states`. Add collisions/interventions here.
- `:45-47``sameMember()` helper to reuse for involvement check.
- `:124``gitState` already in scope; `:169` uses `gitState.changedFiles.length`.
- `:161-173` — the returned object (add `roi`).
- `backend/src/memory/db.ts:47-48` — canonical collection names `collisions`, `interventions` (typed `Collision`, `Intervention`).
- `backend/src/activity/store.ts:167-178` — reference query shape for both collections.
- `shared/src/collision.ts:7-36``Collision` (`engineers`, `severity`, `overlapKind`, `gitOverlap`, `researcher`, `editor`, `file`, `detectedAt`).
- `shared/src/intervention.ts:8-18``Intervention` (`collisionId`, `podId`).
- `frontend/src/components/PodView.tsx:33-35` — type imports from `@podman/shared`.
- `frontend/src/components/PodView.tsx:1035-1041` — the `history && !loading && !error` block; the 3-stat grid is the insert point (band goes ABOVE it).
- `frontend/src/components/PodView.tsx:1093-1100``HistoryStat` (style reference for new sub-stats).
- `frontend/src/components/PodView.tsx``timeLabel()` exists for relative time; reuse if needed.
- Route is unchanged: `backend/src/server.ts:502-506` already returns whatever `getMemberWorkHistory` produces.
## Edit 1 — shared type (`shared/src/member-history.ts`)
Add to the `MemberWorkHistory` interface (after `timeline`):
```ts
/** Coordination ROI summary — clashes Hermes caught for this member. Optional
* so pods with no collisions / older payloads render without the band. */
roi?: MemberWorkHistoryRoi;
}
export interface MemberWorkHistoryRoi {
/** Estimated rework minutes saved (heuristic, labeled "~/est." in UI). */
savedMinutes: number;
/** Eligible collisions caught early (hard count). */
clashesCaught: number;
/** Distinct files that hit an eligible clash. */
filesDeconflicted: number;
/** Member files in flight that never hit a clash. */
conflictFreeFiles: number;
/** Total member files in flight (git changedFiles). */
totalFiles: number;
/** Per-kind breakdown for the tooltip. */
breakdown: { label: string; count: number; minutesEach: number }[];
}
```
`shared/src/index.ts:56-59` already re-exports the member-history types via a
`export type { ... }` block — add `MemberWorkHistoryRoi` to that list.
## Edit 2 — backend (`backend/src/activity/member-history.ts`)
1. Import the types: `import type { Collision } from '@podman/shared';` (Intervention only needs `collisionId`, can stay untyped or import too).
2. In the `Promise.all` at `:83`, add two queries (windowed, podId-scoped):
```ts
db.collection<Collision>('collisions')
.find({ podId, detectedAt: { $gte: since } }, { projection: { _id: 0 } })
.sort({ detectedAt: -1 })
.limit(200)
.toArray(),
db.collection<{ collisionId: string }>('interventions')
.find({ podId }, { projection: { collisionId: 1, _id: 0 } })
.toArray(),
```
3. After building `fileRows`, compute `roi` with a new local helper
`computeRoi(member, collisions, interventions, gitState)`:
```ts
function computeRoi(
member: string,
collisions: Collision[],
interventionCollisionIds: Set<string>,
changedFileCount: number,
): MemberWorkHistoryRoi {
const involved = (c: Collision) =>
c.engineers?.some((e) => sameMember(e, member)) ||
sameMember(c.researcher, member) ||
sameMember(c.editor, member);
const eligible = collisions.filter(
(c) =>
involved(c) &&
interventionCollisionIds.has(c.id) &&
(c.gitOverlap === true || c.severity === 'critical'),
);
const weightOf = (c: Collision): { label: string; minutes: number } => {
if (c.overlapKind === 'research') return { label: 'research overlap', minutes: 10 };
if (c.severity === 'critical') return { label: 'critical same-file', minutes: 45 };
if (c.severity === 'warn') return { label: 'warn same-file', minutes: 20 };
return { label: 'info same-file', minutes: 10 };
};
let savedMinutes = 0;
const groups = new Map<string, { count: number; minutesEach: number }>();
for (const c of eligible) {
const { label, minutes } = weightOf(c);
savedMinutes += minutes / Math.max(1, c.engineers?.length ?? 1);
const g = groups.get(label) ?? { count: 0, minutesEach: minutes };
g.count += 1;
groups.set(label, g);
}
const filesDeconflicted = new Set(eligible.map((c) => c.file)).size;
const totalFiles = changedFileCount;
return {
savedMinutes: Math.round(savedMinutes),
clashesCaught: eligible.length,
filesDeconflicted,
conflictFreeFiles: Math.max(0, totalFiles - filesDeconflicted),
totalFiles,
breakdown: [...groups.entries()].map(([label, g]) => ({
label,
count: g.count,
minutesEach: g.minutesEach,
})),
};
}
```
4. Build the intervention id set and attach to the return:
```ts
const interventionIds = new Set(interventions.map((i) => i.collisionId));
const roi = computeRoi(member, collisions, interventionIds, gitState?.changedFiles?.length ?? 0);
```
Add `roi` to the returned object at `:161-173`. Always return it (it self-zeroes
when there are no clashes); the frontend decides whether to show the band based
on `clashesCaught`/`savedMinutes`.
> Note `Collision.id` survives the `{ _id: 0 }` projection — `id` is a real field
> (`shared/src/collision.ts:8`), distinct from Mongo `_id`. Do not project it out.
## Edit 3 — frontend (`frontend/src/components/PodView.tsx`)
1. Add `MemberWorkHistoryRoi` to the `@podman/shared` type import (`:33-35`).
2. Insert `<RoiBand roi={history.roi} />` immediately inside the
`history && !loading && !error` block, BEFORE the `grid ... sm:grid-cols-3`
stat grid (`:1037`).
3. New presentational component (place near `HistoryStat`, `:1093`):
```tsx
function formatSaved(minutes: number): string {
if (minutes < 60) return `~${minutes}m`;
const h = Math.floor(minutes / 60);
const m = minutes % 60;
return m ? `~${h}h ${m}m` : `~${h}h`;
}
function RoiBand({ roi }: { roi?: MemberWorkHistoryRoi }) {
if (!roi || roi.clashesCaught === 0) return null; // no clashes → no band
const conflictFree = roi.totalFiles
? Math.round((roi.conflictFreeFiles / roi.totalFiles) * 100)
: 100;
return (
<section className="rounded-lg border bg-primary/5 p-4">
<div className="flex flex-wrap items-end justify-between gap-3">
<div>
<div className="flex items-center gap-1.5">
<p className="font-mono text-2xl font-semibold">{formatSaved(roi.savedMinutes)}</p>
<span className="text-sm text-muted-foreground">rework saved</span>
<RoiTooltip roi={roi} />
</div>
<p className="mt-0.5 text-xs text-muted-foreground">
estimated · clashes caught pre-commit
</p>
</div>
<div className="text-right">
<p className="font-mono text-lg font-medium">{roi.clashesCaught}</p>
<p className="text-xs text-muted-foreground">clashes caught early</p>
</div>
</div>
<div className="mt-3 h-2 overflow-hidden rounded-full bg-muted">
<div className="h-full rounded-full bg-primary" style={{ width: `${conflictFree}%` }} />
</div>
<p className="mt-1.5 text-[0.68rem] text-muted-foreground">
conflict-free: {roi.conflictFreeFiles} of {roi.totalFiles} files ·{' '}
{roi.filesDeconflicted} auto-deconflicted
</p>
</section>
);
}
```
4. Tooltip — reuse the existing Tooltip primitives already imported in this file
(`Tooltip`, `TooltipTrigger`, `TooltipContent` — see the History button at
`:982-988`). `RoiTooltip` renders an `ⓘ`/`InfoIcon` trigger; content lists
`breakdown` rows as `{count} × {minutesEach}m {label}` plus a footer line
`"split across engineers · est. only"`. Keep it a few lines.
## Verification
```bash
pnpm -r build # @podman/shared first, then backend + frontend typecheck
```
Manual / demo check (against a pod that has collisions, e.g. demo-pod):
1. Open the app, click a teammate who was in a clash → History.
2. Band shows at top: `~Xh Ym rework saved`, clashes-caught integer, conflict-free bar.
3. Hover `ⓘ` → breakdown rows match the heuristic table.
4. Click a teammate with **no** clashes → band absent, rest of dialog unchanged.
5. Existing Recent files + Timeline sections render unchanged below the band.
Quick data sanity (optional, on the box or via mongosh):
```js
db.collisions.find({ podId: "demo-pod" }).count() // > 0 for a real band
db.interventions.find({ podId: "demo-pod" }).count() // surfaced clashes
```
## Coordination / merge risk (team is concurrent)
- Touches two shared hot files: `shared/src/member-history.ts` and
`frontend/src/components/PodView.tsx`. Both edits are **additive** (one
optional field + one new component + one insert line). Low conflict risk, but
announce before pushing.
- `git pull --rebase origin main` immediately before push. Never force-push main.
- No changes to `/api/...` route signatures, no new deps, no Mongo writes.
## Deploy (after merge, on the box — see CLAUDE.md ops)
```bash
cd /root/podman && git pull && pnpm -r build
rm -rf /var/www/podman/* && cp -r frontend/dist/* /var/www/podman/
systemctl restart podman-platform-api podman-platform-agent
```
(Frontend-visible change + backend payload change → both the static build and
the API service must be refreshed.)
+2
View File
@@ -11,6 +11,8 @@
},
"dependencies": {
"@base-ui/react": "^1.6.0",
"@clerk/react": "^6.11.1",
"@clerk/ui": "^1.23.0",
"@fontsource-variable/geist": "^5.2.9",
"@podman/shared": "workspace:*",
"@shadcn/react": "^0.1.0",
+135 -110
View File
@@ -1,15 +1,18 @@
import { useEffect, useState } from 'react';
import {
Show,
SignUp,
SignInButton,
SignUpButton,
UserButton,
useAuth,
useUser,
} from '@clerk/react';
import type { Room } from 'livekit-client';
import {
AlertCircleIcon,
BrainCircuitIcon,
CircleDotIcon,
RadioTowerIcon,
RefreshCwIcon,
SparklesIcon,
UsersIcon,
WifiIcon,
ShieldCheckIcon,
} from 'lucide-react';
import type { Pod, PodInput } from '@podman/shared';
import { joinPod } from './lib/pod.js';
@@ -19,7 +22,6 @@ import { CreatePodForm } from './components/CreatePodForm.js';
import { PodView } from './components/PodView.js';
import { GraphView } from './components/GraphView.js';
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
Card,
@@ -40,7 +42,6 @@ import {
import { Skeleton } from '@/components/ui/skeleton';
const SESSION_KEY = 'podman.session';
const fmt = new Intl.NumberFormat('en', { notation: 'compact' });
function pathPodId(): string | null {
const [segment] = window.location.pathname.split('/').filter(Boolean);
@@ -62,13 +63,18 @@ function replacePath(path: string): void {
window.history.replaceState({}, '', path || '/');
}
function firstNameFrom(value: string | null | undefined): string {
return value?.trim().split(/\s+/).filter(Boolean)[0] ?? '';
}
export default function App() {
const { getToken, isLoaded, isSignedIn } = useAuth();
const { user } = useUser();
const [pods, setPods] = useState<Pod[]>([]);
const [loading, setLoading] = useState(true);
const [pending, setPending] = useState<Set<string>>(new Set());
const [error, setError] = useState<string | null>(null);
const [presence, setPresence] = useState<Record<string, string[]>>({});
const [memory, setMemory] = useState<api.MemoryStats | null>(null);
const [joinedPodId, setJoinedPodId] = useState<string | null>(null);
const [member, setMember] = useState('');
const [devMode, setDevMode] = useState(false);
@@ -77,18 +83,31 @@ export default function App() {
const [graphPodId, setGraphPodId] = useState<string | null>(null);
const joinedPod = joinedPodId ? (pods.find((p) => p.id === joinedPodId) ?? null) : null;
const userEmail = user?.primaryEmailAddress?.emailAddress;
const defaultMemberName =
user?.firstName?.trim() ||
firstNameFrom(user?.fullName) ||
firstNameFrom(userEmail?.split('@')[0]);
const currentUserProfile = {
displayName: defaultMemberName,
email: userEmail,
imageUrl: user?.imageUrl,
};
useEffect(() => {
api.setAuthTokenGetter(isSignedIn ? getToken : null);
return () => api.setAuthTokenGetter(null);
}, [getToken, isSignedIn]);
async function refresh() {
setLoading(true);
try {
const [nextPods, nextPresence, nextMemory] = await Promise.all([
const [nextPods, nextPresence] = await Promise.all([
api.listPods(),
api.getPresence().catch(() => presence),
api.getMemoryStats().catch(() => memory),
]);
setPods(nextPods);
setPresence(nextPresence);
setMemory(nextMemory);
setError(null);
} catch (e) {
setError((e as Error).message);
@@ -109,7 +128,13 @@ export default function App() {
const previousPath = window.location.pathname;
setPodPath(podId, replaceRoute);
try {
const result = await joinPod(podId, who, who);
const result = await joinPod(
podId,
who,
who,
isSignedIn ? getToken : undefined,
currentUserProfile,
);
setRoom(result.room);
setDevMode(result.mode === 'dev');
setMember(who);
@@ -122,21 +147,21 @@ export default function App() {
}
useEffect(() => {
if (!isSignedIn) {
setLoading(false);
return;
}
void refresh();
}, []);
}, [isSignedIn]);
useEffect(() => {
if (joinedPodId) return;
if (!isSignedIn || joinedPodId) return;
let alive = true;
const tick = async () => {
try {
const [p, m] = await Promise.all([
api.getPresence(),
api.getMemoryStats().catch(() => memory),
]);
const p = await api.getPresence();
if (alive) {
setPresence(p);
setMemory(m);
}
} catch {
/* presence is best-effort */
@@ -148,15 +173,13 @@ export default function App() {
alive = false;
window.clearInterval(id);
};
}, [joinedPodId]);
}, [isSignedIn, joinedPodId]);
useEffect(() => {
if (!isSignedIn) return;
const routedPodId = pathPodId();
const raw = sessionStorage.getItem(SESSION_KEY);
if (!raw) {
if (routedPodId) {
setError(`Enter your name to join ${routedPodId}.`);
}
return;
}
let saved: { podId: string; member: string };
@@ -177,10 +200,11 @@ export default function App() {
setRestoring(false);
}
})();
}, []);
}, [isSignedIn]);
useEffect(() => {
const onPopState = () => {
if (!isSignedIn) return;
const routedPodId = pathPodId();
if (!routedPodId) {
room?.disconnect();
@@ -201,7 +225,7 @@ export default function App() {
};
window.addEventListener('popstate', onPopState);
return () => window.removeEventListener('popstate', onPopState);
}, [joinedPodId, room]);
}, [isSignedIn, joinedPodId, room]);
async function run(key: string, fn: () => Promise<void>) {
startPending(key);
@@ -221,7 +245,7 @@ export default function App() {
startPending('new');
setError(null);
try {
const created = await api.createPod(input);
const created = await api.createPod(input, currentUserProfile);
setPods((cur) => [...cur, created]);
} catch (e) {
setError((e as Error).message);
@@ -240,7 +264,7 @@ export default function App() {
if (pathPodId() === id) setHomePath();
});
const handleAddMember = (id: string, name: string) =>
run(id, async () => upsert(await api.addMember(id, name)));
run(id, async () => upsert(await api.addMember(id, name, currentUserProfile)));
const handleRemoveMember = (id: string, name: string) =>
run(id, async () => upsert(await api.removeMember(id, name)));
@@ -256,12 +280,15 @@ export default function App() {
}
}
async function handleAddAndJoin(pod: Pod, name: string) {
async function handleAddAndJoin(pod: Pod) {
startPending(pod.id);
setError(null);
try {
upsert(await api.addMember(pod.id, name));
await connectToPod(pod.id, name);
if (!defaultMemberName) {
throw new Error('Sign in with Clerk before joining a pod.');
}
upsert(await api.addMember(pod.id, defaultMemberName, currentUserProfile));
await connectToPod(pod.id, defaultMemberName);
} catch (e) {
setError((e as Error).message);
} finally {
@@ -279,18 +306,29 @@ export default function App() {
}
const showReconnecting = restoring || (joinedPodId !== null && joinedPod === null);
const liveNames = Array.from(new Set(Object.values(presence).flat()));
const liveTotal = liveNames.length;
const totalMembers = pods.reduce((sum, p) => sum + p.members.length, 0);
const activeRooms = Object.values(presence).filter((names) => names.length > 0).length;
const podManOnline = liveNames.some((name) => name.toLowerCase() === 'podman');
const latestActivity = memory
? memory.observations + memory.collisions + memory.interventions + memory.outcomes
: 0;
if (!isLoaded) {
return (
<div className="grid min-h-screen place-items-center bg-background text-foreground">
<Skeleton className="h-12 w-64" />
</div>
);
}
if (!isSignedIn) {
return <AuthGate />;
}
if (joinedPod) {
return (
<PodView team={joinedPod} me={member} room={room} devMode={devMode} onLeave={handleLeave} />
<PodView
team={joinedPod}
me={member}
room={room}
devMode={devMode}
currentUserProfile={currentUserProfile}
onLeave={handleLeave}
/>
);
}
@@ -301,46 +339,35 @@ export default function App() {
return (
<div className="min-h-screen bg-background text-foreground">
<div className="mx-auto flex min-h-screen w-full max-w-[1440px] flex-col gap-6 px-4 py-4 sm:px-6 lg:px-8">
<header className="sticky top-0 z-10 -mx-4 flex flex-col gap-5 border-b bg-background/86 px-4 pb-5 pt-2 backdrop-blur-xl sm:-mx-6 sm:px-6 lg:-mx-8 lg:px-8">
<div className="flex flex-col gap-4 lg:flex-row lg:items-end lg:justify-between">
<header className="sticky top-0 z-10 -mx-4 border-b bg-background/86 px-4 pb-4 pt-2 backdrop-blur-xl sm:-mx-6 sm:px-6 lg:-mx-8 lg:px-8">
<div className="flex flex-col gap-4 sm:flex-row sm:items-center sm:justify-between">
<div className="flex min-w-0 items-center gap-3">
<div className="grid size-10 place-items-center rounded-lg bg-primary text-sm font-semibold text-primary-foreground shadow-sm">
PM
</div>
<div className="min-w-0">
<div className="flex flex-wrap items-center gap-2">
<h1 className="text-[1.95rem] font-semibold leading-none tracking-tight">
PodMan
</h1>
<Badge variant={podManOnline ? 'default' : 'secondary'}>
<CircleDotIcon data-icon="inline-start" />
{podManOnline ? 'online' : 'standby'}
</Badge>
</div>
<p className="text-sm text-muted-foreground">
Live engineering rooms, team memory, and intervention routing.
</p>
<h1 className="text-[1.95rem] font-semibold leading-none tracking-tight">PodMan</h1>
</div>
</div>
<div className="flex items-center gap-2">
<Badge variant="outline" className="h-8 rounded-lg px-3">
<ShieldCheckIcon data-icon="inline-start" />
Privacy-limited
</Badge>
<Show when="signed-out">
<SignInButton mode="modal">
<Button variant="outline">Sign in</Button>
</SignInButton>
<SignUpButton mode="modal">
<Button>Sign up</Button>
</SignUpButton>
</Show>
<Show when="signed-in">
<UserButton />
</Show>
<Button variant="outline" onClick={() => void refresh()} disabled={loading}>
<RefreshCwIcon data-icon="inline-start" />
Refresh
</Button>
</div>
</div>
<div className="grid gap-2 sm:grid-cols-2 lg:grid-cols-4">
<StatPill icon={WifiIcon} label="Live" value={fmt.format(liveTotal)} />
<StatPill icon={RadioTowerIcon} label="Rooms" value={fmt.format(activeRooms)} />
<StatPill icon={UsersIcon} label="Roster" value={fmt.format(totalMembers)} />
<StatPill icon={BrainCircuitIcon} label="Memory" value={fmt.format(latestActivity)} />
</div>
</header>
{error && (
@@ -374,9 +401,6 @@ export default function App() {
<p className="text-xs font-medium uppercase text-muted-foreground">Workspaces</p>
<h2 className="text-xl font-semibold tracking-tight">Active pods</h2>
</div>
<p className="max-w-xl text-sm leading-6 text-muted-foreground">
Join the room that matches your current workstream.
</p>
</div>
{loading ? (
@@ -392,6 +416,7 @@ export default function App() {
pod={pod}
busy={pending.has(pod.id)}
presence={presence[pod.id] ?? []}
currentUserProfile={currentUserProfile}
onJoin={handleJoin}
onAddAndJoin={handleAddAndJoin}
onAddMember={handleAddMember}
@@ -412,29 +437,23 @@ export default function App() {
<EmptyDescription>Create the first room for this team.</EmptyDescription>
</EmptyHeader>
<EmptyContent>
<CreatePodForm busy={pending.has('new')} onCreate={handleCreate} compact />
<CreatePodForm
busy={pending.has('new')}
defaultMemberName={defaultMemberName}
onCreate={handleCreate}
compact
/>
</EmptyContent>
</Empty>
)}
</section>
<aside className="flex flex-col gap-4">
<CreatePodForm busy={pending.has('new')} onCreate={handleCreate} />
<Card>
<CardHeader>
<CardTitle>Operating brief</CardTitle>
<CardDescription>Current coordination signals.</CardDescription>
</CardHeader>
<CardContent className="space-y-3">
<BriefLine label="Notification default" value="Card" />
<BriefLine label="Escalation" value="Hermes voice only when urgent" />
<BriefLine label="Memory events" value={fmt.format(latestActivity)} />
<BriefLine
label="Live people"
value={liveNames.length ? liveNames.join(', ') : 'None'}
/>
</CardContent>
</Card>
<CreatePodForm
busy={pending.has('new')}
defaultMemberName={defaultMemberName}
onCreate={handleCreate}
/>
</aside>
</main>
)}
@@ -443,33 +462,39 @@ export default function App() {
);
}
function StatPill({
icon: Icon,
label,
value,
}: {
icon: React.ComponentType<{ className?: string }>;
label: string;
value: string;
}) {
function AuthGate() {
return (
<div className="flex min-h-16 items-center gap-3 rounded-lg border bg-card/90 px-3 py-2 shadow-sm">
<div className="grid size-8 place-items-center rounded-md bg-muted">
<Icon className="size-4 text-muted-foreground" />
</div>
<div className="min-w-0">
<p className="text-xs font-medium uppercase text-muted-foreground">{label}</p>
<p className="text-base font-medium">{value}</p>
</div>
</div>
);
}
<div className="min-h-screen bg-background text-foreground">
<div className="mx-auto flex min-h-screen w-full max-w-[1120px] flex-col px-4 py-4 sm:px-6 lg:px-8">
<header className="flex items-center justify-between border-b pb-4 pt-2">
<div className="flex min-w-0 items-center gap-3">
<div className="grid size-10 place-items-center rounded-lg bg-primary text-sm font-semibold text-primary-foreground shadow-sm">
PM
</div>
<h1 className="text-[1.95rem] font-semibold leading-none tracking-tight">PodMan</h1>
</div>
<SignInButton mode="modal">
<Button variant="outline">Sign in</Button>
</SignInButton>
</header>
function BriefLine({ label, value }: { label: string; value: string }) {
return (
<div className="flex items-start justify-between gap-4 rounded-md bg-muted/45 px-3 py-2">
<span className="text-sm text-muted-foreground">{label}</span>
<span className="max-w-44 text-right text-sm font-medium">{value}</span>
<main className="grid flex-1 items-center gap-8 py-8 lg:grid-cols-[minmax(0,1fr)_420px]">
<section className="max-w-xl">
<p className="text-xs font-medium uppercase text-muted-foreground">Team memory</p>
<h2 className="mt-2 text-3xl font-semibold tracking-tight">
Create your account to enter PodMan
</h2>
<p className="mt-3 text-base text-muted-foreground">
PodMan saves your context across pods so agents can learn from your work in every
room you join.
</p>
</section>
<div className="flex justify-center lg:justify-end">
<SignUp routing="hash" />
</div>
</main>
</div>
</div>
);
}
+4 -2
View File
@@ -17,17 +17,19 @@ import { cn } from '@/lib/utils';
export function CreatePodForm({
busy,
onCreate,
defaultMemberName = '',
compact = false,
}: {
busy: boolean;
onCreate: (input: PodInput) => Promise<void>;
defaultMemberName?: string;
compact?: boolean;
}) {
const [open, setOpen] = useState(compact);
const [name, setName] = useState('');
const [repo, setRepo] = useState('karti-ai/podman');
const [description, setDescription] = useState('');
const [firstMember, setFirstMember] = useState('');
const [firstMember, setFirstMember] = useState(defaultMemberName);
async function submit() {
if (!name.trim()) return;
@@ -40,7 +42,7 @@ export function CreatePodForm({
});
setName('');
setDescription('');
setFirstMember('');
setFirstMember(defaultMemberName);
if (!compact) setOpen(false);
} catch {
/* parent owns the visible error */
+34 -25
View File
@@ -7,7 +7,13 @@ import {
VideoIcon,
} from 'lucide-react';
import type { Pod, PodInput } from '@podman/shared';
import { Avatar, AvatarBadge, AvatarFallback, AvatarGroup } from '@/components/ui/avatar';
import {
Avatar,
AvatarBadge,
AvatarFallback,
AvatarGroup,
AvatarImage,
} from '@/components/ui/avatar';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
@@ -41,6 +47,7 @@ export function PodCard({
pod,
busy,
presence,
currentUserProfile,
onJoin: _onJoin,
onAddAndJoin,
onAddMember: _onAddMember,
@@ -52,15 +59,19 @@ export function PodCard({
pod: Pod;
busy: boolean;
presence: string[];
currentUserProfile?: {
displayName: string;
email?: string;
imageUrl?: string;
};
onJoin: (pod: Pod, member: string) => void;
onAddAndJoin: (pod: Pod, name: string) => void;
onAddAndJoin: (pod: Pod) => void;
onAddMember: (id: string, name: string) => void;
onRemoveMember: (id: string, name: string) => void;
onUpdate: (id: string, patch: PodInput) => void;
onDelete: (id: string) => void;
onOpenGraph: (id: string) => void;
}) {
const [newMember, setNewMember] = useState('');
const [editing, setEditing] = useState(false);
const [draft, setDraft] = useState<PodInput>({
name: pod.name,
@@ -69,6 +80,13 @@ export function PodCard({
});
const inRoom = (name: string) => presence.some((p) => p.toLowerCase() === name.toLowerCase());
const profileForMember = (name: string) =>
pod.memberProfiles?.[name] ??
([currentUserProfile?.displayName, currentUserProfile?.email]
.filter(Boolean)
.some((value) => value?.toLowerCase() === name.toLowerCase())
? currentUserProfile
: undefined);
const active = presence.length > 0;
function saveEdit() {
@@ -80,13 +98,8 @@ export function PodCard({
setEditing(false);
}
// One action: enter your name, hit Join → added to the roster (deduped
// server-side) and connected to the room in a single step.
function join() {
const name = newMember.trim();
if (!name) return;
onAddAndJoin(pod, name);
setNewMember('');
onAddAndJoin(pod);
}
return (
@@ -143,12 +156,16 @@ export function PodCard({
<div className="flex items-center justify-between gap-4">
<AvatarGroup>
{pod.members.slice(0, 4).map((member) => (
<Avatar key={member} title={member}>
<AvatarFallback>{initials(member)}</AvatarFallback>
{inRoom(member) && <AvatarBadge />}
</Avatar>
))}
{pod.members.slice(0, 4).map((member) => {
const profile = profileForMember(member);
return (
<Avatar key={member} title={profile?.email ?? member}>
{profile?.imageUrl && <AvatarImage src={profile.imageUrl} alt={member} />}
<AvatarFallback>{initials(member)}</AvatarFallback>
{inRoom(member) && <AvatarBadge />}
</Avatar>
);
})}
{pod.members.length > 4 && <span className="text-sm text-muted-foreground">+</span>}
</AvatarGroup>
<div className="flex items-center gap-1.5 text-sm text-muted-foreground">
@@ -157,16 +174,8 @@ export function PodCard({
</div>
</div>
<div className="flex gap-2">
<Input
placeholder="Your name"
value={newMember}
onChange={(e) => setNewMember(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') join();
}}
/>
<Button className="min-w-24" onClick={join} disabled={busy || !newMember.trim()}>
<div className="flex justify-end">
<Button className="min-w-24" onClick={join} disabled={busy}>
<VideoIcon data-icon="inline-start" />
Join
</Button>
+249 -108
View File
@@ -9,10 +9,12 @@ import {
EyeIcon,
GitBranchIcon,
ExternalLinkIcon,
InfoIcon,
MessageSquareIcon,
MicIcon,
MicOffIcon,
MonitorUpIcon,
Music2Icon,
PhoneCallIcon,
PhoneOffIcon,
PanelLeftIcon,
@@ -25,6 +27,7 @@ import {
VolumeXIcon,
WorkflowIcon,
XIcon,
type LucideIcon,
} from 'lucide-react';
import type { Room, RemoteTrack, RemoteTrackPublication } from 'livekit-client';
import type {
@@ -33,6 +36,7 @@ import type {
MemberWorkHistory,
MemberWorkHistoryEvent,
MemberWorkHistoryFile,
MemberWorkHistoryRoi,
Pod,
PodActivityEvent,
PodActivityKind,
@@ -51,9 +55,8 @@ import {
} from '../lib/api.js';
import { useInterventions, primeSpeech } from '../livekit/useInterventions.js';
import { usePodActivity } from '../hooks/use-pod-activity.js';
import LiveWaveform from '@/components/ruixen/live-waveform';
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
import { Avatar, AvatarBadge, AvatarFallback } from '@/components/ui/avatar';
import { Avatar, AvatarBadge, AvatarFallback, AvatarImage } from '@/components/ui/avatar';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
@@ -97,24 +100,46 @@ const STREAM_RAIL_WIDTH = '4rem';
interface PInfo {
id: string;
name: string;
email?: string;
imageUrl?: string;
isLocal: boolean;
speaking: boolean;
}
function profileFromMetadata(metadata?: string): { email?: string; imageUrl?: string } {
try {
const parsed = JSON.parse(metadata || '{}') as { email?: unknown; imageUrl?: unknown };
return {
email: typeof parsed.email === 'string' ? parsed.email : undefined,
imageUrl: typeof parsed.imageUrl === 'string' ? parsed.imageUrl : undefined,
};
} catch {
return {};
}
}
function snapshot(room: Room, fallbackName: string): PInfo[] {
const lp = room.localParticipant;
const localProfile = profileFromMetadata(lp.metadata);
const local: PInfo = {
id: lp.identity,
name: lp.name || fallbackName,
email: localProfile.email,
imageUrl: localProfile.imageUrl,
isLocal: true,
speaking: lp.isSpeaking,
};
const remotes = Array.from(room.remoteParticipants.values()).map((p) => ({
id: p.identity,
name: p.name || p.identity,
isLocal: false,
speaking: p.isSpeaking,
}));
const remotes = Array.from(room.remoteParticipants.values()).map((p) => {
const profile = profileFromMetadata(p.metadata);
return {
id: p.identity,
name: p.name || p.identity,
email: profile.email,
imageUrl: profile.imageUrl,
isLocal: false,
speaking: p.isSpeaking,
};
});
return [local, ...remotes];
}
@@ -133,11 +158,17 @@ export function PodView({
room,
devMode,
onLeave,
currentUserProfile,
}: {
team: Pod;
me: string;
room: Room | null;
devMode: boolean;
currentUserProfile?: {
displayName: string;
email?: string;
imageUrl?: string;
};
onLeave: () => void;
}) {
const [participants, setParticipants] = useState<PInfo[]>([]);
@@ -145,7 +176,7 @@ export function PodView({
const [testingVoice, setTestingVoice] = useState(false);
const [note, setNote] = useState<string | null>(null);
const [audioBlocked, setAudioBlocked] = useState(false);
const [remoteAudioTracks, setRemoteAudioTracks] = useState(0);
const [, setRemoteAudioTracks] = useState(0);
const [micOn, setMicOn] = useState(false);
const [historyMember, setHistoryMember] = useState<string | null>(null);
const [history, setHistory] = useState<MemberWorkHistory | null>(null);
@@ -238,6 +269,7 @@ export function PodView({
room
.on(RoomEvent.ParticipantConnected, refresh)
.on(RoomEvent.ParticipantDisconnected, refresh)
.on(RoomEvent.ParticipantMetadataChanged, refresh)
.on(RoomEvent.ActiveSpeakersChanged, refresh)
.on(RoomEvent.TrackSubscribed, onAudio)
.on(RoomEvent.TrackUnsubscribed, onAudioGone)
@@ -249,6 +281,7 @@ export function PodView({
room
.off(RoomEvent.ParticipantConnected, refresh)
.off(RoomEvent.ParticipantDisconnected, refresh)
.off(RoomEvent.ParticipantMetadataChanged, refresh)
.off(RoomEvent.ActiveSpeakersChanged, refresh)
.off(RoomEvent.TrackSubscribed, onAudio)
.off(RoomEvent.TrackUnsubscribed, onAudioGone)
@@ -567,8 +600,6 @@ export function PodView({
}
}
const podmanPresent = participants.some((p) => p.name.toLowerCase() === 'podman');
return (
<SidebarProvider
open={leftStreamOpen}
@@ -586,7 +617,7 @@ export function PodView({
title="My stream"
collapsedLabel="Mine"
testId="my-stream-sidebar"
description={`${me}'s live screen, git, intervention, and conflict log.`}
description={`${me}'s activity feed.`}
events={activity.mine}
connected={activity.connected}
open={leftStreamOpen}
@@ -638,54 +669,62 @@ export function PodView({
</div>
</div>
<div className="grid grid-cols-2 gap-2 sm:flex sm:flex-wrap sm:items-center">
<Button
variant={audioBlocked ? 'default' : 'outline'}
onClick={() => void enableSound()}
disabled={!room}
>
{audioBlocked ? (
<VolumeXIcon data-icon="inline-start" />
) : (
<Volume2Icon data-icon="inline-start" />
)}
{audioBlocked ? 'Enable audio' : 'Audio on'}
</Button>
<Button
variant={micOn ? 'default' : 'outline'}
onClick={() => void toggleMic()}
disabled={!room}
>
{micOn ? (
<MicIcon data-icon="inline-start" />
) : (
<MicOffIcon data-icon="inline-start" />
)}
{micOn ? 'Mic on' : 'Enable mic'}
</Button>
<Button variant="outline" onClick={onToggleBeat} disabled={!room}>
<Volume2Icon data-icon="inline-start" />
{beat.on ? (beat.mine ? 'Stop music' : `Stop (${beat.by})`) : 'Background Music'}
</Button>
<Button
variant="outline"
onClick={() => void playPodManVoiceTest()}
disabled={!room || testingVoice}
>
<RadioTowerIcon data-icon="inline-start" />
{testingVoice ? 'Sending voice' : 'Test PodMan voice'}
</Button>
<Button onClick={toggleScreen} disabled={!room}>
<MonitorUpIcon data-icon="inline-start" />
{sharing ? 'Stop sharing' : 'Share screen'}
</Button>
</div>
</div>
<div className="flex flex-wrap items-center gap-2">
<div className="flex items-center gap-0.5 rounded-xl border bg-card/80 p-1 shadow-sm backdrop-blur-sm">
<PodControlButton
label={audioBlocked ? 'Enable audio' : 'Audio on'}
icon={audioBlocked ? VolumeXIcon : Volume2Icon}
onClick={() => void enableSound()}
disabled={!room}
active={!audioBlocked}
attention={audioBlocked}
/>
<PodControlButton
label={micOn ? 'Mic on' : 'Enable mic'}
icon={micOn ? MicIcon : MicOffIcon}
onClick={() => void toggleMic()}
disabled={!room}
active={micOn}
/>
<PodControlButton
label={
beat.on
? beat.mine
? 'Stop background music'
: `Stop music (${beat.by})`
: 'Background music'
}
icon={Music2Icon}
onClick={onToggleBeat}
disabled={!room}
active={beat.on}
/>
<PodControlButton
label={testingVoice ? 'Sending voice…' : 'Test PodMan voice'}
icon={RadioTowerIcon}
onClick={() => void playPodManVoiceTest()}
disabled={!room || testingVoice}
active={testingVoice}
busy={testingVoice}
/>
</div>
<div className="grid gap-3 sm:grid-cols-3">
<Metric label="Participants" value={participants.length || 1} />
<Metric label="Screen" value={sharing ? 'sharing' : 'idle'} />
<Metric label="PodMan" value={podmanPresent ? 'online' : 'waiting'} />
<Tooltip>
<TooltipTrigger asChild>
<Button
onClick={toggleScreen}
disabled={!room}
variant={sharing ? 'destructive' : 'default'}
>
<MonitorUpIcon data-icon="inline-start" />
{sharing ? 'Stop sharing' : 'Share screen'}
</Button>
</TooltipTrigger>
<TooltipContent>
{sharing ? 'Stop sharing your screen' : 'Share your screen with PodMan'}
</TooltipContent>
</Tooltip>
</div>
</div>
</section>
@@ -729,6 +768,8 @@ export function PodView({
<Participant
key={p.id}
participant={p}
rosterProfile={team.memberProfiles?.[p.name]}
currentUserProfile={currentUserProfile}
onOpenHistory={setHistoryMember}
/>
))}
@@ -922,41 +963,6 @@ export function PodView({
</CardContent>
</Card>
<Card>
<CardHeader>
<CardTitle>Status</CardTitle>
<CardDescription>Connection, media, and agent presence.</CardDescription>
</CardHeader>
<CardContent>
<LiveWaveform
processing={!!room || beat.on}
active={false}
height={32}
barWidth={2}
barGap={3}
className="mb-4 px-3 py-2 text-muted-foreground"
/>
<div className="flex flex-col gap-3">
<StatusLine label="LiveKit" value={room ? 'connected' : 'offline'} />
<StatusLine label="Screen" value={sharing ? 'published' : 'not shared'} />
<StatusLine
label="Audio"
value={
audioBlocked
? 'blocked'
: remoteAudioTracks > 0
? `${remoteAudioTracks} remote track${remoteAudioTracks === 1 ? '' : 's'}`
: beat.on
? beat.mine
? 'publishing'
: `${beat.by} playing`
: 'ready'
}
/>
<StatusLine label="Agent" value={podmanPresent ? 'watching' : 'waiting'} />
</div>
</CardContent>
</Card>
</aside>
</main>
<div
@@ -980,7 +986,7 @@ export function PodView({
title="Team stream"
collapsedLabel="Team"
testId="team-stream-sidebar"
description="Everyone else in this pod, merged into one realtime feed."
description="Team activity feed."
events={activity.team}
connected={activity.connected}
open={rightStreamOpen}
@@ -993,24 +999,30 @@ export function PodView({
);
}
function Metric({ label, value }: { label: string; value: number | string }) {
return (
<Card size="sm">
<CardContent>
<p className="text-xs text-muted-foreground">{label}</p>
<p className="mt-1 text-lg font-medium">{value}</p>
</CardContent>
</Card>
);
}
function Participant({
participant,
rosterProfile,
currentUserProfile,
onOpenHistory,
}: {
participant: PInfo;
rosterProfile?: {
displayName: string;
email?: string;
imageUrl?: string;
};
currentUserProfile?: {
displayName: string;
email?: string;
imageUrl?: string;
};
onOpenHistory: (member: string) => void;
}) {
const localProfile = participant.isLocal ? currentUserProfile : undefined;
const profile = {
email: participant.email ?? localProfile?.email ?? rosterProfile?.email,
imageUrl: participant.imageUrl ?? localProfile?.imageUrl ?? rosterProfile?.imageUrl,
};
return (
<div
className={cn(
@@ -1020,12 +1032,15 @@ function Participant({
>
<div className="flex min-w-0 items-center gap-3">
<Avatar>
{profile.imageUrl && <AvatarImage src={profile.imageUrl} alt={participant.name} />}
<AvatarFallback>{initials(participant.name)}</AvatarFallback>
{participant.speaking && <AvatarBadge />}
</Avatar>
<div className="min-w-0">
<p className="truncate text-sm font-medium">{participant.name}</p>
<p className="text-xs text-muted-foreground">{participant.isLocal ? 'you' : 'remote'}</p>
<p className="truncate text-xs text-muted-foreground">
{profile.email ?? (participant.isLocal ? 'you' : 'remote')}
</p>
</div>
</div>
<div className="flex shrink-0 items-center gap-2">
@@ -1089,6 +1104,7 @@ function WorkHistoryDialog({
{history && !loading && !error && (
<div className="flex flex-col gap-5">
<RoiBand roi={history.roi} />
<div className="grid gap-2 sm:grid-cols-3">
<HistoryStat label="Files" value={history.totals.files} />
<HistoryStat label="Screen logs" value={history.totals.observations} />
@@ -1145,6 +1161,88 @@ function WorkHistoryDialog({
);
}
function formatSaved(minutes: number): string {
if (minutes < 60) return `~${minutes}m`;
const h = Math.floor(minutes / 60);
const m = minutes % 60;
return m ? `~${h}h ${m}m` : `~${h}h`;
}
function RoiTooltip({ roi }: { roi: MemberWorkHistoryRoi }) {
return (
<Tooltip>
<TooltipTrigger asChild>
<button
type="button"
aria-label="How rework saved is estimated"
className="text-muted-foreground transition-colors hover:text-foreground"
>
<InfoIcon className="size-3.5" />
</button>
</TooltipTrigger>
<TooltipContent className="max-w-64">
<div className="flex flex-col gap-1">
<p className="font-medium">Estimated rework saved</p>
{roi.breakdown.length ? (
roi.breakdown.map((row) => (
<p key={row.label} className="font-mono text-[0.7rem]">
{row.count} × {row.minutesEach}m · {row.label}
</p>
))
) : (
<p className="text-[0.7rem]">No eligible clashes.</p>
)}
<p className="text-[0.68rem] text-muted-foreground">
credit split across engineers · est. only
</p>
</div>
</TooltipContent>
</Tooltip>
);
}
function RoiBand({ roi }: { roi?: MemberWorkHistoryRoi }) {
if (!roi || roi.clashesCaught === 0) return null;
const conflictFree = roi.totalFiles
? Math.round((roi.conflictFreeFiles / roi.totalFiles) * 100)
: 100;
return (
<section className="rounded-lg border bg-primary/5 p-4">
<div className="flex flex-wrap items-end justify-between gap-3">
<div>
<div className="flex items-center gap-1.5">
<p className="font-mono text-2xl font-semibold">{formatSaved(roi.savedMinutes)}</p>
<span className="text-sm text-muted-foreground">rework saved</span>
<RoiTooltip roi={roi} />
</div>
<p className="mt-0.5 text-xs text-muted-foreground">
estimated · clashes caught pre-commit
</p>
</div>
<div className="text-right">
<p className="font-mono text-lg font-medium">{roi.clashesCaught}</p>
<p className="text-xs text-muted-foreground">clashes caught early</p>
</div>
</div>
{roi.totalFiles > 0 && (
<>
<div className="mt-3 h-2 overflow-hidden rounded-full bg-muted">
<div
className="h-full rounded-full bg-primary"
style={{ width: `${conflictFree}%` }}
aria-label={`${roi.conflictFreeFiles} of ${roi.totalFiles} files conflict-free`}
/>
</div>
<p className="mt-1.5 text-[0.68rem] text-muted-foreground">
conflict-free: {roi.conflictFreeFiles} of {roi.totalFiles} files ·{' '}
{roi.filesDeconflicted} auto-deconflicted
</p>
</>
)}
</section>
);
}
function HistoryStat({ label, value }: { label: string; value: number }) {
return (
<div className="rounded-lg border bg-background px-3 py-2">
@@ -1548,7 +1646,7 @@ const ACTIVITY_CATEGORIES: {
{
id: 'signal',
label: 'Signals',
hint: 'Screen logs and local git activity routed to this stream.',
hint: 'Recent screen and git activity.',
icon: RadioTowerIcon,
},
{
@@ -1663,3 +1761,46 @@ function initials(name: string): string {
.slice(0, 2)
.toUpperCase();
}
/**
* Compact, icon-only control for the pod toolbar. The label is hidden by
* default and surfaced on hover/focus via tooltip, so the row stays tight and
* responsive. `active` fills the button (engaged state), `attention` promotes
* it to the primary colour (needs a user action, e.g. audio is blocked).
*/
function PodControlButton({
label,
icon: Icon,
onClick,
disabled,
active = false,
attention = false,
busy = false,
}: {
label: string;
icon: LucideIcon;
onClick: () => void;
disabled?: boolean;
active?: boolean;
attention?: boolean;
busy?: boolean;
}) {
return (
<Tooltip>
<TooltipTrigger asChild>
<Button
type="button"
size="icon"
variant={attention ? 'default' : active ? 'secondary' : 'ghost'}
aria-pressed={active}
aria-label={label}
onClick={onClick}
disabled={disabled}
>
<Icon className={cn(busy && 'animate-pulse')} />
</Button>
</TooltipTrigger>
<TooltipContent>{label}</TooltipContent>
</Tooltip>
);
}
@@ -28,6 +28,7 @@ export const ACTIVITY_TAG: Record<PodGraphActivityKind, { color: string; label:
outcome: { color: GREEN, label: 'OUTCOME' },
learned: { color: VIOLET, label: 'LEARNED' },
agent: { color: BLUE, label: 'AGENT' },
suppressed: { color: VIOLET, label: 'SUPPRESSED' },
};
export const KIND_COLOR: Record<PodGraphNodeKind, string> = {
+66 -30
View File
@@ -14,11 +14,34 @@ const BACKEND_URL =
? 'http://localhost:8787'
: '');
type AuthTokenGetter = () => Promise<string | null>;
let authTokenGetter: AuthTokenGetter | null = null;
export function setAuthTokenGetter(getter: AuthTokenGetter | null): void {
authTokenGetter = getter;
}
async function requestHeaders(init?: HeadersInit): Promise<Headers> {
const next = new Headers(init);
const token = await authTokenGetter?.();
if (token) next.set('authorization', `Bearer ${token}`);
return next;
}
async function apiFetch(input: string, init: RequestInit = {}): Promise<Response> {
return fetch(input, {
...init,
headers: await requestHeaders(init.headers),
});
}
export interface MemoryStats {
observations: number;
collisions: number;
interventions: number;
outcomes: number;
userPodContext?: number;
}
export interface LiveConversationSession {
@@ -34,6 +57,12 @@ export interface LiveConversationSession {
endedAt?: string;
}
export interface UserProfilePayload {
displayName?: string;
email?: string;
imageUrl?: string;
}
async function json<T>(res: Response): Promise<T> {
if (!res.ok) {
const body = (await res.json().catch(() => ({}))) as { error?: string };
@@ -42,16 +71,19 @@ async function json<T>(res: Response): Promise<T> {
return res.json() as Promise<T>;
}
const JSON_HEADERS = { 'content-type': 'application/json' } as const;
/** Mint a LiveKit token from the backend. */
export async function fetchToken(params: {
room: string;
identity: string;
name: string;
githubLogin?: string;
profile?: UserProfilePayload;
}): Promise<{ token: string; url: string }> {
const res = await fetch(`${BACKEND_URL}/api/token`, {
const res = await apiFetch(`${BACKEND_URL}/api/token`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify(params),
});
return json(res);
@@ -59,9 +91,9 @@ export async function fetchToken(params: {
/** Record an intervention outcome for the policy learning loop. */
export async function postOutcome(outcome: InterventionOutcome): Promise<void> {
const res = await fetch(`${BACKEND_URL}/api/outcome`, {
const res = await apiFetch(`${BACKEND_URL}/api/outcome`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify(outcome),
});
if (!res.ok) throw new Error(`outcome post failed: ${res.status}`);
@@ -73,9 +105,9 @@ export async function createSyncPr(input: {
summary?: string;
}): Promise<{ url: string; number: number }> {
return json(
await fetch(`${BACKEND_URL}/api/sync-pr`, {
await apiFetch(`${BACKEND_URL}/api/sync-pr`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify(input),
}),
);
@@ -84,21 +116,21 @@ export async function createSyncPr(input: {
// --- Pods CRUD ---
export async function listPods(): Promise<Pod[]> {
return json(await fetch(`${BACKEND_URL}/api/pods`));
return json(await apiFetch(`${BACKEND_URL}/api/pods`));
}
/** Display names currently connected per pod id (= LiveKit room name). */
export async function getPresence(): Promise<Record<string, string[]>> {
return json(await fetch(`${BACKEND_URL}/api/presence`));
return json(await apiFetch(`${BACKEND_URL}/api/presence`));
}
export async function getMemoryStats(): Promise<MemoryStats> {
return json(await fetch(`${BACKEND_URL}/api/memory/stats`));
return json(await apiFetch(`${BACKEND_URL}/api/memory/stats`));
}
export async function getPodActivity(id: string, limit = 80): Promise<PodActivityEvent[]> {
return json(
await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/activity?limit=${limit}`),
await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/activity?limit=${limit}`),
);
}
@@ -116,7 +148,7 @@ export async function getMemberWorkHistory(
member: string,
): Promise<MemberWorkHistory> {
return json(
await fetch(
await apiFetch(
`${BACKEND_URL}/api/pods/${encodeURIComponent(podId)}/members/${encodeURIComponent(
member,
)}/history?hours=24&limit=80`,
@@ -124,47 +156,51 @@ export async function getMemberWorkHistory(
);
}
export async function createPod(input: PodInput): Promise<Pod> {
export async function createPod(input: PodInput, profile?: UserProfilePayload): Promise<Pod> {
return json(
await fetch(`${BACKEND_URL}/api/pods`, {
await apiFetch(`${BACKEND_URL}/api/pods`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(input),
headers: JSON_HEADERS,
body: JSON.stringify({ ...input, profile }),
}),
);
}
export async function updatePod(id: string, patch: PodInput): Promise<Pod> {
return json(
await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}`, {
await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}`, {
method: 'PATCH',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify(patch),
}),
);
}
export async function deletePod(id: string): Promise<void> {
const res = await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}`, {
const res = await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}`, {
method: 'DELETE',
});
if (!res.ok) throw new Error(`delete pod failed: ${res.status}`);
}
export async function addMember(id: string, name: string): Promise<Pod> {
export async function addMember(
id: string,
name: string,
profile?: UserProfilePayload,
): Promise<Pod> {
return json(
await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/members`, {
await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/members`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ name }),
headers: JSON_HEADERS,
body: JSON.stringify({ name, profile }),
}),
);
}
export async function testPodVoice(id: string): Promise<void> {
const res = await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/voice-test`, {
const res = await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/voice-test`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify({
message: 'PodMan voice test. Gemini TTS is playing through LiveKit.',
}),
@@ -177,16 +213,16 @@ export async function startLiveConversation(
input: { identity: string; displayName?: string },
): Promise<LiveConversationSession> {
return json(
await fetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(podId)}/live-conversation/start`, {
await apiFetch(`${BACKEND_URL}/api/pods/${encodeURIComponent(podId)}/live-conversation/start`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
headers: JSON_HEADERS,
body: JSON.stringify(input),
}),
);
}
export async function stopLiveConversation(podId: string, sessionId: string): Promise<void> {
const res = await fetch(
const res = await apiFetch(
`${BACKEND_URL}/api/pods/${encodeURIComponent(
podId,
)}/live-conversation/${encodeURIComponent(sessionId)}/stop`,
@@ -200,7 +236,7 @@ export async function getLiveConversationHermesJob(
sessionId: string,
): Promise<{ job: HermesJob | null; events: HermesJobEvent[] }> {
return json(
await fetch(
await apiFetch(
`${BACKEND_URL}/api/pods/${encodeURIComponent(
podId,
)}/live-conversation/${encodeURIComponent(sessionId)}/hermes-job`,
@@ -213,7 +249,7 @@ export async function abortLiveConversationHermesJob(
sessionId: string,
): Promise<{ job: HermesJob | null }> {
return json(
await fetch(
await apiFetch(
`${BACKEND_URL}/api/pods/${encodeURIComponent(
podId,
)}/live-conversation/${encodeURIComponent(sessionId)}/hermes-job/abort`,
@@ -224,7 +260,7 @@ export async function abortLiveConversationHermesJob(
export async function removeMember(id: string, name: string): Promise<Pod> {
return json(
await fetch(
await apiFetch(
`${BACKEND_URL}/api/pods/${encodeURIComponent(id)}/members/${encodeURIComponent(name)}`,
{ method: 'DELETE' },
),
+16 -4
View File
@@ -11,11 +11,17 @@ export async function fetchPodToken(
podId: string,
identity: string,
name: string,
getToken?: () => Promise<string | null>,
profile?: { displayName?: string; email?: string; imageUrl?: string },
): Promise<{ token: string; url: string }> {
const clerkToken = await getToken?.();
const res = await fetch(`${BACKEND_URL}/api/token`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ room: podId, identity, name }),
headers: {
'content-type': 'application/json',
...(clerkToken ? { authorization: `Bearer ${clerkToken}` } : {}),
},
body: JSON.stringify({ room: podId, identity, name, profile }),
});
if (!res.ok) throw new Error(`token request failed: ${res.status}`);
return res.json();
@@ -33,8 +39,14 @@ export type JoinResult = { mode: 'live'; room: Room } | { mode: 'dev'; room: nul
* connected — screen sharing is a separate, deliberate action (see PodView) so
* a denied/slow screen prompt never blocks or fails the join.
*/
export async function joinPod(podId: string, identity: string, name: string): Promise<JoinResult> {
const { token, url } = await fetchPodToken(podId, identity, name);
export async function joinPod(
podId: string,
identity: string,
name: string,
getToken?: () => Promise<string | null>,
profile?: { displayName?: string; email?: string; imageUrl?: string },
): Promise<JoinResult> {
const { token, url } = await fetchPodToken(podId, identity, name, getToken, profile);
if (!isLiveKitConfigured(url)) {
console.warn('[podman] LiveKit not configured — dev mock join');
+7
View File
@@ -39,6 +39,11 @@ export function useInterventions(room: Room | null) {
if (msg.type === 'COLLISION') {
setActive(msg.intervention);
setActionUrl(null);
// Clear the prior intervention's cue/message so a new card never shows a
// stale voice line. The fresh ones arrive right after on the same
// reliable channel (ordered: COLLISION -> HERMES_MESSAGE -> VOICE_CUE).
setVoiceCue(null);
setHermes(null);
}
if (msg.type === 'HERMES_MESSAGE') setHermes(msg.message);
if (msg.type === 'VOICE_CUE') {
@@ -79,6 +84,8 @@ export function useInterventions(room: Room | null) {
{ reliable: true, topic: DATA_TOPIC },
);
setActive(null);
setVoiceCue(null);
setHermes(null);
return status;
},
[active, room],
+14 -3
View File
@@ -1,13 +1,24 @@
import { StrictMode } from 'react';
import { createRoot } from 'react-dom/client';
import { ClerkProvider } from '@clerk/react';
import { shadcn } from '@clerk/ui/themes';
import App from './App.js';
import './index.css';
import '@clerk/ui/themes/shadcn.css';
import { TooltipProvider } from '@/components/ui/tooltip';
const clerkPublishableKey = import.meta.env.VITE_CLERK_PUBLISHABLE_KEY;
if (!clerkPublishableKey) {
throw new Error('Missing VITE_CLERK_PUBLISHABLE_KEY');
}
createRoot(document.getElementById('root')!).render(
<StrictMode>
<TooltipProvider>
<App />
</TooltipProvider>
<ClerkProvider publishableKey={clerkPublishableKey} appearance={{ theme: shadcn }}>
<TooltipProvider>
<App />
</TooltipProvider>
</ClerkProvider>
</StrictMode>,
);
+3245 -20
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -14,6 +14,14 @@ export interface Collision {
/** Engineer ids involved in the overlap. */
engineers: string[];
severity: CollisionSeverity;
/** Overlap type; undefined preserves historical same-file behavior. */
overlapKind?: 'file' | 'research';
/** Research topic/source for cross-channel code-edit ↔ research overlaps. */
researchTopic?: string;
researchSource?: string;
/** Engineer doing research and engineer editing the matched file. */
researcher?: string;
editor?: string;
/** Snapshot of relevant GitHub state at detection time. */
githubState?: GithubStateSnapshot;
/**
+6
View File
@@ -12,6 +12,12 @@ export interface EngineerContext {
currentSymbol?: string;
/** Higher-level feature/action inferred from the screen. */
activity?: string;
/** Broad screen mode: editor work or browser/docs research. */
mode?: 'editing' | 'research';
/** Topic being researched when mode is "research", e.g. "LiveKit agents". */
researchTopic?: string;
/** Research source domain, e.g. "docs.livekit.io". */
researchSource?: string;
/** Whether the screen shows uncommitted/unpushed changes (gutter/diff). */
hasUnpushedChanges?: boolean;
/** 01 confidence in this read of the screen. */
+1
View File
@@ -71,6 +71,7 @@ export type PodGraphActivityKind =
| 'intervention'
| 'outcome'
| 'learned'
| 'suppressed'
| 'agent';
export interface PodGraphActivity {
+2
View File
@@ -1,5 +1,6 @@
export type { Pod, PodInput, Engineer } from './pod.js';
export type { EngineerContext } from './engineer.js';
export type { UserLearningPodSummary, UserLearningProfile } from './user-learning.js';
export type {
PodActivityEvent,
PodActivityKind,
@@ -56,5 +57,6 @@ export type {
MemberWorkHistory,
MemberWorkHistoryEvent,
MemberWorkHistoryFile,
MemberWorkHistoryRoi,
MemberWorkHistorySource,
} from './member-history.js';
+20
View File
@@ -33,4 +33,24 @@ export interface MemberWorkHistory {
};
files: MemberWorkHistoryFile[];
timeline: MemberWorkHistoryEvent[];
/**
* Coordination ROI summary — clashes Hermes caught for this member. Optional
* so pods with no collisions / older payloads render without the band.
*/
roi?: MemberWorkHistoryRoi;
}
export interface MemberWorkHistoryRoi {
/** Estimated rework minutes saved (heuristic, labeled "~/est." in UI). */
savedMinutes: number;
/** Eligible collisions caught early (hard count). */
clashesCaught: number;
/** Distinct files that hit an eligible clash. */
filesDeconflicted: number;
/** Member files in flight that never hit a clash. */
conflictFreeFiles: number;
/** Total member files in flight (git changedFiles). */
totalFiles: number;
/** Per-kind breakdown for the tooltip. */
breakdown: { label: string; count: number; minutesEach: number }[];
}
+9
View File
@@ -11,6 +11,15 @@ export interface Pod {
description?: string;
/** Engineer display names PodMan uses when it speaks. */
members: string[];
/** Latest known Clerk/Gmail profile data per member display name. */
memberProfiles?: Record<
string,
{
displayName: string;
email?: string;
imageUrl?: string;
}
>;
createdAt: string;
updatedAt: string;
}
+37
View File
@@ -0,0 +1,37 @@
export interface UserLearningPodSummary {
podId: string;
podName?: string;
visits: number;
actions: string[];
firstSeenAt: string;
lastSeenAt: string;
}
export interface UserLearningProfile {
clerkUserId: string;
displayName?: string;
email?: string;
imageUrl?: string;
identities: string[];
pods: UserLearningPodSummary[];
recentWork: Array<{
podId: string;
file?: string;
activity?: string;
at: string;
}>;
collaborationStyle: string[];
workingStyle: string[];
goals: string[];
knowledge: string[];
counts: {
podActions: number;
observations: number;
gitStates: number;
collisionsInvolved: number;
outcomes: number;
conversationNotes: number;
hermesJobs: number;
};
updatedAt: string;
}