Add member work history and learning docs
This commit is contained in:
Yahya Alhinai
+151
-114
@@ -1,143 +1,180 @@
|
||||
# MongoDB Atlas Integration Spec
|
||||
|
||||
MongoDB Atlas is PodMan's shared memory. It stores live engineer state, the ownership map that enables continual learning, coordination events, and nudge history.
|
||||
Status: demo-backed / active
|
||||
|
||||
MongoDB Atlas is PodMan's shared memory. It stores live work observations,
|
||||
collision predictions, interventions, outcomes, latest engineer state, the
|
||||
materialized Team memory graph, and optional future recall records.
|
||||
|
||||
See also:
|
||||
|
||||
- [`docs/continual-learning/`](continual-learning/) for outcome-backed team
|
||||
memory.
|
||||
- [`docs/graph-discovery/`](graph-discovery/) for graph materialization and
|
||||
`$graphLookup` traversal.
|
||||
- [`docs/agent-learning/`](agent-learning/) for planned strategy-version
|
||||
records.
|
||||
|
||||
---
|
||||
|
||||
## Collections
|
||||
## Current Collections
|
||||
|
||||
### `engineer_states`
|
||||
|
||||
Latest context per engineer. Two writers, one collection — vision pipeline upserts vision fields, git watcher script upserts git fields independently. Hermes reads the merged document for event detection.
|
||||
Latest context per engineer. The local git watcher writes git fields; the vision
|
||||
pipeline may write screen-derived fields. Each writer updates only its own
|
||||
fields so MongoDB upserts merge cleanly.
|
||||
|
||||
```ts
|
||||
{
|
||||
_id: string, // engineerId (stable across sessions)
|
||||
podId: string,
|
||||
name: string, // display name
|
||||
Key fields:
|
||||
|
||||
// --- Vision fields (written by Hermes via POST /ingest) ---
|
||||
currentFile: string | null, // active file inferred from screen
|
||||
inferredTask: string | null, // what engineer appears to be doing
|
||||
terminalVisible: boolean,
|
||||
recentTerminalOutput: string | null,
|
||||
confidence: number, // Gemini Vision confidence (0–1)
|
||||
visionUpdatedAt: Date,
|
||||
- `podId`
|
||||
- `name`
|
||||
- `currentFile`
|
||||
- `inferredTask`
|
||||
- `confidence`
|
||||
- `changedFiles`
|
||||
- `diffStat`
|
||||
- `recentCommit`
|
||||
- `branch`
|
||||
- `visionUpdatedAt`
|
||||
- `gitUpdatedAt`
|
||||
- `updatedAt`
|
||||
|
||||
// --- Git fields (written directly by scripts/podman-agent.mjs) ---
|
||||
changedFiles: string[], // files with uncommitted changes (git status)
|
||||
diffStat: string | null, // e.g. "auth/middleware.ts | 24 +++++"
|
||||
recentCommit: string | null, // most recent commit message
|
||||
branch: string | null, // current branch name
|
||||
gitUpdatedAt: Date,
|
||||
Primary use: deterministic dirty/unpushed truth for collision detection and
|
||||
graph discovery.
|
||||
|
||||
// --- Shared ---
|
||||
updatedAt: Date // most recent write from either source
|
||||
}
|
||||
```
|
||||
### `observations`
|
||||
|
||||
**Index:** `{ podId: 1, updatedAt: -1 }`
|
||||
Structured perception events from consented screen context and agent inference.
|
||||
|
||||
**Two writers, no conflict:** vision upsert uses `$set` on vision fields only; git upsert uses `$set` on git fields only. MongoDB upsert semantics merge them cleanly.
|
||||
Key fields:
|
||||
|
||||
**Usage:** Hermes reads all documents for a given `podId` after each update to run event detection. Both vision and git context are available in the same document — `changedFiles` provides ground truth, `currentFile` provides screen context.
|
||||
- `podId`
|
||||
- `engineerId`
|
||||
- `currentFile`
|
||||
- `symbol`
|
||||
- `activity`
|
||||
- `confidence`
|
||||
- `observedAt`
|
||||
|
||||
Primary use: observe/store proof and active editing edges in the Team memory
|
||||
graph.
|
||||
|
||||
### `collisions`
|
||||
|
||||
Predicted coordination risks.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `id`
|
||||
- `podId`
|
||||
- `file`
|
||||
- `symbol`
|
||||
- `engineers`
|
||||
- `severity`
|
||||
- `memorySignature`
|
||||
- `githubState`
|
||||
- `detectedAt`
|
||||
|
||||
Primary use: collision cards, exact signature recall, and graph risk paths.
|
||||
|
||||
### `interventions`
|
||||
|
||||
Actions PodMan sent or suggested.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `id`
|
||||
- `podId`
|
||||
- `collisionId`
|
||||
- `kind`
|
||||
- `message`
|
||||
- `suggestedAction`
|
||||
- `status`
|
||||
- `createdAt`
|
||||
|
||||
Primary use: closing the loop from prediction to a visible card, Hermes message,
|
||||
or urgent voice cue.
|
||||
|
||||
### `outcomes`
|
||||
|
||||
Human or verifier supervision recorded through `POST /api/outcome`.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `interventionId`
|
||||
- `collisionId`
|
||||
- `accepted`
|
||||
- `wasRealCollision`
|
||||
- `recordedAt`
|
||||
|
||||
Primary use: accepted and dismissed outcomes drive exact recall, suppression,
|
||||
and learned graph paths.
|
||||
|
||||
### `team_model`
|
||||
|
||||
Durable per-pod summary memory.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `ownership`
|
||||
- `hotspots`
|
||||
- `graph`
|
||||
- `updatedAt`
|
||||
|
||||
Primary use: stable Team memory, including seeded `graph` snapshots used after
|
||||
live materialization and before demo fallback.
|
||||
|
||||
### `graph_nodes` and `graph_edges`
|
||||
|
||||
Normalized mirror of the Team memory graph for MongoDB traversal.
|
||||
|
||||
Indexes:
|
||||
|
||||
- `graph_nodes`: `{ podId: 1, id: 1 }` unique
|
||||
- `graph_edges`: `{ podId: 1, source: 1 }`
|
||||
|
||||
Primary use: `GET /api/pods/:podId/graph/reach/:id` with `$graphLookup`.
|
||||
|
||||
### Optional Future Collections
|
||||
|
||||
These are documented for planned work and should not be treated as active write
|
||||
paths unless implementation is added:
|
||||
|
||||
- `memory_vectors`
|
||||
- `agent_runs`
|
||||
- `agent_trace_events`
|
||||
- `strategy_versions`
|
||||
- `learning_proposals`
|
||||
|
||||
---
|
||||
|
||||
### `ownership_map`
|
||||
## Graph Truth Order
|
||||
|
||||
Tracks who works on which files. Built up over the session. **Persists across sessions** — this is the continual learning artifact.
|
||||
`GET /api/pods/:podId/graph` follows this order:
|
||||
|
||||
```ts
|
||||
{
|
||||
_id: string, // `${podId}:${file}`
|
||||
podId: string,
|
||||
file: string,
|
||||
primaryOwner: string, // engineerId with most recent activity on this file
|
||||
contributors: string[], // all engineerIds observed on this file
|
||||
observationCount: number, // total frames where this file was seen
|
||||
lastSeenAt: Date
|
||||
}
|
||||
```
|
||||
1. Live graph from real collections.
|
||||
2. Seeded graph from `team_model.graph` and mirrored graph records.
|
||||
3. Demo fallback graph for stage safety.
|
||||
|
||||
**Index:** `{ podId: 1, file: 1 }` (unique)
|
||||
|
||||
**Upsert logic:**
|
||||
|
||||
- On each context update where `currentFile` is non-null:
|
||||
- Increment `observationCount`
|
||||
- Update `primaryOwner` to the engineer with the most recent `lastSeenAt` on this file
|
||||
- Add engineerId to `contributors` if not present
|
||||
- Update `lastSeenAt`
|
||||
|
||||
**Continual learning:** Hermes loads this collection on startup for the pod. If history exists, it pre-populates the in-memory ownership cache before the first frame arrives.
|
||||
Seeded and fallback graphs are acceptable for demos only when labeled honestly.
|
||||
|
||||
---
|
||||
|
||||
### `events`
|
||||
## Demo Proof Path
|
||||
|
||||
Every coordination event detected by Hermes.
|
||||
|
||||
```ts
|
||||
{
|
||||
_id: ObjectId,
|
||||
podId: string,
|
||||
type: 'DEPENDENCY_READY' | 'BLOCKER_DETECTED' | 'DUPLICATE_WORK',
|
||||
involvedEngineers: string[],
|
||||
file: string | null,
|
||||
reason: string, // 1-sentence explanation from Gemini
|
||||
nudgeSent: boolean, // false if suppressed by cooldown
|
||||
detectedAt: Date
|
||||
}
|
||||
```
|
||||
|
||||
**Index:** `{ podId: 1, detectedAt: -1 }`
|
||||
Observe screen/git state -> detect collision -> send intervention -> accept or
|
||||
dismiss outcome -> recall similar event -> show changed graph or changed
|
||||
behavior.
|
||||
|
||||
---
|
||||
|
||||
### `nudges`
|
||||
## What MongoDB Does Not Store
|
||||
|
||||
Every voice nudge sent to the room.
|
||||
|
||||
```ts
|
||||
{
|
||||
_id: ObjectId,
|
||||
podId: string,
|
||||
eventId: ObjectId, // ref to events collection
|
||||
targetEngineers: string[],
|
||||
message: string, // the spoken text
|
||||
sentAt: Date
|
||||
}
|
||||
```
|
||||
|
||||
**Index:** `{ podId: 1, sentAt: -1 }`
|
||||
|
||||
**Cooldown check:** before sending a nudge, Hermes queries this collection for any nudge in the last 3 minutes for the same `podId`. If found, suppresses the new nudge and marks the event as `nudgeSent: false`.
|
||||
|
||||
---
|
||||
|
||||
## Hermes startup sequence
|
||||
|
||||
```
|
||||
1. Connect to Atlas using MONGODB_URI
|
||||
2. Load ownership_map for this podId
|
||||
3. Build in-memory cache: Map<file, { primaryOwner, contributors }>
|
||||
4. Begin accepting /ingest requests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Atlas configuration
|
||||
|
||||
- **Cluster tier:** M0 (free) is sufficient for hackathon scale
|
||||
- **Region:** same as DigitalOcean deployment (e.g. NYC1)
|
||||
- **Auth:** connection string in `MONGODB_URI` env var
|
||||
- **Collections created automatically** on first write (no schema migration needed)
|
||||
|
||||
---
|
||||
|
||||
## What MongoDB does NOT store
|
||||
|
||||
- Raw screenshot frames (too large — frames are processed in-memory by Hermes and discarded)
|
||||
- Full Gemini response objects (only extracted fields are stored)
|
||||
- Session recordings
|
||||
- Raw screenshot frames.
|
||||
- Screen recordings.
|
||||
- Secrets or credentials.
|
||||
- Full terminal logs.
|
||||
- Full Gemini response objects beyond extracted fields needed for memory.
|
||||
|
||||
Reference in New Issue
Block a user