docs: add learning and graph specs
This commit is contained in:
sb-iam
@@ -0,0 +1,89 @@
|
||||
# Agent Learning Plan
|
||||
|
||||
Status: draft
|
||||
Goal: ship a visible recursive self-improvement loop without overbuilding
|
||||
|
||||
## Must-Have
|
||||
|
||||
1. Store agent runs.
|
||||
2. Store trace summaries.
|
||||
3. Store active and candidate strategy versions.
|
||||
4. Attach verifier or outcome evidence.
|
||||
5. Show one strategy improvement in the demo narrative.
|
||||
|
||||
## Build Order
|
||||
|
||||
### R1: Trace the run
|
||||
|
||||
Write one `agent_runs` record for an important coordination decision and append
|
||||
trace events for:
|
||||
|
||||
- observation
|
||||
- recall
|
||||
- prediction
|
||||
- intervention
|
||||
- outcome
|
||||
- adaptation
|
||||
|
||||
### R2: Version the strategy
|
||||
|
||||
Create an active strategy version for one of:
|
||||
|
||||
- collision detector threshold
|
||||
- intervention routing
|
||||
- graph discovery filter
|
||||
- card wording prompt
|
||||
|
||||
### R3: Score the outcome
|
||||
|
||||
Use the simplest verifier:
|
||||
|
||||
- accepted real collision = useful
|
||||
- dismissed = noisy
|
||||
- no response after cooldown = uncertain
|
||||
|
||||
### R4: Propose a narrow change
|
||||
|
||||
Examples:
|
||||
|
||||
- "For this exact signature, prefer sync PR card."
|
||||
- "For dismissed docs-only overlaps, suppress voice escalation."
|
||||
- "For repeated auth.ts collisions, raise severity."
|
||||
|
||||
### R5: Promote or reject
|
||||
|
||||
Promote only when evidence is strong enough. Otherwise keep the candidate as
|
||||
rejected or open.
|
||||
|
||||
## Demo Path
|
||||
|
||||
1. Show baseline strategy.
|
||||
2. Trigger a collision.
|
||||
3. Accept or dismiss the intervention.
|
||||
4. Store outcome.
|
||||
5. Show a candidate strategy update.
|
||||
6. Promote it.
|
||||
7. Trigger a similar event.
|
||||
8. Show changed behavior.
|
||||
|
||||
## Nice-to-Have
|
||||
|
||||
- Strategy comparison panel.
|
||||
- Model-generated prompt patch with verifier.
|
||||
- Vector recall over strategy history.
|
||||
- Rollback UI.
|
||||
|
||||
## Cut
|
||||
|
||||
- Full autonomous code rewriting.
|
||||
- Multi-agent strategy debates.
|
||||
- Long-term benchmark suite.
|
||||
- Training a model.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The demo can point to a MongoDB record proving the agent changed behavior.
|
||||
- The changed behavior is visible.
|
||||
- The strategy has a parent and evidence.
|
||||
- Rejected or failed changes are not deleted.
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# Agent Learning Policy
|
||||
|
||||
Status: draft
|
||||
Scope: guardrails for recursive self-improvement
|
||||
|
||||
## Prime Rule
|
||||
|
||||
PodMan may improve its agent behavior only when the improvement is narrow,
|
||||
evidence-backed, versioned, and reversible.
|
||||
|
||||
## Allowed Learning
|
||||
|
||||
PodMan may learn:
|
||||
|
||||
- Which prompt version produces clearer interventions.
|
||||
- Which detector threshold reduces false positives.
|
||||
- Which routing channel gets accepted without being intrusive.
|
||||
- Which verifier best predicts user acceptance.
|
||||
- Which graph-discovery rule produces cleaner risk paths.
|
||||
|
||||
## Disallowed Learning
|
||||
|
||||
PodMan must not:
|
||||
|
||||
- Promote a strategy because the model says it is better.
|
||||
- Rewrite broad system behavior from one example.
|
||||
- Hide failures, dismissals, or rejected candidates.
|
||||
- Learn from raw screenshots, secrets, or private terminal content.
|
||||
- Turn voice into the default route.
|
||||
- Create irreversible actions without human approval.
|
||||
|
||||
## Promotion Rules
|
||||
|
||||
A candidate strategy can become active only when all are true:
|
||||
|
||||
1. It has a parent strategy version.
|
||||
2. It describes one concrete behavior change.
|
||||
3. It has a verifier plan.
|
||||
4. It has evidence from a run, outcome, or test.
|
||||
5. It improves or fixes the target metric.
|
||||
6. It does not increase user interruption without payoff.
|
||||
|
||||
## Rejection Rules
|
||||
|
||||
Reject and retain the candidate when:
|
||||
|
||||
- The verifier regresses.
|
||||
- The change is too broad.
|
||||
- The evidence is missing.
|
||||
- The candidate conflicts with privacy rules.
|
||||
- The candidate makes the demo less stable.
|
||||
|
||||
## Evidence Strength
|
||||
|
||||
| Evidence | Strength | Use |
|
||||
| --- | --- | --- |
|
||||
| Model opinion | Weak | Proposal only |
|
||||
| Trace observation | Medium | Candidate rationale |
|
||||
| Human accepted outcome | Strong | Promotion candidate |
|
||||
| Human dismissed outcome | Strong | Suppression or rejection |
|
||||
| Automated verifier | Strong | Promotion or rejection |
|
||||
| Repeated accepted exact signature | Strong | Policy confidence increase |
|
||||
|
||||
## Versioning Rules
|
||||
|
||||
- Strategy versions are immutable after promotion or rejection.
|
||||
- There is one active version per `podId + kind`.
|
||||
- A rollback activates the previous version; it does not edit history.
|
||||
- Parent-child lineage must be preserved.
|
||||
|
||||
## Safety Rules
|
||||
|
||||
- Store summaries, not raw sensitive content.
|
||||
- Prefer deterministic checks over model judgment.
|
||||
- Use exact MongoDB recall before vector recall.
|
||||
- Ask for approval before changing code or data with external effects.
|
||||
- Treat hackathon demo stability as a hard constraint.
|
||||
|
||||
## Demo Honesty
|
||||
|
||||
Seeded strategy versions are acceptable when labeled as demo-backed. Do not claim
|
||||
a strategy was learned live unless a run and outcome actually created the
|
||||
promotion evidence.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# Agent Learning Prompt
|
||||
|
||||
Use this prompt for an agent responsible for improving PodMan's own behavior.
|
||||
|
||||
## Prompt
|
||||
|
||||
You are PodMan's agent-learning evaluator.
|
||||
|
||||
Your job is to inspect a completed agent run, identify one narrow improvement,
|
||||
define how to verify it, and decide whether to propose, promote, or reject a
|
||||
strategy change.
|
||||
|
||||
You must not claim improvement without evidence. You must not propose broad
|
||||
rewrites. Keep every change small, reversible, and tied to a run or outcome.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Current active strategy version.
|
||||
- Agent run summary.
|
||||
- Trace events.
|
||||
- Intervention outcome.
|
||||
- Verifier result.
|
||||
- Recent false positives or accepted events.
|
||||
- Current demo constraints.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Identify the target behavior.
|
||||
2. Identify the failure or success evidence.
|
||||
3. Decide whether a strategy change is warranted.
|
||||
4. Propose one narrow change.
|
||||
5. Define the verifier.
|
||||
6. Decide status: no change, candidate, promote, reject.
|
||||
7. Write a short explanation suitable for the Team memory activity stream.
|
||||
|
||||
## Output Format
|
||||
|
||||
```text
|
||||
Target
|
||||
- Strategy kind:
|
||||
- Active version:
|
||||
- Behavior under review:
|
||||
|
||||
Evidence
|
||||
- Run:
|
||||
- Outcome:
|
||||
- Verifier:
|
||||
- Confidence:
|
||||
|
||||
Decision
|
||||
- Status:
|
||||
- Proposed change:
|
||||
- Why this is narrow:
|
||||
- Risk:
|
||||
|
||||
Verifier
|
||||
- Metric:
|
||||
- Passing condition:
|
||||
- Failing condition:
|
||||
|
||||
Memory Write
|
||||
- Collection:
|
||||
- Record summary:
|
||||
- Graph/activity summary:
|
||||
```
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- Exact outcomes beat model opinion.
|
||||
- Rejected candidates stay in memory.
|
||||
- No raw screenshots or secrets.
|
||||
- No broad policy change from one weak signal.
|
||||
- No voice-first behavior.
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
# Agent Learning Spec
|
||||
|
||||
Status: draft
|
||||
Scope: how PodMan agents improve their own prompts, policies, detectors, and routing behavior
|
||||
Owner: agent learning / recursive self-improvement
|
||||
|
||||
## Purpose
|
||||
|
||||
Agent learning is the recursive self-improvement layer. It is not the same as
|
||||
team memory. Team memory learns about engineers and work. Agent learning learns
|
||||
which agent strategies produce better outcomes.
|
||||
|
||||
The demo claim:
|
||||
|
||||
1. PodMan tries a coordination strategy.
|
||||
2. The run is traced in MongoDB.
|
||||
3. A verifier or human outcome scores it.
|
||||
4. Gemini or another agent proposes a narrow strategy change.
|
||||
5. The new strategy is versioned.
|
||||
6. A later run uses the improved strategy and shows a better result.
|
||||
|
||||
## Core Objects
|
||||
|
||||
### Agent run
|
||||
|
||||
One attempt to execute a goal.
|
||||
|
||||
```text
|
||||
agent_runs
|
||||
runId
|
||||
podId
|
||||
goal
|
||||
trigger
|
||||
strategyVersionId
|
||||
status
|
||||
startedAt
|
||||
completedAt
|
||||
score
|
||||
verifierSummary
|
||||
inputRefs
|
||||
outputRefs
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
running, succeeded, failed, improved, regressed, abandoned
|
||||
```
|
||||
|
||||
### Trace event
|
||||
|
||||
Append-only event log for a run.
|
||||
|
||||
```text
|
||||
agent_trace_events
|
||||
runId
|
||||
podId
|
||||
step
|
||||
phase
|
||||
eventType
|
||||
inputSummary
|
||||
outputSummary
|
||||
toolName
|
||||
error
|
||||
metrics
|
||||
createdAt
|
||||
```
|
||||
|
||||
### Strategy version
|
||||
|
||||
Versioned prompt, detector rule, policy, verifier, or routing strategy.
|
||||
|
||||
```text
|
||||
strategy_versions
|
||||
strategyVersionId
|
||||
podId
|
||||
kind
|
||||
name
|
||||
parentVersionId
|
||||
status
|
||||
summary
|
||||
promptText
|
||||
policy
|
||||
verifier
|
||||
metrics
|
||||
createdAt
|
||||
promotedAt
|
||||
```
|
||||
|
||||
Allowed `kind` values:
|
||||
|
||||
```text
|
||||
prompt, policy, detector, verifier, routing
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
candidate, active, retired, rejected
|
||||
```
|
||||
|
||||
### Learning proposal
|
||||
|
||||
A candidate change before promotion.
|
||||
|
||||
```text
|
||||
learning_proposals
|
||||
proposalId
|
||||
podId
|
||||
sourceRunId
|
||||
targetKind
|
||||
parentVersionId
|
||||
proposedChange
|
||||
rationale
|
||||
verifierPlan
|
||||
status
|
||||
createdAt
|
||||
resolvedAt
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
open, accepted, rejected, superseded
|
||||
```
|
||||
|
||||
## MongoDB Indexes
|
||||
|
||||
| Collection | Index | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `agent_runs` | `{ podId: 1, startedAt: -1 }` | Recent run history |
|
||||
| `agent_runs` | `{ podId: 1, strategyVersionId: 1 }` | Compare strategy performance |
|
||||
| `agent_trace_events` | `{ runId: 1, step: 1 }` | Reconstruct run |
|
||||
| `strategy_versions` | `{ podId: 1, kind: 1, status: 1 }` | Find active strategy |
|
||||
| `strategy_versions` | `{ podId: 1, createdAt: -1 }` | Version history |
|
||||
| `learning_proposals` | `{ podId: 1, status: 1 }` | Open candidate changes |
|
||||
|
||||
## Learning Loop
|
||||
|
||||
```text
|
||||
observe run -> score run -> propose change -> test candidate -> promote or reject
|
||||
```
|
||||
|
||||
Agent learning must always connect these records:
|
||||
|
||||
```text
|
||||
agent_run -> trace_events -> verifier result -> learning_proposal -> strategy_version
|
||||
```
|
||||
|
||||
## Verifier Contract
|
||||
|
||||
Every promoted strategy needs a verifier signal.
|
||||
|
||||
Allowed verifier types:
|
||||
|
||||
- Human accepted or dismissed outcome.
|
||||
- Test pass or fail result.
|
||||
- Reduced false positive rate.
|
||||
- Reduced intervention count with same or better accepted outcomes.
|
||||
- Faster successful run.
|
||||
- Better graph discovery precision.
|
||||
- Explicit demo operator approval.
|
||||
|
||||
Self-evaluation alone is not enough to promote a strategy.
|
||||
|
||||
## Relationship to Team Graph
|
||||
|
||||
Agent learning can appear in the Team memory graph as activity and loop status,
|
||||
but it should not clutter the main risk graph by default.
|
||||
|
||||
Graph discovery may show:
|
||||
|
||||
- `agent_run` activity in the stream.
|
||||
- `strategy_versions` count in the learning loop.
|
||||
- A selected-node detail saying a policy changed because a prior outcome was
|
||||
dismissed or accepted.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Every strategy change has a parent.
|
||||
- Every promoted strategy cites evidence.
|
||||
- Rejected strategies are retained with a reason.
|
||||
- Agent traces are append-only.
|
||||
- The system can answer: "What changed, why, and did it help?"
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
# Continual Learning Plan
|
||||
|
||||
Status: draft
|
||||
Goal: prove PodMan learns from outcomes in the hackathon demo
|
||||
|
||||
## Must-Have Demo Loop
|
||||
|
||||
1. Observe two engineers touching the same file.
|
||||
2. Store the observation and git state in MongoDB.
|
||||
3. Predict a collision.
|
||||
4. Send a card or Hermes message.
|
||||
5. Record accept or dismiss outcome.
|
||||
6. Adapt `team_model`.
|
||||
7. Show the learned graph edge or changed future behavior.
|
||||
|
||||
## Build Order
|
||||
|
||||
### R1: Make exact recall reliable
|
||||
|
||||
- Normalize file paths.
|
||||
- Build stable memory signatures.
|
||||
- Look up prior accepted and dismissed outcomes.
|
||||
- Prefer exact recall over vector recall.
|
||||
|
||||
### R2: Make outcomes update memory
|
||||
|
||||
- Accepted real collision creates or strengthens ownership.
|
||||
- Accepted real collision creates `learned_from`.
|
||||
- Dismissed outcome lowers confidence or suppresses route.
|
||||
|
||||
### R3: Expose loop data to the graph
|
||||
|
||||
- Add optional loop snapshot.
|
||||
- Add optional activity stream.
|
||||
- Keep existing `PodGraph` fields stable.
|
||||
|
||||
### R4: Show the observatory
|
||||
|
||||
- Render observe/store/predict/outcome/adapt.
|
||||
- Show recent activity.
|
||||
- Make selected-node detail explain why memory changed.
|
||||
|
||||
### R5: Prepare a clean demo chain
|
||||
|
||||
- Ensure one collision -> intervention -> accepted outcome exists.
|
||||
- Ensure repeated signature recalls prior memory.
|
||||
- Verify graph shows learned ownership.
|
||||
|
||||
## Nice-to-Have
|
||||
|
||||
- Atlas Vector Search over memory summaries.
|
||||
- Confidence scoring per ownership edge.
|
||||
- Per-file memory timeline.
|
||||
- Strategy promotion tied to outcomes.
|
||||
|
||||
## Cut
|
||||
|
||||
- Raw screenshot storage.
|
||||
- Full autonomous training.
|
||||
- Broad dashboard metrics.
|
||||
- Multi-pod learning generalization.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A judge can see what changed in memory.
|
||||
- The second similar event behaves differently.
|
||||
- Exact MongoDB records prove the loop.
|
||||
- The graph remains legible with real data.
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
# Continual Learning Policy
|
||||
|
||||
Status: draft
|
||||
Scope: what PodMan may learn about a team
|
||||
|
||||
## Prime Rule
|
||||
|
||||
PodMan learns coordination patterns, not personal surveillance profiles.
|
||||
|
||||
## Allowed Memory
|
||||
|
||||
PodMan may store:
|
||||
|
||||
- File and symbol ownership.
|
||||
- Active file overlap.
|
||||
- Repeated collision signatures.
|
||||
- Intervention history.
|
||||
- Accepted and dismissed outcomes.
|
||||
- Routing preferences by event type and severity.
|
||||
- Summaries of decisions relevant to future coordination.
|
||||
|
||||
## Forbidden Memory
|
||||
|
||||
PodMan must not store:
|
||||
|
||||
- Raw screenshots.
|
||||
- Screen recordings.
|
||||
- Secrets or credentials.
|
||||
- Full terminal logs.
|
||||
- Personal performance judgments.
|
||||
- Private content unrelated to the coding task.
|
||||
|
||||
## Evidence Policy
|
||||
|
||||
| Evidence | Can predict? | Can adapt memory? |
|
||||
| --- | --- | --- |
|
||||
| Vision only | Yes, low confidence | No |
|
||||
| Git watcher | Yes | No, unless repeated |
|
||||
| GitHub state | Yes | No, unless verified |
|
||||
| Accepted real outcome | Yes | Yes |
|
||||
| Dismissed outcome | Yes, for suppression | Yes, as negative signal |
|
||||
| Verifier result | Yes | Yes |
|
||||
|
||||
## Intervention Policy
|
||||
|
||||
Use the least intrusive channel:
|
||||
|
||||
1. Watch quietly.
|
||||
2. Card.
|
||||
3. Hermes message.
|
||||
4. Voice.
|
||||
|
||||
Voice is only for urgent, high-confidence, time-sensitive risks.
|
||||
|
||||
## Adaptation Policy
|
||||
|
||||
Allowed adaptations:
|
||||
|
||||
- Add learned ownership after accepted real outcome.
|
||||
- Raise confidence for repeated accepted signatures.
|
||||
- Lower confidence for dismissed signatures.
|
||||
- Prefer the previously accepted intervention kind.
|
||||
- Suppress repeated low-value warnings.
|
||||
|
||||
Disallowed adaptations:
|
||||
|
||||
- Broad threshold changes from one example.
|
||||
- Treating vector similarity as proof.
|
||||
- Hiding dismissals.
|
||||
- Making interruption more aggressive without evidence.
|
||||
|
||||
## Retention Policy
|
||||
|
||||
Keep:
|
||||
|
||||
- Outcomes.
|
||||
- Signatures.
|
||||
- Team model memory.
|
||||
- Strategy metrics.
|
||||
|
||||
Summarize or expire:
|
||||
|
||||
- Old observations.
|
||||
- Low-confidence vision-only events.
|
||||
- Detailed trace text.
|
||||
|
||||
Delete immediately:
|
||||
|
||||
- Secrets.
|
||||
- Accidental raw sensitive captures.
|
||||
|
||||
## Demo Policy
|
||||
|
||||
Seeded data is acceptable only if the demo script is honest about it. Live
|
||||
learning requires a live or staged outcome write that visibly updates the graph
|
||||
or future decision.
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
# Continual Learning Prompt
|
||||
|
||||
Use this prompt for the agent that decides what PodMan should remember from a
|
||||
coordination event.
|
||||
|
||||
## Prompt
|
||||
|
||||
You are PodMan's continual-learning memory agent.
|
||||
|
||||
Your job is to inspect observations, collisions, interventions, and outcomes,
|
||||
then decide what team memory should be updated. You must separate observed
|
||||
facts, inferred risks, human outcomes, and durable learned memory.
|
||||
|
||||
Do not claim something was learned unless an accepted real outcome, verifier, or
|
||||
human label supports it.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Pod id.
|
||||
- Recent engineer states.
|
||||
- Recent observations.
|
||||
- Candidate collision.
|
||||
- Prior exact-signature memory.
|
||||
- Intervention record.
|
||||
- Outcome record.
|
||||
- Current team model.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Normalize file and symbol.
|
||||
2. Build exact signature.
|
||||
3. Check prior accepted and dismissed outcomes.
|
||||
4. Classify the current event.
|
||||
5. Decide whether memory should change.
|
||||
6. Emit the graph impact.
|
||||
7. Write a short explanation.
|
||||
|
||||
## Output Format
|
||||
|
||||
```text
|
||||
Event
|
||||
- Signature:
|
||||
- Engineers:
|
||||
- File:
|
||||
- Symbol:
|
||||
- Evidence:
|
||||
|
||||
Prior Memory
|
||||
- Accepted matches:
|
||||
- Dismissed matches:
|
||||
- Ownership:
|
||||
|
||||
Decision
|
||||
- Memory action:
|
||||
- Confidence:
|
||||
- Reason:
|
||||
|
||||
Graph Impact
|
||||
- Nodes:
|
||||
- Edges:
|
||||
- Activity text:
|
||||
|
||||
Safety
|
||||
- Sensitive data present:
|
||||
- Redaction needed:
|
||||
```
|
||||
|
||||
## Memory Actions
|
||||
|
||||
Allowed actions:
|
||||
|
||||
- no_change
|
||||
- strengthen_signature
|
||||
- weaken_signature
|
||||
- create_learned_owner
|
||||
- update_route_preference
|
||||
- suppress_signature
|
||||
- request_human_label
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- Exact recall before vector recall.
|
||||
- Dismissals are learning signals.
|
||||
- `learned_from` requires accepted real outcome.
|
||||
- Store summaries, not raw screen content.
|
||||
- Prefer less intrusive future behavior when uncertain.
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
# Continual Learning Spec
|
||||
|
||||
Status: draft
|
||||
Scope: how PodMan learns team memory from live work and outcomes
|
||||
Owner: continual learning / Team memory
|
||||
|
||||
## Purpose
|
||||
|
||||
Continual learning is the product proof that PodMan gets more useful from use.
|
||||
It learns team-level coordination memory: ownership, repeated collisions,
|
||||
accepted interventions, dismissed noise, and preferred routing.
|
||||
|
||||
The visible loop:
|
||||
|
||||
```text
|
||||
observe -> store -> predict -> outcome -> adapt
|
||||
```
|
||||
|
||||
## Source Collections
|
||||
|
||||
### `engineer_states`
|
||||
|
||||
Latest per-engineer state from vision and local git.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `name`
|
||||
- `currentFile`
|
||||
- `changedFiles`
|
||||
- `branch`
|
||||
- `confidence`
|
||||
- `visionUpdatedAt`
|
||||
- `gitUpdatedAt`
|
||||
- `updatedAt`
|
||||
|
||||
### `observations`
|
||||
|
||||
Structured perception events.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `engineerId`
|
||||
- `currentFile`
|
||||
- `symbol`
|
||||
- `activity`
|
||||
- `confidence`
|
||||
- `observedAt`
|
||||
|
||||
### `collisions`
|
||||
|
||||
Predicted risk events.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `id`
|
||||
- `podId`
|
||||
- `file`
|
||||
- `symbol`
|
||||
- `engineers`
|
||||
- `severity`
|
||||
- `status`
|
||||
- `memorySignature`
|
||||
- `detectedAt`
|
||||
|
||||
### `interventions`
|
||||
|
||||
Actions PodMan sent or suggested.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `id`
|
||||
- `podId`
|
||||
- `collisionId`
|
||||
- `kind`
|
||||
- `channel`
|
||||
- `message`
|
||||
- `suggestedAction`
|
||||
- `createdAt`
|
||||
|
||||
### `outcomes`
|
||||
|
||||
Human or verifier supervision.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `id`
|
||||
- `podId`
|
||||
- `interventionId`
|
||||
- `collisionId`
|
||||
- `accepted`
|
||||
- `wasRealCollision`
|
||||
- `learnedOwner`
|
||||
- `recordedAt`
|
||||
|
||||
### `team_model`
|
||||
|
||||
Durable pod memory.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `graph`
|
||||
- `ownership`
|
||||
- `collisionSignatures`
|
||||
- `interventionPolicy`
|
||||
- `updatedAt`
|
||||
|
||||
### `memory_vectors`
|
||||
|
||||
Optional semantic recall. Exact recall comes first.
|
||||
|
||||
Key fields:
|
||||
|
||||
- `podId`
|
||||
- `sourceKind`
|
||||
- `sourceId`
|
||||
- `text`
|
||||
- `embedding`
|
||||
- `embeddingModel`
|
||||
- `tags`
|
||||
|
||||
## Learning Rules
|
||||
|
||||
### Observe
|
||||
|
||||
Write structured evidence from vision, git, GitHub, and agent traces.
|
||||
|
||||
### Store
|
||||
|
||||
Persist source records and materialized summaries. Do not store raw screenshots
|
||||
or recordings.
|
||||
|
||||
### Predict
|
||||
|
||||
Create a collision when multiple engineers converge on the same normalized file
|
||||
or symbol and at least one signal shows active or unpushed work.
|
||||
|
||||
### Outcome
|
||||
|
||||
Record whether the intervention was accepted, dismissed, real, or false.
|
||||
|
||||
### Adapt
|
||||
|
||||
Only accepted real outcomes can create `learned_from` graph edges. Dismissals
|
||||
adapt suppression, routing, or confidence.
|
||||
|
||||
## Exact Signature
|
||||
|
||||
Use deterministic signatures:
|
||||
|
||||
```text
|
||||
podId:eventType:normalizedFile:symbol:sortedEngineers
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Sort engineer names.
|
||||
- Normalize file paths.
|
||||
- Use `*` for missing symbol.
|
||||
- Never include timestamps.
|
||||
|
||||
## UI-Facing Loop Snapshot
|
||||
|
||||
The graph response may include:
|
||||
|
||||
```text
|
||||
loop
|
||||
activeStep
|
||||
steps[]
|
||||
key
|
||||
label
|
||||
value
|
||||
detail
|
||||
status
|
||||
```
|
||||
|
||||
Step mapping:
|
||||
|
||||
| Step | Source |
|
||||
| --- | --- |
|
||||
| Observe | recent observations and git updates |
|
||||
| Store | team model, graph records, memory vectors |
|
||||
| Predict | open collisions |
|
||||
| Outcome | accepted and dismissed outcomes |
|
||||
| Adapt | learned owners, learned edges, strategy changes |
|
||||
|
||||
## Activity Stream
|
||||
|
||||
The graph response may include:
|
||||
|
||||
```text
|
||||
activity[]
|
||||
id
|
||||
at
|
||||
kind
|
||||
title
|
||||
detail
|
||||
nodeId
|
||||
edgeId
|
||||
```
|
||||
|
||||
Allowed `kind` values:
|
||||
|
||||
```text
|
||||
editing, collision, intervention, outcome, learned, agent
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The system can show one accepted outcome changing future memory.
|
||||
- Exact recall works without vector search.
|
||||
- The Team memory graph can explain the learning loop.
|
||||
- Dismissals and false positives are retained.
|
||||
- The demo does not rely on raw screenshots or hidden state.
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
# Graph Discovery Plan
|
||||
|
||||
Status: draft
|
||||
Goal: make MongoDB graph discovery visible as a dynamic learning observatory
|
||||
|
||||
## Must-Have
|
||||
|
||||
1. Keep live materializer as source of graph truth.
|
||||
2. Add optional loop and activity fields.
|
||||
3. Build a dynamic graph layout.
|
||||
4. Default to risk path.
|
||||
5. Make selected-node detail explain the story.
|
||||
|
||||
## Build Order
|
||||
|
||||
### R1: Stabilize discovered graph
|
||||
|
||||
- Keep file and engineer noise filters.
|
||||
- Keep collision collapse.
|
||||
- Keep priority for accepted-outcome paths.
|
||||
- Keep graph size capped.
|
||||
|
||||
### R2: Add observatory data
|
||||
|
||||
- Compute learning-loop snapshot.
|
||||
- Compute activity stream.
|
||||
- Preserve current graph contract.
|
||||
|
||||
### R3: Improve path selection
|
||||
|
||||
- Pick one primary risk path.
|
||||
- Include learned path when present.
|
||||
- Dim unrelated collisions and repeated interventions.
|
||||
|
||||
### R4: Render dynamically
|
||||
|
||||
- Use `d3-force` or animated layered layout.
|
||||
- Make nodes draggable.
|
||||
- Curve or bundle edges.
|
||||
- Animate `learned_from`.
|
||||
|
||||
### R5: Verify with real data
|
||||
|
||||
- Fetch live `demo-pod` graph.
|
||||
- Confirm labels do not collide badly.
|
||||
- Confirm red edges do not dominate.
|
||||
- Confirm activity and loop explain the graph.
|
||||
|
||||
## Nice-to-Have
|
||||
|
||||
- Reachability panel using `$graphLookup`.
|
||||
- Hover path previews.
|
||||
- Edge bundling by file or collision.
|
||||
- Time scrubber for graph snapshots.
|
||||
|
||||
## Cut
|
||||
|
||||
- Generic analytics dashboard.
|
||||
- Large graph database migration.
|
||||
- Rendering every historical event.
|
||||
- Static fixed-column final layout.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Risk path is obvious in 10 seconds.
|
||||
- Learned path is visible when data exists.
|
||||
- Whole graph mode exists but is not the default.
|
||||
- The graph remains backed by MongoDB, not hardcoded mock data.
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# Graph Discovery Policy
|
||||
|
||||
Status: draft
|
||||
Scope: graph hygiene, evidence thresholds, and UI truthfulness
|
||||
|
||||
## Prime Rule
|
||||
|
||||
The graph must be sparse enough to explain the learning loop and truthful enough
|
||||
to audit from MongoDB.
|
||||
|
||||
## Node Policy
|
||||
|
||||
Create nodes only when they add explanation value.
|
||||
|
||||
Allowed:
|
||||
|
||||
- Current engineers.
|
||||
- Real files.
|
||||
- Current or recent collisions.
|
||||
- Interventions tied to surviving collisions.
|
||||
- Learned ownership paths.
|
||||
|
||||
Avoid:
|
||||
|
||||
- Test engineers.
|
||||
- Scratch files.
|
||||
- URLs or environment values misread as files.
|
||||
- Repeated identical intervention diamonds.
|
||||
- Orphan nodes with no story value.
|
||||
|
||||
## Edge Policy
|
||||
|
||||
Edges need evidence.
|
||||
|
||||
| Edge | Required evidence |
|
||||
| --- | --- |
|
||||
| `editing` | observation or git state |
|
||||
| `touches` | file involved in collision |
|
||||
| `collides` | collision prediction |
|
||||
| `warns` | intervention record |
|
||||
| `learned_from` | accepted real outcome |
|
||||
| `owns` | learned or configured ownership |
|
||||
|
||||
## De-Hairball Policy
|
||||
|
||||
Default mode must not show every relationship equally.
|
||||
|
||||
Rules:
|
||||
|
||||
- Default to risk path.
|
||||
- Collapse repeated collision signatures.
|
||||
- Cap files and collisions.
|
||||
- Dim non-risk edges.
|
||||
- Bundle or curve dense edges.
|
||||
- Hide low-priority labels until hover or select.
|
||||
- Prefer selected-node explanation over labels everywhere.
|
||||
|
||||
## Truthfulness Policy
|
||||
|
||||
- Do not show `learned_from` for orphaned or dismissed outcomes.
|
||||
- Do not label vector similarity as learned memory.
|
||||
- Do not show demo seed as live learning unless labeled.
|
||||
- Do not hide false positives from activity or memory.
|
||||
|
||||
## Privacy Policy
|
||||
|
||||
Graph labels should not expose secrets, raw terminal output, or sensitive file
|
||||
contents. File paths are acceptable when they are repo paths and not secret
|
||||
values.
|
||||
|
||||
## Visual Policy
|
||||
|
||||
Semantic colors stay stable:
|
||||
|
||||
- Engineer: blue.
|
||||
- File: slate.
|
||||
- Feature: amber.
|
||||
- Collision: red.
|
||||
- Intervention: violet.
|
||||
- Learned: violet dashed edge.
|
||||
|
||||
Chrome should use the app's light shadcn tokens.
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Graph Discovery Prompt
|
||||
|
||||
Use this prompt for an agent that materializes or reviews PodMan's Team memory
|
||||
graph.
|
||||
|
||||
## Prompt
|
||||
|
||||
You are PodMan's graph discovery agent.
|
||||
|
||||
Your job is to turn MongoDB records into a sparse, truthful graph that explains
|
||||
the continual-learning loop. Do not maximize node count. Maximize legibility and
|
||||
evidence.
|
||||
|
||||
The default output should show the risk path and learned path, not every
|
||||
possible edge.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Pod id.
|
||||
- Pod roster.
|
||||
- Recent engineer states.
|
||||
- Recent observations.
|
||||
- Collisions.
|
||||
- Interventions.
|
||||
- Outcomes.
|
||||
- Team model.
|
||||
- Existing graph nodes and edges.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Normalize file paths.
|
||||
2. Remove noise.
|
||||
3. Create engineer and file nodes.
|
||||
4. Collapse repeated collisions by signature.
|
||||
5. Preserve accepted-outcome paths.
|
||||
6. Create intervention nodes for surviving collisions.
|
||||
7. Create learned edges only from accepted real outcomes.
|
||||
8. Select the primary risk path.
|
||||
9. Build activity and loop summaries.
|
||||
10. Explain selected-node stories.
|
||||
|
||||
## Output Format
|
||||
|
||||
```text
|
||||
Graph Summary
|
||||
- Pod:
|
||||
- Nodes:
|
||||
- Edges:
|
||||
- Primary risk path:
|
||||
- Learned path:
|
||||
|
||||
Discovery Decisions
|
||||
- Collapsed:
|
||||
- Dropped as noise:
|
||||
- Preserved because learned:
|
||||
|
||||
Loop
|
||||
- Observe:
|
||||
- Store:
|
||||
- Predict:
|
||||
- Outcome:
|
||||
- Adapt:
|
||||
|
||||
Activity
|
||||
- Recent events:
|
||||
|
||||
Risks
|
||||
- Missing evidence:
|
||||
- Potential hairball:
|
||||
- Demo caveat:
|
||||
```
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- No `learned_from` without accepted real outcome.
|
||||
- No raw screenshots or secrets in labels.
|
||||
- Do not rewrite the backend materializer unless explicitly asked.
|
||||
- Prefer additive graph fields.
|
||||
- Default to risk path.
|
||||
- Keep whole graph optional.
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
# Graph Discovery Spec
|
||||
|
||||
Status: draft
|
||||
Scope: how PodMan discovers graph nodes, edges, risk paths, and learning paths from MongoDB
|
||||
Owner: graph discovery / Team memory observatory
|
||||
|
||||
## Purpose
|
||||
|
||||
Graph discovery turns MongoDB memory into a legible Team memory graph. It is not
|
||||
only layout. It decides which relationships matter, which path is highlighted,
|
||||
and which evidence explains the graph.
|
||||
|
||||
The graph must answer:
|
||||
|
||||
1. Who is working?
|
||||
2. Which files or symbols overlap?
|
||||
3. Where is the risk?
|
||||
4. What did PodMan do?
|
||||
5. What outcome changed memory?
|
||||
|
||||
## Source Data
|
||||
|
||||
Graph discovery reads:
|
||||
|
||||
- `pods`
|
||||
- `engineer_states`
|
||||
- `observations`
|
||||
- `collisions`
|
||||
- `interventions`
|
||||
- `outcomes`
|
||||
- `team_model`
|
||||
- `graph_nodes`
|
||||
- `graph_edges`
|
||||
- optional `memory_vectors`
|
||||
- optional `agent_runs`
|
||||
- optional `strategy_versions`
|
||||
|
||||
## UI Graph Contract
|
||||
|
||||
```text
|
||||
PodGraph
|
||||
podId
|
||||
generatedAt
|
||||
nodes
|
||||
edges
|
||||
metrics
|
||||
loop?
|
||||
activity?
|
||||
```
|
||||
|
||||
Node kinds:
|
||||
|
||||
```text
|
||||
engineer, feature, file, collision, intervention
|
||||
```
|
||||
|
||||
Edge kinds:
|
||||
|
||||
```text
|
||||
owns, editing, touches, collides, warns, learned_from
|
||||
```
|
||||
|
||||
## Discovery Rules
|
||||
|
||||
### Engineer nodes
|
||||
|
||||
Create from pod roster, recent observations, git state, or collision membership.
|
||||
|
||||
### File nodes
|
||||
|
||||
Create only from normalized real file paths. Reject noise such as URLs, env
|
||||
values, scratch names, and non-file strings.
|
||||
|
||||
### Collision nodes
|
||||
|
||||
Create from distinct collision signatures. Collapse repeats. Prioritize
|
||||
collisions referenced by accepted outcomes.
|
||||
|
||||
### Intervention nodes
|
||||
|
||||
Create one visible intervention per surviving collision unless whole-graph mode
|
||||
explicitly expands history.
|
||||
|
||||
### Learned paths
|
||||
|
||||
Create `learned_from` only when an accepted real outcome links an intervention
|
||||
to a durable memory update.
|
||||
|
||||
## Path Modes
|
||||
|
||||
### Risk path
|
||||
|
||||
Default mode. Highlight the clearest current chain:
|
||||
|
||||
```text
|
||||
engineer -> file -> collision -> intervention -> learned owner
|
||||
```
|
||||
|
||||
Dim unrelated graph material.
|
||||
|
||||
### Learning edges
|
||||
|
||||
Highlight `learned_from`, `owns`, and the outcomes that produced them.
|
||||
|
||||
### Whole graph
|
||||
|
||||
Show all materialized nodes and edges with de-emphasized non-critical edges.
|
||||
|
||||
## MongoDB Traversal
|
||||
|
||||
Use `graph_edges` for reachability:
|
||||
|
||||
```text
|
||||
source -> target -> next target
|
||||
```
|
||||
|
||||
Primary traversal questions:
|
||||
|
||||
- What risks does this engineer reach?
|
||||
- Which files feed this collision?
|
||||
- Which intervention came from this collision?
|
||||
- Which learned owner came from this intervention?
|
||||
|
||||
## Metrics
|
||||
|
||||
Minimum metrics:
|
||||
|
||||
- Learned owners.
|
||||
- Open risk paths.
|
||||
- Accept rate.
|
||||
|
||||
Optional metrics:
|
||||
|
||||
- Observations.
|
||||
- Interventions.
|
||||
- Memory vectors.
|
||||
- Strategy versions.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Default graph is not a hairball.
|
||||
- Every visible learned edge has outcome evidence.
|
||||
- Every selected node can explain why it matters.
|
||||
- Activity stream matches graph events.
|
||||
- Graph can be rebuilt from MongoDB source records.
|
||||
|
||||
Reference in New Issue
Block a user