docs: add learning and graph specs
This commit is contained in:
sb-iam
@@ -0,0 +1,89 @@
|
||||
# Agent Learning Plan
|
||||
|
||||
Status: draft
|
||||
Goal: ship a visible recursive self-improvement loop without overbuilding
|
||||
|
||||
## Must-Have
|
||||
|
||||
1. Store agent runs.
|
||||
2. Store trace summaries.
|
||||
3. Store active and candidate strategy versions.
|
||||
4. Attach verifier or outcome evidence.
|
||||
5. Show one strategy improvement in the demo narrative.
|
||||
|
||||
## Build Order
|
||||
|
||||
### R1: Trace the run
|
||||
|
||||
Write one `agent_runs` record for an important coordination decision and append
|
||||
trace events for:
|
||||
|
||||
- observation
|
||||
- recall
|
||||
- prediction
|
||||
- intervention
|
||||
- outcome
|
||||
- adaptation
|
||||
|
||||
### R2: Version the strategy
|
||||
|
||||
Create an active strategy version for one of:
|
||||
|
||||
- collision detector threshold
|
||||
- intervention routing
|
||||
- graph discovery filter
|
||||
- card wording prompt
|
||||
|
||||
### R3: Score the outcome
|
||||
|
||||
Use the simplest verifier:
|
||||
|
||||
- accepted real collision = useful
|
||||
- dismissed = noisy
|
||||
- no response after cooldown = uncertain
|
||||
|
||||
### R4: Propose a narrow change
|
||||
|
||||
Examples:
|
||||
|
||||
- "For this exact signature, prefer sync PR card."
|
||||
- "For dismissed docs-only overlaps, suppress voice escalation."
|
||||
- "For repeated auth.ts collisions, raise severity."
|
||||
|
||||
### R5: Promote or reject
|
||||
|
||||
Promote only when evidence is strong enough. Otherwise keep the candidate as
|
||||
rejected or open.
|
||||
|
||||
## Demo Path
|
||||
|
||||
1. Show baseline strategy.
|
||||
2. Trigger a collision.
|
||||
3. Accept or dismiss the intervention.
|
||||
4. Store outcome.
|
||||
5. Show a candidate strategy update.
|
||||
6. Promote it.
|
||||
7. Trigger a similar event.
|
||||
8. Show changed behavior.
|
||||
|
||||
## Nice-to-Have
|
||||
|
||||
- Strategy comparison panel.
|
||||
- Model-generated prompt patch with verifier.
|
||||
- Vector recall over strategy history.
|
||||
- Rollback UI.
|
||||
|
||||
## Cut
|
||||
|
||||
- Full autonomous code rewriting.
|
||||
- Multi-agent strategy debates.
|
||||
- Long-term benchmark suite.
|
||||
- Training a model.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The demo can point to a MongoDB record proving the agent changed behavior.
|
||||
- The changed behavior is visible.
|
||||
- The strategy has a parent and evidence.
|
||||
- Rejected or failed changes are not deleted.
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# Agent Learning Policy
|
||||
|
||||
Status: draft
|
||||
Scope: guardrails for recursive self-improvement
|
||||
|
||||
## Prime Rule
|
||||
|
||||
PodMan may improve its agent behavior only when the improvement is narrow,
|
||||
evidence-backed, versioned, and reversible.
|
||||
|
||||
## Allowed Learning
|
||||
|
||||
PodMan may learn:
|
||||
|
||||
- Which prompt version produces clearer interventions.
|
||||
- Which detector threshold reduces false positives.
|
||||
- Which routing channel gets accepted without being intrusive.
|
||||
- Which verifier best predicts user acceptance.
|
||||
- Which graph-discovery rule produces cleaner risk paths.
|
||||
|
||||
## Disallowed Learning
|
||||
|
||||
PodMan must not:
|
||||
|
||||
- Promote a strategy because the model says it is better.
|
||||
- Rewrite broad system behavior from one example.
|
||||
- Hide failures, dismissals, or rejected candidates.
|
||||
- Learn from raw screenshots, secrets, or private terminal content.
|
||||
- Turn voice into the default route.
|
||||
- Create irreversible actions without human approval.
|
||||
|
||||
## Promotion Rules
|
||||
|
||||
A candidate strategy can become active only when all are true:
|
||||
|
||||
1. It has a parent strategy version.
|
||||
2. It describes one concrete behavior change.
|
||||
3. It has a verifier plan.
|
||||
4. It has evidence from a run, outcome, or test.
|
||||
5. It improves or fixes the target metric.
|
||||
6. It does not increase user interruption without payoff.
|
||||
|
||||
## Rejection Rules
|
||||
|
||||
Reject and retain the candidate when:
|
||||
|
||||
- The verifier regresses.
|
||||
- The change is too broad.
|
||||
- The evidence is missing.
|
||||
- The candidate conflicts with privacy rules.
|
||||
- The candidate makes the demo less stable.
|
||||
|
||||
## Evidence Strength
|
||||
|
||||
| Evidence | Strength | Use |
|
||||
| --- | --- | --- |
|
||||
| Model opinion | Weak | Proposal only |
|
||||
| Trace observation | Medium | Candidate rationale |
|
||||
| Human accepted outcome | Strong | Promotion candidate |
|
||||
| Human dismissed outcome | Strong | Suppression or rejection |
|
||||
| Automated verifier | Strong | Promotion or rejection |
|
||||
| Repeated accepted exact signature | Strong | Policy confidence increase |
|
||||
|
||||
## Versioning Rules
|
||||
|
||||
- Strategy versions are immutable after promotion or rejection.
|
||||
- There is one active version per `podId + kind`.
|
||||
- A rollback activates the previous version; it does not edit history.
|
||||
- Parent-child lineage must be preserved.
|
||||
|
||||
## Safety Rules
|
||||
|
||||
- Store summaries, not raw sensitive content.
|
||||
- Prefer deterministic checks over model judgment.
|
||||
- Use exact MongoDB recall before vector recall.
|
||||
- Ask for approval before changing code or data with external effects.
|
||||
- Treat hackathon demo stability as a hard constraint.
|
||||
|
||||
## Demo Honesty
|
||||
|
||||
Seeded strategy versions are acceptable when labeled as demo-backed. Do not claim
|
||||
a strategy was learned live unless a run and outcome actually created the
|
||||
promotion evidence.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# Agent Learning Prompt
|
||||
|
||||
Use this prompt for an agent responsible for improving PodMan's own behavior.
|
||||
|
||||
## Prompt
|
||||
|
||||
You are PodMan's agent-learning evaluator.
|
||||
|
||||
Your job is to inspect a completed agent run, identify one narrow improvement,
|
||||
define how to verify it, and decide whether to propose, promote, or reject a
|
||||
strategy change.
|
||||
|
||||
You must not claim improvement without evidence. You must not propose broad
|
||||
rewrites. Keep every change small, reversible, and tied to a run or outcome.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Current active strategy version.
|
||||
- Agent run summary.
|
||||
- Trace events.
|
||||
- Intervention outcome.
|
||||
- Verifier result.
|
||||
- Recent false positives or accepted events.
|
||||
- Current demo constraints.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Identify the target behavior.
|
||||
2. Identify the failure or success evidence.
|
||||
3. Decide whether a strategy change is warranted.
|
||||
4. Propose one narrow change.
|
||||
5. Define the verifier.
|
||||
6. Decide status: no change, candidate, promote, reject.
|
||||
7. Write a short explanation suitable for the Team memory activity stream.
|
||||
|
||||
## Output Format
|
||||
|
||||
```text
|
||||
Target
|
||||
- Strategy kind:
|
||||
- Active version:
|
||||
- Behavior under review:
|
||||
|
||||
Evidence
|
||||
- Run:
|
||||
- Outcome:
|
||||
- Verifier:
|
||||
- Confidence:
|
||||
|
||||
Decision
|
||||
- Status:
|
||||
- Proposed change:
|
||||
- Why this is narrow:
|
||||
- Risk:
|
||||
|
||||
Verifier
|
||||
- Metric:
|
||||
- Passing condition:
|
||||
- Failing condition:
|
||||
|
||||
Memory Write
|
||||
- Collection:
|
||||
- Record summary:
|
||||
- Graph/activity summary:
|
||||
```
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- Exact outcomes beat model opinion.
|
||||
- Rejected candidates stay in memory.
|
||||
- No raw screenshots or secrets.
|
||||
- No broad policy change from one weak signal.
|
||||
- No voice-first behavior.
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
# Agent Learning Spec
|
||||
|
||||
Status: draft
|
||||
Scope: how PodMan agents improve their own prompts, policies, detectors, and routing behavior
|
||||
Owner: agent learning / recursive self-improvement
|
||||
|
||||
## Purpose
|
||||
|
||||
Agent learning is the recursive self-improvement layer. It is not the same as
|
||||
team memory. Team memory learns about engineers and work. Agent learning learns
|
||||
which agent strategies produce better outcomes.
|
||||
|
||||
The demo claim:
|
||||
|
||||
1. PodMan tries a coordination strategy.
|
||||
2. The run is traced in MongoDB.
|
||||
3. A verifier or human outcome scores it.
|
||||
4. Gemini or another agent proposes a narrow strategy change.
|
||||
5. The new strategy is versioned.
|
||||
6. A later run uses the improved strategy and shows a better result.
|
||||
|
||||
## Core Objects
|
||||
|
||||
### Agent run
|
||||
|
||||
One attempt to execute a goal.
|
||||
|
||||
```text
|
||||
agent_runs
|
||||
runId
|
||||
podId
|
||||
goal
|
||||
trigger
|
||||
strategyVersionId
|
||||
status
|
||||
startedAt
|
||||
completedAt
|
||||
score
|
||||
verifierSummary
|
||||
inputRefs
|
||||
outputRefs
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
running, succeeded, failed, improved, regressed, abandoned
|
||||
```
|
||||
|
||||
### Trace event
|
||||
|
||||
Append-only event log for a run.
|
||||
|
||||
```text
|
||||
agent_trace_events
|
||||
runId
|
||||
podId
|
||||
step
|
||||
phase
|
||||
eventType
|
||||
inputSummary
|
||||
outputSummary
|
||||
toolName
|
||||
error
|
||||
metrics
|
||||
createdAt
|
||||
```
|
||||
|
||||
### Strategy version
|
||||
|
||||
Versioned prompt, detector rule, policy, verifier, or routing strategy.
|
||||
|
||||
```text
|
||||
strategy_versions
|
||||
strategyVersionId
|
||||
podId
|
||||
kind
|
||||
name
|
||||
parentVersionId
|
||||
status
|
||||
summary
|
||||
promptText
|
||||
policy
|
||||
verifier
|
||||
metrics
|
||||
createdAt
|
||||
promotedAt
|
||||
```
|
||||
|
||||
Allowed `kind` values:
|
||||
|
||||
```text
|
||||
prompt, policy, detector, verifier, routing
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
candidate, active, retired, rejected
|
||||
```
|
||||
|
||||
### Learning proposal
|
||||
|
||||
A candidate change before promotion.
|
||||
|
||||
```text
|
||||
learning_proposals
|
||||
proposalId
|
||||
podId
|
||||
sourceRunId
|
||||
targetKind
|
||||
parentVersionId
|
||||
proposedChange
|
||||
rationale
|
||||
verifierPlan
|
||||
status
|
||||
createdAt
|
||||
resolvedAt
|
||||
```
|
||||
|
||||
Allowed `status` values:
|
||||
|
||||
```text
|
||||
open, accepted, rejected, superseded
|
||||
```
|
||||
|
||||
## MongoDB Indexes
|
||||
|
||||
| Collection | Index | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `agent_runs` | `{ podId: 1, startedAt: -1 }` | Recent run history |
|
||||
| `agent_runs` | `{ podId: 1, strategyVersionId: 1 }` | Compare strategy performance |
|
||||
| `agent_trace_events` | `{ runId: 1, step: 1 }` | Reconstruct run |
|
||||
| `strategy_versions` | `{ podId: 1, kind: 1, status: 1 }` | Find active strategy |
|
||||
| `strategy_versions` | `{ podId: 1, createdAt: -1 }` | Version history |
|
||||
| `learning_proposals` | `{ podId: 1, status: 1 }` | Open candidate changes |
|
||||
|
||||
## Learning Loop
|
||||
|
||||
```text
|
||||
observe run -> score run -> propose change -> test candidate -> promote or reject
|
||||
```
|
||||
|
||||
Agent learning must always connect these records:
|
||||
|
||||
```text
|
||||
agent_run -> trace_events -> verifier result -> learning_proposal -> strategy_version
|
||||
```
|
||||
|
||||
## Verifier Contract
|
||||
|
||||
Every promoted strategy needs a verifier signal.
|
||||
|
||||
Allowed verifier types:
|
||||
|
||||
- Human accepted or dismissed outcome.
|
||||
- Test pass or fail result.
|
||||
- Reduced false positive rate.
|
||||
- Reduced intervention count with same or better accepted outcomes.
|
||||
- Faster successful run.
|
||||
- Better graph discovery precision.
|
||||
- Explicit demo operator approval.
|
||||
|
||||
Self-evaluation alone is not enough to promote a strategy.
|
||||
|
||||
## Relationship to Team Graph
|
||||
|
||||
Agent learning can appear in the Team memory graph as activity and loop status,
|
||||
but it should not clutter the main risk graph by default.
|
||||
|
||||
Graph discovery may show:
|
||||
|
||||
- `agent_run` activity in the stream.
|
||||
- `strategy_versions` count in the learning loop.
|
||||
- A selected-node detail saying a policy changed because a prior outcome was
|
||||
dismissed or accepted.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Every strategy change has a parent.
|
||||
- Every promoted strategy cites evidence.
|
||||
- Rejected strategies are retained with a reason.
|
||||
- Agent traces are append-only.
|
||||
- The system can answer: "What changed, why, and did it help?"
|
||||
|
||||
Reference in New Issue
Block a user