Initial commit: Agent Governance System Phase 8
Phase 8 Production Hardening with complete governance infrastructure: - Vault integration with tiered policies (T0-T4) - DragonflyDB state management - SQLite audit ledger - Pipeline DSL and templates - Promotion/revocation engine - Checkpoint system for session persistence - Health manager and circuit breaker for fault tolerance - GitHub/Slack integrations - Architectural test pipeline with bug watcher, suggestion engine, council review - Multi-agent chaos testing framework Test Results: - Governance tests: 68/68 passing - E2E workflow: 16/16 passing - Phase 2 Vault: 14/14 passing - Integration tests: 27/27 passing Coverage: 57.6% average across 12 phases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
profit
@@ -0,0 +1,740 @@
|
||||
# AI Agent Governance System Architecture
|
||||
|
||||
**Version:** 0.2.0
|
||||
**Status:** Active Development
|
||||
**Last Updated:** 2026-01-23
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Executive Summary](#executive-summary)
|
||||
2. [System Architecture](#system-architecture)
|
||||
3. [Core Components](#core-components)
|
||||
4. [Agent Taxonomy](#agent-taxonomy)
|
||||
5. [Runtime Governance](#runtime-governance)
|
||||
6. [Multi-Agent Coordination](#multi-agent-coordination)
|
||||
7. [Current Capabilities](#current-capabilities)
|
||||
8. [Engineering Focus Areas](#engineering-focus-areas)
|
||||
9. [Sample Implementations](#sample-implementations)
|
||||
10. [Future Potential](#future-potential)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This system implements a **governed AI agent framework** designed for safe, auditable, and scalable automation. The architecture enforces:
|
||||
|
||||
- **Trust-tiered access control** via HashiCorp Vault
|
||||
- **Real-time governance** via DragonflyDB
|
||||
- **Structured agent lifecycles** with mandatory phases
|
||||
- **Multi-agent coordination** with parallel execution and conditional spawning
|
||||
- **Complete audit trails** via SQLite ledger
|
||||
|
||||
The system prioritizes **legibility over magic** — every agent action must be explainable, reproducible, and auditable.
|
||||
|
||||
---
|
||||
|
||||
## System Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ GOVERNANCE LAYER │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
|
||||
│ │ HashiCorp │ │ DragonflyDB │ │ SQLite Ledger │ │
|
||||
│ │ Vault │ │ (Runtime) │ │ (Audit) │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ • Policies │ │ • State │ │ • agent_actions │ │
|
||||
│ │ • Secrets │ │ • Locks │ │ • agent_metrics │ │
|
||||
│ │ • AppRole Auth │ │ • Heartbeats │ │ • violations │ │
|
||||
│ │ • Token Leases │ │ • Errors │ │ • promotions │ │
|
||||
│ └─────────────────┘ │ • Blackboard │ └─────────────────────────────┘ │
|
||||
│ │ • Messages │ │
|
||||
│ └─────────────────┘ │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ ORCHESTRATION LAYER │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Multi-Agent Orchestrator │ │
|
||||
│ │ • Parallel agent execution • Spawn condition monitoring │ │
|
||||
│ │ • Performance metrics • Consensus coordination │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ AGENT LAYER │
|
||||
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ┌─────────────┐ │
|
||||
│ │ Agent ALPHA │ │ Agent BETA │ │ Agent GAMMA │ │ Governed │ │
|
||||
│ │ (Research) │ │ (Synthesis) │ │ (Mediator) │ │ LLM Agent │ │
|
||||
│ │ │ │ │ │ │ │ │ │
|
||||
│ │ Parallel │◄─┼─► Direct ─┼──┼─► Spawned │ │ Single │ │
|
||||
│ │ Execution │ │ Messages │ │ on │ │ Pipeline │ │
|
||||
│ └───────┬───────┘ └───────┬───────┘ │ Condition │ └──────┬──────┘ │
|
||||
│ │ │ └───────────────┘ │ │
|
||||
│ └──────────┬───────┴────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────▼──────┐ │
|
||||
│ │ Blackboard │ (Shared Memory) │
|
||||
│ │ • problem │ │
|
||||
│ │ • solutions│ │
|
||||
│ │ • progress │ │
|
||||
│ │ • consensus│ │
|
||||
│ └─────────────┘ │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ INFRASTRUCTURE LAYER │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
|
||||
│ │ OpenRouter │ │ Bun Runtime │ │ WireGuard VPN │ │
|
||||
│ │ (LLM API) │ │ (TypeScript) │ │ (Network) │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. HashiCorp Vault (Policy Engine)
|
||||
|
||||
**Purpose:** Centralized secrets management and trust-tier enforcement.
|
||||
|
||||
```
|
||||
Location: https://127.0.0.1:8200
|
||||
Storage: /opt/vault/data
|
||||
Policies: /opt/vault/policies/t{0-4}-*.hcl
|
||||
```
|
||||
|
||||
**Key Features:**
|
||||
- AppRole authentication for agents
|
||||
- Dynamic secret generation
|
||||
- Token TTLs based on trust tier
|
||||
- Immediate revocation capabilities
|
||||
|
||||
### 2. DragonflyDB (Runtime State)
|
||||
|
||||
**Purpose:** Real-time agent state, coordination, and governance signals.
|
||||
|
||||
```
|
||||
Location: redis://127.0.0.1:6379
|
||||
Credentials: vault:secret/data/services/dragonfly
|
||||
```
|
||||
|
||||
**Keyspace Design:**
|
||||
```
|
||||
agent:{id}:packet → Instruction packet (JSON)
|
||||
agent:{id}:state → Runtime state (JSON)
|
||||
agent:{id}:errors → Error counters (Hash)
|
||||
agent:{id}:heartbeat → Last seen (String + TTL)
|
||||
agent:{id}:lock → Execution lock (String + TTL)
|
||||
task:{id}:active_agent → Current agent
|
||||
task:{id}:artifacts → Artifact references (List)
|
||||
blackboard:{task}:* → Shared memory sections
|
||||
msg:{task}:* → Direct message channels
|
||||
revocations:ledger → Revocation history (List)
|
||||
handoff:{task}:latest → Handoff objects (JSON)
|
||||
```
|
||||
|
||||
### 3. SQLite Ledger (Audit Trail)
|
||||
|
||||
**Purpose:** Immutable record of all agent actions for compliance and replay.
|
||||
|
||||
```
|
||||
Location: /opt/agent-governance/ledger/governance.db
|
||||
```
|
||||
|
||||
**Schema:**
|
||||
```sql
|
||||
CREATE TABLE agent_actions (
|
||||
id INTEGER PRIMARY KEY,
|
||||
timestamp TEXT,
|
||||
agent_id TEXT,
|
||||
agent_version TEXT,
|
||||
tier INTEGER,
|
||||
action TEXT,
|
||||
decision TEXT,
|
||||
confidence REAL,
|
||||
success INTEGER,
|
||||
error_type TEXT,
|
||||
error_message TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE agent_metrics (
|
||||
agent_id TEXT PRIMARY KEY,
|
||||
current_tier INTEGER,
|
||||
total_runs INTEGER,
|
||||
compliant_runs INTEGER,
|
||||
consecutive_compliant INTEGER,
|
||||
last_active_at TEXT
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent Taxonomy
|
||||
|
||||
### Trust Tiers
|
||||
|
||||
| Tier | Name | Capabilities | Token TTL |
|
||||
|------|------|-------------|-----------|
|
||||
| 0 | Observer | Read docs, inventory, logs; Generate plans | 1h |
|
||||
| 1 | Operator | Sandbox SSH, basic Proxmox, Ansible check-mode | 30m |
|
||||
| 2 | Builder | Sandbox admin, create frameworks/modules | 30m |
|
||||
| 3 | Executor | Staging access, limited prod read, root-controlled | 15m |
|
||||
| 4 | Architect | Policy read, governance write, requires approval | 15m |
|
||||
|
||||
### Agent Lifecycle Phases
|
||||
|
||||
```
|
||||
BOOTSTRAP → PREFLIGHT → PLAN → EXECUTE → VERIFY → PACKAGE → REPORT → EXIT
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ │ │ │ │ │ └─ Release lock
|
||||
│ │ │ │ │ │ └─ Generate report
|
||||
│ │ │ │ │ └─ Collect artifacts
|
||||
│ │ │ │ └─ Verify results
|
||||
│ │ │ └─ Execute plan (if approved)
|
||||
│ │ └─ Generate plan artifact
|
||||
│ └─ Scope/dependency checks
|
||||
└─ Read revocations, load packet, acquire lock
|
||||
```
|
||||
|
||||
### Error Budget System
|
||||
|
||||
```json
|
||||
{
|
||||
"max_total_errors": 8,
|
||||
"max_same_error_repeats": 2,
|
||||
"max_procedure_violations": 1
|
||||
}
|
||||
```
|
||||
|
||||
**Automatic Revocation Triggers:**
|
||||
- `procedure_violations >= 1`
|
||||
- `same_error >= max_same_error_repeats`
|
||||
- `total_errors >= max_total_errors`
|
||||
- Missing required artifact after EXECUTE
|
||||
- Forbidden action detected
|
||||
|
||||
---
|
||||
|
||||
## Runtime Governance
|
||||
|
||||
### Instruction Packets
|
||||
|
||||
Every agent receives a structured instruction packet defining its mission:
|
||||
|
||||
```typescript
|
||||
interface InstructionPacket {
|
||||
agent_id: string;
|
||||
task_id: string;
|
||||
created_for: string;
|
||||
objective: string;
|
||||
deliverables: string[];
|
||||
constraints: {
|
||||
scope: string[];
|
||||
forbidden: string[];
|
||||
required_steps: string[];
|
||||
};
|
||||
success_criteria: string[];
|
||||
error_budget: ErrorBudget;
|
||||
escalation_rules: string[];
|
||||
created_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Handoff Objects
|
||||
|
||||
When an agent is revoked, it must create a handoff for the next agent:
|
||||
|
||||
```typescript
|
||||
interface HandoffObject {
|
||||
task_id: string;
|
||||
previous_agent_id: string;
|
||||
revoked: boolean;
|
||||
revocation_reason: { type: string; details: string };
|
||||
last_known_state: { phase: string; step: string };
|
||||
what_was_tried: string[];
|
||||
blocking_issue: string;
|
||||
required_next_actions: string[];
|
||||
constraints_reminder: string[];
|
||||
artifacts: string[];
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Multi-Agent Coordination
|
||||
|
||||
### Communication Patterns
|
||||
|
||||
**1. Direct Messaging (Point-to-Point)**
|
||||
```
|
||||
Agent ALPHA ──PROPOSAL──► Agent BETA
|
||||
Agent BETA ──FEEDBACK──► Agent ALPHA
|
||||
Agent GAMMA ──HANDOFF───► ALL
|
||||
```
|
||||
|
||||
**2. Blackboard (Shared Memory)**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ BLACKBOARD │
|
||||
├──────────┬──────────┬──────────┬────────┤
|
||||
│ problem │ solutions│ progress │consensus│
|
||||
├──────────┼──────────┼──────────┼────────┤
|
||||
│objective │proposal_1│eval_1 │votes │
|
||||
│analysis │proposal_2│eval_2 │final │
|
||||
│constraints│synthesis │gamma_res │ │
|
||||
└──────────┴──────────┴──────────┴────────┘
|
||||
```
|
||||
|
||||
### Conditional Agent Spawning
|
||||
|
||||
Agent GAMMA is spawned when thresholds are exceeded:
|
||||
|
||||
| Condition | Threshold | Description |
|
||||
|-----------|-----------|-------------|
|
||||
| STUCK | 30s | Agents inactive for 30+ seconds |
|
||||
| CONFLICT | 3 | 3+ unresolved proposal conflicts |
|
||||
| COMPLEXITY | 0.8 | Task complexity score > 0.8 |
|
||||
| SUCCESS | 1.0 | Task complete, validation needed |
|
||||
|
||||
### Consensus Mechanism
|
||||
|
||||
```typescript
|
||||
interface ConsensusVote {
|
||||
agent: AgentRole;
|
||||
proposal_id: string;
|
||||
vote: "ACCEPT" | "REJECT" | "ABSTAIN";
|
||||
reasoning: string;
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
// Consensus requires:
|
||||
// 1. All required agents have voted
|
||||
// 2. Accept votes > Reject votes
|
||||
// 3. No rejects from required agents
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Current Capabilities
|
||||
|
||||
### Implemented Features
|
||||
|
||||
| Feature | Status | Location |
|
||||
|---------|--------|----------|
|
||||
| Vault policy engine | ✅ Complete | `/opt/vault/policies/` |
|
||||
| Trust tier system (T0-T4) | ✅ Complete | Vault policies |
|
||||
| DragonflyDB runtime | ✅ Complete | `runtime/governance.py` |
|
||||
| SQLite audit ledger | ✅ Complete | `ledger/governance.db` |
|
||||
| Single-agent pipeline | ✅ Complete | `llm-planner-ts/governed-agent.ts` |
|
||||
| Multi-agent parallel execution | ✅ Complete | `multi-agent/orchestrator.ts` |
|
||||
| Blackboard shared memory | ✅ Complete | `multi-agent/coordination.ts` |
|
||||
| Direct messaging | ✅ Complete | `multi-agent/coordination.ts` |
|
||||
| Conditional spawning | ✅ Complete | `multi-agent/orchestrator.ts` |
|
||||
| Performance metrics | ✅ Complete | `multi-agent/coordination.ts` |
|
||||
| Error budget tracking | ✅ Complete | `GovernanceManager` class |
|
||||
| Revocation handling | ✅ Complete | `GovernanceManager` class |
|
||||
|
||||
### Performance Benchmarks
|
||||
|
||||
| Metric | Single Agent | Multi-Agent (3) |
|
||||
|--------|-------------|-----------------|
|
||||
| Avg. task duration | 60-120s | 45-90s |
|
||||
| Messages per task | N/A | 20-30 |
|
||||
| Blackboard ops | 5-10 | 40-60 |
|
||||
| LLM calls | 2-4 | 6-12 |
|
||||
|
||||
---
|
||||
|
||||
## Engineering Focus Areas
|
||||
|
||||
### 1. Pipeline Programming
|
||||
|
||||
**Goal:** Create composable, reusable agent pipelines.
|
||||
|
||||
**Current Work:**
|
||||
```typescript
|
||||
// Pipeline stages as composable functions
|
||||
type PipelineStage<T, U> = (input: T, context: GovernanceContext) => Promise<U>;
|
||||
|
||||
// Example pipeline composition
|
||||
const agentPipeline = compose(
|
||||
bootstrap,
|
||||
preflight,
|
||||
plan,
|
||||
execute,
|
||||
verify,
|
||||
package,
|
||||
report
|
||||
);
|
||||
```
|
||||
|
||||
**Planned Features:**
|
||||
- Stage-level error handling and retry
|
||||
- Pipeline branching based on conditions
|
||||
- Pipeline templates for common patterns
|
||||
- Hot-swappable stages for testing
|
||||
|
||||
### 2. Bun Integration
|
||||
|
||||
**Goal:** Leverage Bun's performance for agent execution.
|
||||
|
||||
**Current Implementation:**
|
||||
```typescript
|
||||
// File: agents/llm-planner-ts/governed-agent.ts
|
||||
import { $ } from "bun";
|
||||
import { Database } from "bun:sqlite";
|
||||
|
||||
// Shell commands via Bun
|
||||
const result = await $`curl -sk ...`.json();
|
||||
|
||||
// SQLite via Bun native
|
||||
const db = new Database("/opt/agent-governance/ledger/governance.db");
|
||||
```
|
||||
|
||||
**Advantages:**
|
||||
- 4x faster startup than Node.js
|
||||
- Native TypeScript execution
|
||||
- Built-in SQLite support
|
||||
- Shell command integration
|
||||
- Excellent npm compatibility
|
||||
|
||||
**Planned Enhancements:**
|
||||
- Bun's built-in test runner integration
|
||||
- Bun's native WebSocket for real-time coordination
|
||||
- Bun's worker threads for parallel LLM calls
|
||||
|
||||
### 3. Testing Framework
|
||||
|
||||
**Goal:** Enable long-term iteration with confidence.
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TESTING FRAMEWORK │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Unit Tests │ Integration Tests │ E2E Tests │
|
||||
│ ───────────── │ ───────────────── │ ────────── │
|
||||
│ • Agent methods │ • Vault + Agent │ • Full task │
|
||||
│ • Blackboard ops │ • Redis + Agent │ • Multi-agent│
|
||||
│ • Message parsing │ • LLM + Governance │ • Failure │
|
||||
│ • Error handling │ • Pipeline stages │ recovery │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Mock Infrastructure │ Test Scenarios │ Metrics │
|
||||
│ ───────────────── │ ────────────── │ ─────── │
|
||||
│ • MockVault │ • Happy path │ • Duration │
|
||||
│ • MockDragonfly │ • Error budget │ • Coverage │
|
||||
│ • MockLLM │ • Revocation │ • Flakiness │
|
||||
│ • MockBlackboard │ • Consensus fail │ • Regression │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Test Categories:**
|
||||
|
||||
```typescript
|
||||
// 1. Unit Tests - Isolated component testing
|
||||
describe("GovernanceManager", () => {
|
||||
it("should track error budget correctly", async () => {
|
||||
const gov = new MockGovernanceManager();
|
||||
await gov.recordError("agent-1", "LLM_ERROR", "timeout");
|
||||
const counts = await gov.getErrorCounts("agent-1");
|
||||
expect(counts.total_errors).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// 2. Integration Tests - Component interaction
|
||||
describe("Agent + Vault Integration", () => {
|
||||
it("should bootstrap with valid token", async () => {
|
||||
const agent = new GovernedAgent("test-agent");
|
||||
const [ok, msg] = await agent.bootstrap();
|
||||
expect(ok).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// 3. Scenario Tests - Full workflow validation
|
||||
describe("Multi-Agent Scenarios", () => {
|
||||
it("should spawn GAMMA on complexity threshold", async () => {
|
||||
const orchestrator = new TestOrchestrator();
|
||||
const metrics = await orchestrator.runTask(highComplexityTask);
|
||||
expect(metrics.gamma_spawned).toBe(true);
|
||||
expect(metrics.gamma_spawn_reason).toBe("COMPLEXITY");
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Mock Infrastructure:**
|
||||
|
||||
```typescript
|
||||
// MockLLM for deterministic testing
|
||||
class MockLLM {
|
||||
private responses: Map<string, string> = new Map();
|
||||
|
||||
setResponse(pattern: string, response: string) {
|
||||
this.responses.set(pattern, response);
|
||||
}
|
||||
|
||||
async complete(prompt: string): Promise<string> {
|
||||
for (const [pattern, response] of this.responses) {
|
||||
if (prompt.includes(pattern)) return response;
|
||||
}
|
||||
return '{"confidence": 0.5, "steps": []}';
|
||||
}
|
||||
}
|
||||
|
||||
// MockDragonfly for state testing
|
||||
class MockDragonfly {
|
||||
private store: Map<string, any> = new Map();
|
||||
|
||||
async set(key: string, value: any) { this.store.set(key, value); }
|
||||
async get(key: string) { return this.store.get(key); }
|
||||
async hSet(key: string, field: string, value: any) { /* ... */ }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sample Implementations
|
||||
|
||||
### Single Governed Agent (TypeScript/Bun)
|
||||
|
||||
```typescript
|
||||
// File: governed-agent.ts
|
||||
import { GovernanceManager } from "./coordination";
|
||||
|
||||
class GovernedAgent {
|
||||
private gov: GovernanceManager;
|
||||
private agentId: string;
|
||||
|
||||
async bootstrap(): Promise<[boolean, string]> {
|
||||
this.gov = new GovernanceManager();
|
||||
await this.gov.connect();
|
||||
|
||||
// Read revocation ledger
|
||||
const revocations = await this.gov.getRecentRevocations(50);
|
||||
for (const rev of revocations) {
|
||||
if (rev.agent_id === this.agentId) {
|
||||
return [false, "AGENT_PREVIOUSLY_REVOKED"];
|
||||
}
|
||||
}
|
||||
|
||||
// Load instruction packet
|
||||
const packet = await this.gov.getPacket(this.agentId);
|
||||
if (!packet) return [false, "NO_INSTRUCTION_PACKET"];
|
||||
|
||||
// Acquire lock
|
||||
if (!await this.gov.acquireLock(this.agentId)) {
|
||||
return [false, "CANNOT_ACQUIRE_LOCK"];
|
||||
}
|
||||
|
||||
return [true, "BOOTSTRAP_COMPLETE"];
|
||||
}
|
||||
|
||||
async transition(phase: string, step: string): Promise<boolean> {
|
||||
await this.gov.heartbeat(this.agentId);
|
||||
await this.gov.refreshLock(this.agentId);
|
||||
|
||||
const [ok, reason] = await this.gov.checkErrorBudget(this.agentId);
|
||||
if (!ok) {
|
||||
await this.gov.revokeAgent(this.agentId, "ERROR_BUDGET_EXCEEDED", reason);
|
||||
return false;
|
||||
}
|
||||
|
||||
await this.gov.setState({ phase, step, /* ... */ });
|
||||
return true;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Multi-Agent Orchestrator
|
||||
|
||||
```typescript
|
||||
// File: orchestrator.ts
|
||||
class MultiAgentOrchestrator {
|
||||
private alphaAgent: AgentAlpha;
|
||||
private betaAgent: AgentBeta;
|
||||
private gammaAgent?: AgentGamma;
|
||||
|
||||
async runTask(task: TaskDefinition): Promise<Metrics> {
|
||||
// Launch ALPHA and BETA in parallel
|
||||
const alphaPromise = this.alphaAgent.run(task);
|
||||
const betaPromise = this.betaAgent.run(task);
|
||||
|
||||
// Monitor spawn conditions
|
||||
this.monitorInterval = setInterval(() => {
|
||||
this.checkSpawnConditions();
|
||||
}, 2000);
|
||||
|
||||
await Promise.all([alphaPromise, betaPromise]);
|
||||
|
||||
// Spawn GAMMA if needed
|
||||
if (this.shouldSpawnGamma()) {
|
||||
await this.spawnGamma(spawnReason);
|
||||
await this.gammaAgent.run(task);
|
||||
}
|
||||
|
||||
return this.metrics.finalize();
|
||||
}
|
||||
|
||||
private async checkSpawnConditions() {
|
||||
// Check stuck, conflict, complexity thresholds
|
||||
const stuckAgents = await this.stateManager.detectStuckAgents(30);
|
||||
if (stuckAgents.length > 0) {
|
||||
await this.spawnController.updateCondition("STUCK", stuckAgents.length);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Future Potential
|
||||
|
||||
### Short-Term (Q1 2026)
|
||||
|
||||
1. **Pipeline DSL**
|
||||
```yaml
|
||||
pipeline:
|
||||
name: infrastructure-deploy
|
||||
stages:
|
||||
- plan:
|
||||
agent: planner
|
||||
artifacts: [terraform-plan]
|
||||
- review:
|
||||
type: human-gate
|
||||
timeout: 30m
|
||||
- execute:
|
||||
agent: executor
|
||||
requires: [plan]
|
||||
```
|
||||
|
||||
2. **Agent Templates**
|
||||
- Pre-configured agents for common tasks
|
||||
- Terraform specialist
|
||||
- Ansible specialist
|
||||
- Code review specialist
|
||||
|
||||
3. **Enhanced Testing**
|
||||
- Chaos testing for agent resilience
|
||||
- Load testing for multi-agent scaling
|
||||
- Regression test suite
|
||||
|
||||
### Medium-Term (Q2-Q3 2026)
|
||||
|
||||
1. **Hierarchical Agent Teams**
|
||||
```
|
||||
Team Lead Agent
|
||||
├── Research Team (3 agents)
|
||||
├── Implementation Team (2 agents)
|
||||
└── Review Team (2 agents)
|
||||
```
|
||||
|
||||
2. **Learning from History**
|
||||
- Analyze past task completions
|
||||
- Suggest optimizations
|
||||
- Predict failure patterns
|
||||
|
||||
3. **External Integrations**
|
||||
- GitHub PR automation
|
||||
- Slack notifications
|
||||
- PagerDuty escalations
|
||||
|
||||
### Long-Term (2027+)
|
||||
|
||||
1. **Self-Optimizing Pipelines**
|
||||
- Agents propose pipeline improvements
|
||||
- A/B testing of agent strategies
|
||||
- Automatic tier promotion
|
||||
|
||||
2. **Cross-System Orchestration**
|
||||
- Multiple infrastructure targets
|
||||
- Hybrid cloud coordination
|
||||
- Edge deployment agents
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
/opt/agent-governance/
|
||||
├── docs/
|
||||
│ └── ARCHITECTURE.md # This document
|
||||
├── ledger/
|
||||
│ └── governance.db # SQLite audit trail
|
||||
├── runtime/
|
||||
│ ├── governance.py # Python governance manager
|
||||
│ └── monitors.py # Monitor agents
|
||||
├── agents/
|
||||
│ ├── llm-planner/
|
||||
│ │ ├── agent.py # Python LLM agent
|
||||
│ │ └── governed_agent.py # Python governed agent
|
||||
│ ├── llm-planner-ts/
|
||||
│ │ ├── index.ts # Basic TypeScript agent
|
||||
│ │ └── governed-agent.ts # Full governed agent (Bun)
|
||||
│ └── multi-agent/
|
||||
│ ├── types.ts # Type definitions
|
||||
│ ├── coordination.ts # Blackboard, messaging, metrics
|
||||
│ ├── agents.ts # Alpha, Beta, Gamma agents
|
||||
│ └── orchestrator.ts # Multi-agent orchestrator
|
||||
└── /opt/vault/
|
||||
├── policies/
|
||||
│ ├── t0-observer.hcl
|
||||
│ ├── t1-operator.hcl
|
||||
│ ├── t2-builder.hcl
|
||||
│ ├── t3-executor.hcl
|
||||
│ └── t4-architect.hcl
|
||||
└── init-keys.json # Vault credentials (chmod 600)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running the System
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
# Vault must be running and unsealed
|
||||
docker ps | grep vault
|
||||
|
||||
# DragonflyDB must be running
|
||||
docker ps | grep dragonfly
|
||||
|
||||
# Bun must be installed
|
||||
~/.bun/bin/bun --version
|
||||
```
|
||||
|
||||
### Single Agent Test
|
||||
|
||||
```bash
|
||||
cd /opt/agent-governance/agents/llm-planner-ts
|
||||
~/.bun/bin/bun run governed-agent.ts \
|
||||
"agent-001" \
|
||||
"task-001" \
|
||||
"Design a microservices architecture"
|
||||
```
|
||||
|
||||
### Multi-Agent Test
|
||||
|
||||
```bash
|
||||
cd /opt/agent-governance/agents/multi-agent
|
||||
~/.bun/bin/bun run orchestrator.ts \
|
||||
"Design a distributed event-driven analytics platform" \
|
||||
--timeout 120
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new features:
|
||||
|
||||
1. **Follow the agent lifecycle** - All agents must implement the standard phases
|
||||
2. **Log to the ledger** - Every action must be auditable
|
||||
3. **Respect error budgets** - Check and track errors properly
|
||||
4. **Write tests** - Unit, integration, and scenario tests required
|
||||
5. **Document changes** - Update this architecture document
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Agent Foundation Document](/root/agents_foundation.md)
|
||||
- [Runtime Governance Spec](/root/agent_runtime_governance.md)
|
||||
- [Implementation Plan](/root/agent-taxonomy-implementation-plan.md)
|
||||
- [Vault Documentation](https://developer.hashicorp.com/vault/docs)
|
||||
- [Bun Documentation](https://bun.sh/docs)
|
||||
@@ -0,0 +1,400 @@
|
||||
# Context Management System
|
||||
|
||||
> Unified guide to checkpoints, STATUS files, and the external memory layer.
|
||||
|
||||
## Overview
|
||||
|
||||
The Agent Governance System uses three integrated components to maintain context across long-running sessions while staying within token limits:
|
||||
|
||||
| Component | Purpose | Token Impact |
|
||||
|-----------|---------|--------------|
|
||||
| **Checkpoints** | Capture full session state at points in time | ~3000 tokens |
|
||||
| **STATUS Files** | Track per-directory progress and tasks | ~50 tokens each |
|
||||
| **Memory Layer** | Store large outputs externally with summaries | Minimal (refs only) |
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ CONTEXT MANAGEMENT SYSTEM │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
||||
│ │ Checkpoint │◄───│ STATUS │◄───│ Memory Layer │ │
|
||||
│ │ │ │ Files │ │ │ │
|
||||
│ │ • Phase │ │ │ │ • Large outputs │ │
|
||||
│ │ • Tasks │ │ • Per-dir │ │ • Auto-chunked │ │
|
||||
│ │ • Deps │ │ • Progress │ │ • Summarized │ │
|
||||
│ │ • Memory refs│ │ • Tasks │ │ • Searchable │ │
|
||||
│ │ • Status sum │ │ • Memory ptr │ │ │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ └───────────────────┴──────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────▼─────────┐ │
|
||||
│ │ Token Window │ │
|
||||
│ │ (Summaries + │ │
|
||||
│ │ References) │ │
|
||||
│ └───────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### CLI Commands
|
||||
|
||||
```bash
|
||||
# Checkpoints
|
||||
checkpoint now --notes "Description" # Create checkpoint
|
||||
checkpoint load # Load latest
|
||||
checkpoint report # Combined status view
|
||||
checkpoint timeline # History view
|
||||
|
||||
# Status
|
||||
status sweep # Check all directories
|
||||
status sweep --fix # Create missing files
|
||||
status update <dir> --phase <phase> # Update directory
|
||||
status dashboard # Overview
|
||||
|
||||
# Memory
|
||||
memory log "content" # Store content
|
||||
memory log --stdin # Store from pipe
|
||||
memory fetch <id> # Retrieve full
|
||||
memory fetch <id> -s # Summary only
|
||||
memory list # List entries
|
||||
memory search "query" # Search
|
||||
memory stats # Statistics
|
||||
```
|
||||
|
||||
## Component Details
|
||||
|
||||
### 1. Checkpoints
|
||||
|
||||
Checkpoints capture complete session state at a point in time.
|
||||
|
||||
**What's Captured:**
|
||||
- Project phase and status
|
||||
- Completed phases list
|
||||
- Active tasks
|
||||
- Dependency states (Vault, DragonflyDB, Ledger)
|
||||
- Directory status summary (aggregated from STATUS files)
|
||||
- Memory references (links to stored content)
|
||||
- Variables and recent outputs
|
||||
|
||||
**When to Create:**
|
||||
- After completing significant work
|
||||
- Before context-heavy operations
|
||||
- At natural stopping points
|
||||
- Automatically when updating STATUS (creates delta)
|
||||
|
||||
**Commands:**
|
||||
```bash
|
||||
# Create with notes
|
||||
checkpoint now --notes "Completed pipeline consolidation"
|
||||
|
||||
# Load latest
|
||||
checkpoint load
|
||||
|
||||
# Load specific
|
||||
checkpoint load ckpt-20260123-123456-abcd
|
||||
|
||||
# View combined report
|
||||
checkpoint report
|
||||
|
||||
# View history
|
||||
checkpoint timeline --limit 10
|
||||
|
||||
# Compare checkpoints
|
||||
checkpoint diff --from ckpt-aaa --to ckpt-bbb
|
||||
```
|
||||
|
||||
**Data Location:** `/opt/agent-governance/checkpoint/storage/`
|
||||
|
||||
---
|
||||
|
||||
### 2. STATUS Files
|
||||
|
||||
Every directory has `README.md` and `STATUS.md` for local context.
|
||||
|
||||
**README.md Contains:**
|
||||
- Directory purpose
|
||||
- Key files and their roles
|
||||
- Interfaces/APIs
|
||||
- Current status badge
|
||||
- Link to parent architecture
|
||||
|
||||
**STATUS.md Contains:**
|
||||
- Current phase (complete/in_progress/blocked/needs_review)
|
||||
- Task checklist
|
||||
- Dependencies
|
||||
- Issues/blockers
|
||||
- Activity log with timestamps
|
||||
|
||||
**Phase Values:**
|
||||
| Phase | Icon | Meaning |
|
||||
|-------|------|---------|
|
||||
| `complete` | ✅ | Work finished |
|
||||
| `in_progress` | 🚧 | Active development |
|
||||
| `blocked` | ❗ | Waiting on dependencies |
|
||||
| `needs_review` | ⚠️ | Requires attention |
|
||||
| `not_started` | ⬜ | No work begun |
|
||||
|
||||
**Commands:**
|
||||
```bash
|
||||
# Check all directories
|
||||
status sweep
|
||||
|
||||
# Auto-create missing files
|
||||
status sweep --fix
|
||||
|
||||
# Update a directory
|
||||
status update ./pipeline --phase complete --task "Unified with core.py"
|
||||
|
||||
# View dashboard
|
||||
status dashboard
|
||||
|
||||
# Initialize new directory
|
||||
status init ./new-module
|
||||
```
|
||||
|
||||
**Integration with Checkpoints:**
|
||||
- `status update` automatically creates a checkpoint delta
|
||||
- Checkpoints aggregate status from all directories
|
||||
- Use `--no-checkpoint` to skip delta creation
|
||||
|
||||
---
|
||||
|
||||
### 3. Memory Layer
|
||||
|
||||
External storage for large content that would overwhelm the token window.
|
||||
|
||||
**Token Thresholds:**
|
||||
| Size | Strategy |
|
||||
|------|----------|
|
||||
| < 500 tokens | Inline in database |
|
||||
| 500-4000 tokens | File + summary |
|
||||
| > 4000 tokens | Auto-chunked + summary |
|
||||
|
||||
**Entry Types:**
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `output` | Command outputs, logs |
|
||||
| `transcript` | Conversation logs |
|
||||
| `context` | Saved state |
|
||||
| `chunk` | Part of larger entry |
|
||||
|
||||
**Commands:**
|
||||
```bash
|
||||
# Store content
|
||||
memory log "Test output here"
|
||||
memory log --file results.txt --tag "tests"
|
||||
pytest 2>&1 | memory log --stdin --tag "pytest"
|
||||
|
||||
# Retrieve
|
||||
memory fetch mem-xxx # Full content
|
||||
memory fetch mem-xxx -s # Summary only
|
||||
memory fetch mem-xxx --chunk 2 # Specific chunk
|
||||
|
||||
# Browse
|
||||
memory list --type output --limit 10
|
||||
memory search "error"
|
||||
memory refs --checkpoint ckpt-xxx
|
||||
|
||||
# Maintain
|
||||
memory stats
|
||||
memory prune --keep-days 7
|
||||
```
|
||||
|
||||
**Data Location:**
|
||||
- Metadata: `/opt/agent-governance/memory/memory.db`
|
||||
- Chunks: `/opt/agent-governance/memory/chunks/`
|
||||
|
||||
---
|
||||
|
||||
## Integration Patterns
|
||||
|
||||
### Pattern 1: Store Large Output
|
||||
|
||||
When a command produces large output:
|
||||
|
||||
```bash
|
||||
# Instead of printing 10000 lines to chat
|
||||
pytest tests/ 2>&1 | memory log --stdin --tag "pytest" --directory ./tests
|
||||
|
||||
# Report summary in chat
|
||||
echo "Tests complete. Results stored in mem-xxx (8500 tokens)."
|
||||
echo "Summary: 156 passed, 2 failed (test_auth, test_db)"
|
||||
```
|
||||
|
||||
### Pattern 2: Link Memory to Status
|
||||
|
||||
In STATUS.md, reference stored content:
|
||||
|
||||
```markdown
|
||||
## Context References
|
||||
|
||||
- Full test output: `mem-20260123-100000-abcd` (8500 tokens)
|
||||
- Build log: `mem-20260123-100001-efgh` (3200 tokens)
|
||||
|
||||
Retrieve with: `memory fetch <id> --summary-only`
|
||||
```
|
||||
|
||||
### Pattern 3: Checkpoint with Memory
|
||||
|
||||
Checkpoints automatically include memory references:
|
||||
|
||||
```bash
|
||||
# Create checkpoint after storing output
|
||||
memory log --file build.log --tag "build"
|
||||
checkpoint now --notes "Build complete"
|
||||
|
||||
# Later, checkpoint report shows memory refs
|
||||
checkpoint report
|
||||
# [MEMORY]
|
||||
# 3 entries, 12000 total tokens
|
||||
```
|
||||
|
||||
### Pattern 4: Recovery Workflow
|
||||
|
||||
After context reset:
|
||||
|
||||
```bash
|
||||
# 1. Load checkpoint
|
||||
checkpoint load
|
||||
# Shows: phase, deps, status summary, memory refs
|
||||
|
||||
# 2. Check directory status
|
||||
checkpoint report
|
||||
# Shows: active directories, completion %
|
||||
|
||||
# 3. Get memory summaries
|
||||
memory list --limit 5
|
||||
memory fetch mem-xxx -s # Summary only
|
||||
|
||||
# 4. Fetch specific chunk if needed
|
||||
memory fetch mem-xxx --chunk 2
|
||||
|
||||
# 5. Resume work
|
||||
status update ./current-dir --task "Resuming: fix auth bug"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Token Efficiency
|
||||
|
||||
### Before Memory Layer
|
||||
```
|
||||
Token Window:
|
||||
├── System prompt: 2000 tokens
|
||||
├── Conversation: 5000 tokens
|
||||
├── Large output: 15000 tokens ← Problem!
|
||||
└── Total: 22000 tokens (overflow risk)
|
||||
```
|
||||
|
||||
### With Memory Layer
|
||||
```
|
||||
Token Window:
|
||||
├── System prompt: 2000 tokens
|
||||
├── Conversation: 5000 tokens
|
||||
├── Memory reference: 50 tokens ← "mem-xxx: test results (15000 tokens)"
|
||||
├── Summary: 200 tokens ← "156 passed, 2 failed..."
|
||||
└── Total: 7250 tokens (safe)
|
||||
|
||||
External Memory:
|
||||
└── Full content: 15000 tokens (retrievable on demand)
|
||||
```
|
||||
|
||||
### Token Budget Guidelines
|
||||
|
||||
| Content | Recommended Action |
|
||||
|---------|-------------------|
|
||||
| < 500 tokens | Include inline |
|
||||
| 500-2000 tokens | Consider storing, include summary |
|
||||
| > 2000 tokens | Always store, use summary + refs |
|
||||
| > 10000 tokens | Store chunked, fetch chunks as needed |
|
||||
|
||||
---
|
||||
|
||||
## File Locations
|
||||
|
||||
```
|
||||
/opt/agent-governance/
|
||||
├── checkpoint/
|
||||
│ ├── checkpoint.py # Checkpoint manager
|
||||
│ ├── storage/ # Checkpoint JSON files
|
||||
│ └── templates/ # Summary templates
|
||||
├── memory/
|
||||
│ ├── memory.py # Memory manager
|
||||
│ ├── memory.db # SQLite metadata
|
||||
│ ├── chunks/ # Compressed content files
|
||||
│ └── summaries/ # Generated summaries
|
||||
├── bin/
|
||||
│ ├── checkpoint # CLI wrapper
|
||||
│ ├── status # CLI wrapper
|
||||
│ └── memory # CLI wrapper
|
||||
├── docs/
|
||||
│ ├── CONTEXT_MANAGEMENT.md # This file
|
||||
│ ├── MEMORY_LAYER.md # Memory details
|
||||
│ └── STATUS_PROTOCOL.md # Status details
|
||||
└── */
|
||||
├── README.md # Directory overview
|
||||
└── STATUS.md # Directory status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### For Checkpoints
|
||||
1. Create checkpoints at natural stopping points
|
||||
2. Include descriptive notes
|
||||
3. Use `checkpoint report` to verify state
|
||||
4. Review timeline before resuming work
|
||||
|
||||
### For STATUS Files
|
||||
1. Update status when entering/leaving directories
|
||||
2. Keep task lists current
|
||||
3. Document blockers immediately
|
||||
4. Link to relevant memory entries
|
||||
|
||||
### For Memory
|
||||
1. Store proactively (before context fills)
|
||||
2. Use meaningful tags for searchability
|
||||
3. Fetch summaries first, full content only when needed
|
||||
4. Prune regularly to manage storage
|
||||
|
||||
### For Recovery
|
||||
1. Always start with `checkpoint load`
|
||||
2. Use `checkpoint report` for overview
|
||||
3. Check `status dashboard` for active work
|
||||
4. Fetch memory summaries before full content
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Checkpoint not found"
|
||||
```bash
|
||||
checkpoint list # See available checkpoints
|
||||
```
|
||||
|
||||
### "Memory entry not found"
|
||||
```bash
|
||||
memory list --limit 50 # Browse recent entries
|
||||
memory search "keyword" # Search by content
|
||||
```
|
||||
|
||||
### "STATUS.md missing"
|
||||
```bash
|
||||
status sweep --fix # Auto-create missing files
|
||||
```
|
||||
|
||||
### "Token window overflow"
|
||||
```bash
|
||||
# Store large content externally
|
||||
memory log --stdin < large-output.txt
|
||||
# Use summary instead of full content
|
||||
memory fetch mem-xxx -s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Part of the [Agent Governance System](/opt/agent-governance/docs/ARCHITECTURE.md)*
|
||||
@@ -0,0 +1,429 @@
|
||||
# Engineering Quick Reference Guide
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
# Check Vault is running
|
||||
curl -sk https://127.0.0.1:8200/v1/sys/health | jq .
|
||||
|
||||
# Check DragonflyDB is running
|
||||
redis-cli -p 6379 -a $(cat /opt/vault/init-keys.json | jq -r .root_token) PING
|
||||
|
||||
# Check Bun is installed
|
||||
~/.bun/bin/bun --version # Should be 1.3.6+
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
/opt/agent-governance/
|
||||
├── docs/ # Documentation
|
||||
│ ├── ARCHITECTURE.md # System architecture
|
||||
│ └── ENGINEERING_GUIDE.md # This file
|
||||
├── ledger/ # Audit trail
|
||||
│ └── governance.db # SQLite database
|
||||
├── runtime/ # Python governance
|
||||
│ ├── governance.py # Core governance manager
|
||||
│ └── monitors.py # Monitor agents
|
||||
├── agents/
|
||||
│ ├── llm-planner/ # Python agents
|
||||
│ ├── llm-planner-ts/ # TypeScript single-agent
|
||||
│ └── multi-agent/ # Multi-agent system
|
||||
└── testing/
|
||||
└── framework.ts # Test framework
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Commands
|
||||
|
||||
### Run Single Agent
|
||||
|
||||
```bash
|
||||
cd /opt/agent-governance/agents/llm-planner-ts
|
||||
~/.bun/bin/bun run governed-agent.ts <agent_id> <task_id> "<objective>"
|
||||
|
||||
# Example
|
||||
~/.bun/bin/bun run governed-agent.ts \
|
||||
"agent-001" \
|
||||
"task-001" \
|
||||
"Design a caching strategy for the API"
|
||||
```
|
||||
|
||||
### Run Multi-Agent System
|
||||
|
||||
```bash
|
||||
cd /opt/agent-governance/agents/multi-agent
|
||||
~/.bun/bin/bun run orchestrator.ts "<objective>" --timeout 120
|
||||
|
||||
# Example
|
||||
~/.bun/bin/bun run orchestrator.ts \
|
||||
"Design a distributed event system" \
|
||||
--timeout 90
|
||||
```
|
||||
|
||||
### Run Tests
|
||||
|
||||
```bash
|
||||
cd /opt/agent-governance/testing
|
||||
~/.bun/bin/bun run framework.ts
|
||||
```
|
||||
|
||||
### Check Agent State (DragonflyDB)
|
||||
|
||||
```bash
|
||||
# Connect to DragonflyDB
|
||||
redis-cli -p 6379
|
||||
|
||||
# Check agent state
|
||||
GET agent:<agent_id>:state
|
||||
HGETALL agent:<agent_id>:errors
|
||||
GET agent:<agent_id>:packet
|
||||
|
||||
# Check blackboard
|
||||
HGETALL blackboard:<task_id>:solutions
|
||||
HGETALL blackboard:<task_id>:progress
|
||||
|
||||
# Check metrics
|
||||
HGETALL metrics:<task_id>
|
||||
```
|
||||
|
||||
### Check Vault Secrets
|
||||
|
||||
```bash
|
||||
# Get root token
|
||||
export VAULT_TOKEN=$(cat /opt/vault/init-keys.json | jq -r .root_token)
|
||||
|
||||
# List secrets
|
||||
curl -sk -H "X-Vault-Token: $VAULT_TOKEN" \
|
||||
https://127.0.0.1:8200/v1/secret/metadata | jq .
|
||||
|
||||
# Read a secret
|
||||
curl -sk -H "X-Vault-Token: $VAULT_TOKEN" \
|
||||
https://127.0.0.1:8200/v1/secret/data/api-keys/openrouter | jq .data.data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Development Patterns
|
||||
|
||||
### Adding a New Agent
|
||||
|
||||
```typescript
|
||||
// 1. Extend BaseAgent or implement from scratch
|
||||
import { BaseAgent } from "./agents";
|
||||
|
||||
class MyAgent extends BaseAgent {
|
||||
constructor(taskId: string, /* dependencies */) {
|
||||
super("MY_ROLE", taskId, /* ... */);
|
||||
}
|
||||
|
||||
// Handle incoming messages
|
||||
protected async handleMessage(msg: AgentMessage): Promise<void> {
|
||||
await super.handleMessage(msg);
|
||||
switch (msg.type) {
|
||||
case "PROPOSAL": await this.handleProposal(msg); break;
|
||||
case "QUERY": await this.handleQuery(msg); break;
|
||||
}
|
||||
}
|
||||
|
||||
// Main execution loop
|
||||
async run(task: TaskDefinition): Promise<void> {
|
||||
await this.updateState({ status: "WORKING" });
|
||||
|
||||
// Phase 1: Initialize
|
||||
// Phase 2: Process
|
||||
// Phase 3: Complete
|
||||
|
||||
await this.updateState({ status: "COMPLETED" });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Writing to the Blackboard
|
||||
|
||||
```typescript
|
||||
// Write a solution proposal
|
||||
await this.writeToBlackboard("solutions", proposalId, {
|
||||
name: "My Solution",
|
||||
approach: "...",
|
||||
confidence: 0.8,
|
||||
});
|
||||
|
||||
// Read problem analysis
|
||||
const analysis = await this.readFromBlackboard("problem", "analysis");
|
||||
|
||||
// Record a vote
|
||||
await this.blackboard.recordVote({
|
||||
agent: this.role,
|
||||
proposal_id: proposalId,
|
||||
vote: "ACCEPT",
|
||||
reasoning: "...",
|
||||
timestamp: now(),
|
||||
});
|
||||
```
|
||||
|
||||
### Sending Direct Messages
|
||||
|
||||
```typescript
|
||||
// Send proposal to specific agent
|
||||
await this.sendMessage("BETA", "PROPOSAL", {
|
||||
proposal_id: id,
|
||||
proposal: data,
|
||||
});
|
||||
|
||||
// Broadcast to all agents
|
||||
await this.sendMessage("ALL", "SYNC", {
|
||||
event: "STATUS_UPDATE",
|
||||
progress: 0.5,
|
||||
});
|
||||
|
||||
// Respond to a query
|
||||
await this.sendMessage(msg.from as AgentRole, "RESPONSE", {
|
||||
answer: result,
|
||||
}, msg.id); // correlation_id
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
```typescript
|
||||
// Record an error
|
||||
const counts = await this.gov.recordError(
|
||||
this.agentId,
|
||||
"LLM_ERROR",
|
||||
error.message
|
||||
);
|
||||
|
||||
// Check error budget
|
||||
const [ok, reason] = await this.gov.checkErrorBudget(this.agentId);
|
||||
if (!ok) {
|
||||
await this.gov.revokeAgent(
|
||||
this.agentId,
|
||||
"ERROR_BUDGET_EXCEEDED",
|
||||
reason!
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Record a procedure violation
|
||||
await this.gov.recordViolation(this.agentId, "EXECUTE_WITHOUT_PLAN");
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Patterns
|
||||
|
||||
### Unit Test with Mocks
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect } from "bun:test";
|
||||
import { MockDragonfly, MockLLM, createTestContext } from "./framework";
|
||||
|
||||
describe("MyAgent", () => {
|
||||
it("should handle proposals correctly", async () => {
|
||||
const ctx = createTestContext();
|
||||
|
||||
// Set up mock LLM response
|
||||
ctx.mockLLM.setResponse("evaluate", JSON.stringify({
|
||||
score: 0.85,
|
||||
accepted: true,
|
||||
}));
|
||||
|
||||
// Set up state in mock DragonflyDB
|
||||
await ctx.mockDragonfly.set(
|
||||
`agent:${ctx.agentId}:packet`,
|
||||
JSON.stringify(/* packet */)
|
||||
);
|
||||
|
||||
// Run test
|
||||
// ... agent execution ...
|
||||
|
||||
// Assert
|
||||
const state = await ctx.mockDragonfly.get(`agent:${ctx.agentId}:state`);
|
||||
expect(JSON.parse(state).status).toBe("COMPLETED");
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Integration Test
|
||||
|
||||
```typescript
|
||||
describe("Agent + Vault Integration", () => {
|
||||
it("should authenticate with Vault", async () => {
|
||||
// Use real Vault but test environment
|
||||
const agent = new GovernedAgent("test-agent");
|
||||
|
||||
const [ok, msg] = await agent.bootstrap();
|
||||
expect(ok).toBe(true);
|
||||
|
||||
await agent.cleanup();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Scenario Test
|
||||
|
||||
```typescript
|
||||
const scenario: TestScenario = {
|
||||
name: "Multi-Agent Consensus",
|
||||
description: "ALPHA and BETA reach consensus",
|
||||
setup: async () => {
|
||||
// Pre-populate blackboard
|
||||
},
|
||||
execute: async (ctx) => {
|
||||
const orchestrator = new MultiAgentOrchestrator();
|
||||
await orchestrator.initialize();
|
||||
await orchestrator.runTask(task);
|
||||
},
|
||||
assertions: async (ctx) => {
|
||||
const metrics = await getMetrics(ctx.taskId);
|
||||
expect(metrics.final_consensus).toBe(true);
|
||||
},
|
||||
cleanup: async () => {
|
||||
// Clean up DragonflyDB keys
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Files Reference
|
||||
|
||||
### Governance Manager (TypeScript)
|
||||
|
||||
```
|
||||
File: agents/llm-planner-ts/governed-agent.ts
|
||||
Class: GovernanceManager
|
||||
|
||||
Key Methods:
|
||||
- connect() / disconnect()
|
||||
- createPacket(packet)
|
||||
- getPacket(agentId)
|
||||
- setState(state) / getState(agentId)
|
||||
- acquireLock(agentId) / releaseLock(agentId)
|
||||
- heartbeat(agentId)
|
||||
- recordError(agentId, type, message)
|
||||
- checkErrorBudget(agentId)
|
||||
- revokeAgent(agentId, reason, details)
|
||||
- registerArtifact(taskId, type, reference)
|
||||
```
|
||||
|
||||
### Multi-Agent Coordination
|
||||
|
||||
```
|
||||
File: agents/multi-agent/coordination.ts
|
||||
|
||||
Classes:
|
||||
- Blackboard # Shared memory
|
||||
- MessageBus # Direct messaging
|
||||
- AgentStateManager # State tracking
|
||||
- SpawnController # Conditional spawning
|
||||
- MetricsCollector # Performance metrics
|
||||
```
|
||||
|
||||
### Agent Implementations
|
||||
|
||||
```
|
||||
File: agents/multi-agent/agents.ts
|
||||
|
||||
Classes:
|
||||
- BaseAgent # Abstract base
|
||||
- AgentAlpha # Research/Analysis
|
||||
- AgentBeta # Implementation/Synthesis
|
||||
- AgentGamma # Mediator (conditionally spawned)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging Tips
|
||||
|
||||
### Enable Verbose Logging
|
||||
|
||||
```typescript
|
||||
// Add to agent constructor
|
||||
this.log = (msg: string) => {
|
||||
const elapsed = ((Date.now() - this.startTime) / 1000).toFixed(3);
|
||||
console.log(`[${elapsed}s] [${this.role}] [DEBUG] ${msg}`);
|
||||
console.log(` State: ${JSON.stringify(this.state)}`);
|
||||
};
|
||||
```
|
||||
|
||||
### Inspect DragonflyDB State
|
||||
|
||||
```bash
|
||||
# Watch all keys in real-time
|
||||
redis-cli -p 6379 MONITOR
|
||||
|
||||
# Dump all agent keys
|
||||
redis-cli -p 6379 KEYS "agent:*"
|
||||
redis-cli -p 6379 KEYS "blackboard:*"
|
||||
redis-cli -p 6379 KEYS "msg:*"
|
||||
```
|
||||
|
||||
### Check Ledger
|
||||
|
||||
```bash
|
||||
sqlite3 /opt/agent-governance/ledger/governance.db \
|
||||
"SELECT * FROM agent_actions ORDER BY timestamp DESC LIMIT 10"
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `CANNOT_ACQUIRE_LOCK` | Previous agent didn't release | `DEL agent:<id>:lock` |
|
||||
| `NO_INSTRUCTION_PACKET` | Packet not created | Create packet before running |
|
||||
| `AGENT_PREVIOUSLY_REVOKED` | Agent in revocation ledger | Use new agent ID |
|
||||
| `LLM_ERROR` | OpenRouter API issue | Check API key in Vault |
|
||||
| JSON parse error | LLM returned markdown | Improve JSON extraction |
|
||||
|
||||
---
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### LLM Call Optimization
|
||||
|
||||
```typescript
|
||||
// Use lower max_tokens for simple tasks
|
||||
const response = await this.llm.chat.completions.create({
|
||||
model: this.model,
|
||||
messages: [...],
|
||||
max_tokens: task.complexity === "low" ? 1000 : 4000,
|
||||
temperature: 0.3, // Lower = more deterministic
|
||||
});
|
||||
```
|
||||
|
||||
### Reduce Blackboard Overhead
|
||||
|
||||
```typescript
|
||||
// Batch writes
|
||||
const entries = [...];
|
||||
for (const entry of entries) {
|
||||
await this.blackboard.write(section, key, value, this.role);
|
||||
}
|
||||
|
||||
// Read section once instead of multiple reads
|
||||
const solutions = await this.blackboard.readSection("solutions");
|
||||
// Instead of multiple readFromBlackboard calls
|
||||
```
|
||||
|
||||
### Monitor Interval Tuning
|
||||
|
||||
```typescript
|
||||
// Adjust based on task duration
|
||||
const monitorIntervalMs = task.timeout_seconds > 60 ? 5000 : 2000;
|
||||
this.monitorInterval = setInterval(() => {
|
||||
this.monitorConditions();
|
||||
}, monitorIntervalMs);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Useful Links
|
||||
|
||||
- [Bun Documentation](https://bun.sh/docs)
|
||||
- [Redis Commands](https://redis.io/commands)
|
||||
- [Vault API](https://developer.hashicorp.com/vault/api-docs)
|
||||
- [OpenRouter API](https://openrouter.ai/docs)
|
||||
@@ -0,0 +1,282 @@
|
||||
# External Memory Layer
|
||||
|
||||
> Token-efficient persistent storage for large outputs, transcripts, and context.
|
||||
|
||||
## Overview
|
||||
|
||||
The External Memory Layer provides a system for storing and retrieving large content outside the token window. Instead of including full outputs in prompts, agents store content in memory and work with summaries + retrieval references.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Token Window │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ Checkpoint │ │ STATUS │ │ Memory References │ │
|
||||
│ │ Summary │ │ Summaries │ │ [ID] summary (tokens) │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ External Memory Layer │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ SQLite │ │ Chunks │ │ DragonflyDB │ │
|
||||
│ │ (metadata) │ │ (files) │ │ (hot cache, opt) │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Token Thresholds
|
||||
|
||||
| Content Size | Storage Strategy |
|
||||
|-------------|------------------|
|
||||
| < 500 tokens | Stored inline in database |
|
||||
| 500-4000 tokens | Stored in compressed file + summary |
|
||||
| > 4000 tokens | Auto-chunked (multiple files) + parent summary |
|
||||
|
||||
## CLI Commands
|
||||
|
||||
### Store Content
|
||||
|
||||
```bash
|
||||
# Store inline content
|
||||
memory log "Test results: all 42 tests passed"
|
||||
|
||||
# Store from file
|
||||
memory log --file /path/to/large-output.txt --tag "test-results"
|
||||
|
||||
# Store from stdin (common pattern)
|
||||
pytest tests/ 2>&1 | memory log --stdin --tag "pytest" --checkpoint ckpt-xxx
|
||||
|
||||
# Store with directory linkage
|
||||
memory log --file output.txt --directory ./pipeline --tag "validation"
|
||||
```
|
||||
|
||||
### Retrieve Content
|
||||
|
||||
```bash
|
||||
# Get full entry (includes content if small, or loads from file)
|
||||
memory fetch mem-20260123-123456-abcd1234
|
||||
|
||||
# Get just the summary (token-efficient)
|
||||
memory fetch mem-20260123-123456-abcd1234 --summary-only
|
||||
|
||||
# Get specific chunk (for large entries)
|
||||
memory fetch mem-20260123-123456-abcd1234 --chunk 2
|
||||
```
|
||||
|
||||
### List and Search
|
||||
|
||||
```bash
|
||||
# List recent entries
|
||||
memory list --limit 10
|
||||
|
||||
# Filter by type
|
||||
memory list --type output --limit 20
|
||||
|
||||
# Filter by directory
|
||||
memory list --directory ./tests
|
||||
|
||||
# Search content
|
||||
memory search "error" --limit 5
|
||||
```
|
||||
|
||||
### Memory References
|
||||
|
||||
```bash
|
||||
# Get references linked to a checkpoint
|
||||
memory refs --checkpoint ckpt-20260123-123456
|
||||
|
||||
# Get references for a directory
|
||||
memory refs --directory ./pipeline
|
||||
```
|
||||
|
||||
### Maintenance
|
||||
|
||||
```bash
|
||||
# Show statistics
|
||||
memory stats
|
||||
|
||||
# Prune old entries
|
||||
memory prune --keep-days 7 --keep-entries 500
|
||||
```
|
||||
|
||||
## Integration with Checkpoint
|
||||
|
||||
When `checkpoint now` runs:
|
||||
1. Collects references to recent memory entries
|
||||
2. Includes memory summary (counts, total tokens)
|
||||
3. Stores lightweight refs instead of full content
|
||||
|
||||
```bash
|
||||
# Checkpoint includes memory refs
|
||||
checkpoint now --notes "After test run"
|
||||
|
||||
# View memory info in checkpoint report
|
||||
checkpoint report
|
||||
# Shows:
|
||||
# [MEMORY REFERENCES]
|
||||
# mem-xxx: pytest results (12000 tokens)
|
||||
# mem-yyy: build output (3200 tokens)
|
||||
```
|
||||
|
||||
## Integration with STATUS
|
||||
|
||||
STATUS.md files can include memory pointers for detailed context:
|
||||
|
||||
```markdown
|
||||
## Context References
|
||||
|
||||
- Test Results: `mem-20260123-123456-abcd` (12000 tokens)
|
||||
- Build Log: `mem-20260123-123457-efgh` (3200 tokens)
|
||||
|
||||
Use `memory fetch <id>` to retrieve full content.
|
||||
```
|
||||
|
||||
## Agent Guidelines
|
||||
|
||||
### When to Use Memory
|
||||
|
||||
1. **Large outputs** - If output would exceed ~500 tokens, store it
|
||||
2. **Test results** - Store full test output, reference summary
|
||||
3. **Build logs** - Store full log, include just errors inline
|
||||
4. **Generated code** - Store in memory, reference in plan
|
||||
|
||||
### Pattern: Store and Reference
|
||||
|
||||
```python
|
||||
# Instead of including large output in response:
|
||||
# "Here are all 500 lines of test output: ..."
|
||||
|
||||
# Do this:
|
||||
# 1. Store the output
|
||||
result = subprocess.run(["pytest"], capture_output=True)
|
||||
# memory log --stdin <<< result.stdout
|
||||
|
||||
# 2. Reference it
|
||||
# "Test completed. Full output stored in mem-xxx (2400 tokens).
|
||||
# Summary: 42 passed, 3 failed. Failed tests: test_auth, test_db, test_cache"
|
||||
```
|
||||
|
||||
### Pattern: Chunk Retrieval
|
||||
|
||||
For very large content (>4000 tokens), memory auto-chunks:
|
||||
|
||||
```bash
|
||||
# Store 50KB log file
|
||||
memory log --file build.log
|
||||
# Output: ID: mem-xxx, Chunks: 12
|
||||
|
||||
# Retrieve specific chunk
|
||||
memory fetch mem-xxx --chunk 5
|
||||
|
||||
# Or get just the summary
|
||||
memory fetch mem-xxx --summary-only
|
||||
```
|
||||
|
||||
## Reset/Recovery Workflow
|
||||
|
||||
After a context reset or session restart:
|
||||
|
||||
### Step 1: Load Checkpoint
|
||||
```bash
|
||||
checkpoint load
|
||||
# Shows: phase, dependencies, memory refs, status summary
|
||||
```
|
||||
|
||||
### Step 2: Check Memory References
|
||||
```bash
|
||||
# See what's in memory
|
||||
memory refs --checkpoint ckpt-latest
|
||||
# Output:
|
||||
# mem-abc: pytest results (12000 tokens)
|
||||
# mem-def: deployment log (8000 tokens)
|
||||
```
|
||||
|
||||
### Step 3: Fetch Needed Context
|
||||
```bash
|
||||
# Get summary of test results
|
||||
memory fetch mem-abc --summary-only
|
||||
# "42 tests: 40 passed, 2 failed (test_auth, test_db)"
|
||||
|
||||
# If needed, get specific chunk
|
||||
memory fetch mem-abc --chunk 0 # First chunk with failures
|
||||
```
|
||||
|
||||
### Step 4: Resume Work
|
||||
```bash
|
||||
# Check directory status
|
||||
cat ./tests/STATUS.md
|
||||
# Shows current phase, pending tasks, memory refs
|
||||
|
||||
# Continue where you left off
|
||||
status update ./tests --task "Fixing test_auth failure"
|
||||
```
|
||||
|
||||
## Memory Entry Types
|
||||
|
||||
| Type | Purpose | Example |
|
||||
|------|---------|---------|
|
||||
| `transcript` | Full conversation logs | Chat history |
|
||||
| `output` | Command/tool outputs | Test results, build logs |
|
||||
| `summary` | Generated summaries | Checkpoint summaries |
|
||||
| `context` | Saved context state | Variables, environment |
|
||||
| `chunk` | Part of larger entry | Auto-generated |
|
||||
|
||||
## Storage Details
|
||||
|
||||
### SQLite Database (`memory/memory.db`)
|
||||
- Entry metadata (ID, type, timestamps)
|
||||
- Content for small entries
|
||||
- Summaries
|
||||
- Links to checkpoints/directories
|
||||
- Tags for searching
|
||||
|
||||
### Chunk Files (`memory/chunks/`)
|
||||
- Gzip-compressed content
|
||||
- Named by entry ID
|
||||
- Auto-pruned after 30 days
|
||||
|
||||
### DragonflyDB (optional)
|
||||
- Hot cache for recent entries
|
||||
- 1-hour TTL
|
||||
- Faster retrieval for active work
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Store proactively** - Don't wait for context overflow
|
||||
2. **Tag consistently** - Use meaningful tags for search
|
||||
3. **Link to context** - Connect to checkpoints and directories
|
||||
4. **Use summaries** - Fetch summary first, full content only if needed
|
||||
5. **Prune regularly** - Keep memory lean with periodic pruning
|
||||
|
||||
## Example Session
|
||||
|
||||
```bash
|
||||
# 1. Run tests, store output
|
||||
pytest tests/ 2>&1 | memory log --stdin --tag "pytest" --tag "integration"
|
||||
# Stored: mem-20260123-100000-abcd (8500 tokens, 3 chunks)
|
||||
|
||||
# 2. Create checkpoint with memory ref
|
||||
checkpoint now --notes "Integration tests complete"
|
||||
|
||||
# 3. Later, after context reset, recover
|
||||
checkpoint load
|
||||
# Phase: Testing, Memory: 1 entry (8500 tokens)
|
||||
|
||||
memory fetch mem-20260123-100000-abcd --summary-only
|
||||
# "Integration tests: 156 passed, 2 failed
|
||||
# Failures: test_oauth_flow (line 234), test_rate_limit (line 567)"
|
||||
|
||||
# 4. Get specific failure details
|
||||
memory fetch mem-20260123-100000-abcd --chunk 1
|
||||
# (Shows chunk containing the failures)
|
||||
|
||||
# 5. Continue work
|
||||
status update ./tests --task "Fixing test_oauth_flow"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Part of the [Agent Governance System](/opt/agent-governance/docs/ARCHITECTURE.md)*
|
||||
@@ -0,0 +1,171 @@
|
||||
# Phase Dependency Analysis
|
||||
|
||||
## Overview
|
||||
|
||||
This document analyzes the dependencies between Phase 8 (Production Hardening) test results and earlier phase components, identifying gaps and required backfills.
|
||||
|
||||
**Analysis Date:** 2026-01-24
|
||||
**Checkpoint:** ckpt-20260124-023706-11d689d4
|
||||
|
||||
---
|
||||
|
||||
## Phase 8 Test Results Summary
|
||||
|
||||
| Test Suite | Tests | Result | Dependencies |
|
||||
|------------|-------|--------|--------------|
|
||||
| GitHub Integration | 10/10 | PASS | Phase 3 (Pipeline scaffolding) |
|
||||
| Slack Integration | 17/17 | PASS | Phase 3 (Pipeline scaffolding) |
|
||||
| Error Recovery | 10/10 | PASS | Phase 4 (Revocation Engine) |
|
||||
| E2E Workflow | 16/16 | PASS | Phases 1-5 (Full stack) |
|
||||
|
||||
---
|
||||
|
||||
## Dependency Map
|
||||
|
||||
### 1. Integration Tests → Phase 3 (Pipeline)
|
||||
|
||||
**Files Tested:**
|
||||
- `/opt/agent-governance/integrations/github/github.py`
|
||||
- `/opt/agent-governance/integrations/slack/slack.py`
|
||||
- `/opt/agent-governance/integrations/common/base.py`
|
||||
|
||||
**Dependencies:**
|
||||
- `BaseIntegration` class with rate limiting, retry logic, audit logging
|
||||
- `IntegrationEvent` dataclass with event_type routing
|
||||
- `IntegrationManager` for multi-integration coordination
|
||||
|
||||
**Event Types Used (not in pipeline/core.py):**
|
||||
```
|
||||
plan_created → Maps to AgentPhase.PLAN completion
|
||||
execution_started → Maps to AgentPhase.EXECUTE start
|
||||
execution_complete → Maps to AgentPhase.EXECUTE completion
|
||||
violation_detected → Maps to RevocationType events
|
||||
promotion_requested → Maps to tier promotion workflow
|
||||
promotion_approved → Maps to tier promotion completion
|
||||
agent_revoked → Maps to RevocationType.* terminal events
|
||||
approval_required → Maps to StageType.GATE
|
||||
heartbeat → Maps to agent health monitoring
|
||||
```
|
||||
|
||||
**Gap Identified:** Integration event types should be formally defined in `pipeline/core.py` as `IntegrationEventType` enum.
|
||||
|
||||
---
|
||||
|
||||
### 2. Error Recovery Tests → Phase 4 (Revocation)
|
||||
|
||||
**Files Tested:**
|
||||
- `/opt/agent-governance/runtime/revocation.py`
|
||||
|
||||
**Dependencies:**
|
||||
- `ViolationType` enum (14 types) - **More comprehensive than core.py**
|
||||
- `Severity` enum (4 levels) - **Not in core.py**
|
||||
- `RevocationEngine` class
|
||||
- DragonflyDB keys: `agent:{id}:revoke_signal`, `revocations:ledger`, `alerts:queue`
|
||||
|
||||
**Type Comparison:**
|
||||
|
||||
| Location | Enum | Count | Coverage |
|
||||
|----------|------|-------|----------|
|
||||
| `pipeline/core.py` | `RevocationType` | 6 | Basic revocation triggers |
|
||||
| `runtime/revocation.py` | `ViolationType` | 14 | Full violation taxonomy |
|
||||
| `runtime/revocation.py` | `Severity` | 4 | Severity classification |
|
||||
|
||||
**Gap Identified:** `pipeline/core.py` should import or re-export the full `ViolationType` and `Severity` from `runtime/revocation.py` to maintain single source of truth.
|
||||
|
||||
---
|
||||
|
||||
### 3. E2E Tests → Phases 1-5 (Full Stack)
|
||||
|
||||
**Files Tested:**
|
||||
- `/opt/agent-governance/tests/real_e2e_test.py`
|
||||
|
||||
**Phase 1 (Foundation) Dependencies:**
|
||||
- SQLite Ledger schema: `agent_actions`, `agent_metrics`, `violations`, `promotions`
|
||||
- File structure: `/opt/agent-governance/ledger/governance.db`
|
||||
|
||||
**Phase 2 (Vault) Dependencies:**
|
||||
- Vault health check: `GET /v1/sys/health`
|
||||
- AppRole authentication: `auth/approle/role/tier1-agent/role-id`
|
||||
- Policy files: `t0-observer.hcl` through `t4-architect.hcl`
|
||||
- Secret paths: `secret/data/agents/{agent_id}`, `secret/data/inventory/*`
|
||||
|
||||
**Phase 3 (Pipeline) Dependencies:**
|
||||
- DragonflyDB key patterns from `RedisKeys` class
|
||||
- Instruction packet structure
|
||||
- Agent state management
|
||||
|
||||
**Phase 5 (Agent Bootstrap) Dependencies:**
|
||||
- Execution lock acquisition/release
|
||||
- Heartbeat management
|
||||
- Error budget tracking
|
||||
|
||||
**Gap Identified:** E2E test uses hardcoded Redis password (`governance2026`) instead of fetching from Vault.
|
||||
|
||||
---
|
||||
|
||||
## STATUS.md File Inconsistencies
|
||||
|
||||
Several STATUS.md files show "Current Phase: NOT STARTED" in the header but "COMPLETE" or "IN_PROGRESS" in the Activity Log:
|
||||
|
||||
| Directory | Header Says | Activity Log Says |
|
||||
|-----------|-------------|-------------------|
|
||||
| `runtime/` | NOT STARTED | COMPLETE |
|
||||
| `preflight/` | NOT STARTED | COMPLETE |
|
||||
| `pipeline/` | NOT STARTED | COMPLETE |
|
||||
| `integrations/` | NOT STARTED | IN_PROGRESS |
|
||||
| `integrations/github/` | NOT STARTED | (needs update) |
|
||||
| `integrations/slack/` | NOT STARTED | (needs update) |
|
||||
|
||||
**Action Required:** Update STATUS.md files to reflect actual completion state.
|
||||
|
||||
---
|
||||
|
||||
## Backfill Requirements
|
||||
|
||||
### Priority 1: Type Synchronization ✅ COMPLETE
|
||||
|
||||
1. **Updated `pipeline/core.py`** to include:
|
||||
- ✅ `ViolationSeverity` enum (4 levels)
|
||||
- ✅ Full `ViolationType` enum (14 types with severity mapping)
|
||||
- ✅ `IntegrationEventType` enum (9 event types)
|
||||
- ✅ `VIOLATION_SEVERITY_MAP` for severity lookups
|
||||
- ✅ `INTEGRATION_EVENT_PHASE_MAP` for lifecycle mapping
|
||||
- ✅ `RevocationType` alias for backwards compatibility
|
||||
|
||||
2. **Import flow established:**
|
||||
```
|
||||
pipeline/core.py (authoritative source)
|
||||
↓
|
||||
runtime/revocation.py (can now import from core)
|
||||
↓
|
||||
integrations/*.py (uses IntegrationEventType)
|
||||
```
|
||||
|
||||
### Priority 2: STATUS.md Corrections ✅ COMPLETE
|
||||
|
||||
Updated files to show correct Current Phase:
|
||||
- ✅ `/opt/agent-governance/runtime/STATUS.md` → COMPLETE
|
||||
- ✅ `/opt/agent-governance/preflight/STATUS.md` → COMPLETE
|
||||
- ✅ `/opt/agent-governance/pipeline/STATUS.md` → COMPLETE
|
||||
- ✅ `/opt/agent-governance/integrations/STATUS.md` → IN_PROGRESS (with test results)
|
||||
- ✅ `/opt/agent-governance/integrations/github/STATUS.md` → COMPLETE (10/10 tests)
|
||||
- ✅ `/opt/agent-governance/integrations/slack/STATUS.md` → COMPLETE (17/17 tests)
|
||||
|
||||
### Priority 3: Credential Hardening (Deferred)
|
||||
|
||||
Replace hardcoded credentials in test files with Vault lookups:
|
||||
- `/opt/agent-governance/tests/real_e2e_test.py:28` - `REDIS_PASSWORD`
|
||||
- **Status:** Low priority - test files only, not production code
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | Items | Status |
|
||||
|----------|-------|--------|
|
||||
| Type definitions in sync | 3 gaps | ✅ COMPLETE |
|
||||
| STATUS.md files accurate | 6 files | ✅ COMPLETE |
|
||||
| Credentials secure | 1 hardcoded | Deferred (test-only) |
|
||||
| Tests passing | 53/53 | ✅ COMPLETE |
|
||||
|
||||
**Result:** All critical backfill requirements addressed. Phase 8 Production Hardening can proceed with confidence that earlier phases are consistent.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Docs
|
||||
|
||||
> System documentation and architecture specs
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains system documentation and architecture specs.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `ENGINEERING_GUIDE.md` | Documentation |
|
||||
| `ARCHITECTURE.md` | Documentation |
|
||||
| `STATUS_PROTOCOL.md` | Documentation |
|
||||
| `tier0-guide.md` | Documentation |
|
||||
|
||||
## Interfaces / APIs
|
||||
|
||||
*Document any APIs, CLI commands, or interfaces here.*
|
||||
|
||||
## Status
|
||||
|
||||
** Not Started**
|
||||
|
||||
See [STATUS.md](./STATUS.md) for detailed progress tracking.
|
||||
|
||||
## Architecture Reference
|
||||
|
||||
Part of the [Agent Governance System](/opt/agent-governance/docs/ARCHITECTURE.md).
|
||||
|
||||
Parent: [Project Root](/opt/agent-governance)
|
||||
|
||||
---
|
||||
*Last updated: 2026-01-23 23:25:09 UTC*
|
||||
@@ -0,0 +1,45 @@
|
||||
# Status: Docs
|
||||
|
||||
## Current Phase
|
||||
|
||||
** NOT STARTED**
|
||||
|
||||
## Tasks
|
||||
|
||||
| Status | Task | Updated |
|
||||
|--------|------|---------|
|
||||
| ☐ | *No tasks defined* | - |
|
||||
|
||||
## Dependencies
|
||||
|
||||
*No external dependencies.*
|
||||
|
||||
## Issues / Blockers
|
||||
|
||||
*No current issues or blockers.*
|
||||
|
||||
## Activity Log
|
||||
|
||||
### 2026-01-23 23:45:57 UTC
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: Documentation complete: CONTEXT_MANAGEMENT.md, updated README, MEMORY_LAYER.md, STATUS_PROTOCOL.md
|
||||
- **Details**: Phase updated to complete
|
||||
|
||||
### 2026-01-23 23:32:09 UTC
|
||||
- **Phase**: IN_PROGRESS
|
||||
- **Action**: Added checkpoint integration docs
|
||||
- **Details**: Phase updated to in_progress
|
||||
|
||||
### 2026-01-23 23:25:31 UTC
|
||||
- **Phase**: IN_PROGRESS
|
||||
- **Action**: STATUS_PROTOCOL.md added, documentation ongoing
|
||||
- **Details**: Phase updated to in_progress
|
||||
|
||||
### 2026-01-23 23:25:09 UTC
|
||||
- **Phase**: NOT STARTED
|
||||
- **Action**: Initialized
|
||||
- **Details**: Status tracking initialized for this directory.
|
||||
|
||||
|
||||
---
|
||||
*Last updated: 2026-01-23 23:45:57 UTC*
|
||||
@@ -0,0 +1,306 @@
|
||||
# Directory-Level README/STATUS Protocol
|
||||
|
||||
> Guidelines for maintaining documentation and status tracking across the Agent Governance System.
|
||||
|
||||
## Overview
|
||||
|
||||
Every significant subdirectory in the project should contain two files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `README.md` | Overview, purpose, key files, architecture references |
|
||||
| `STATUS.md` | Ongoing status log with phase, tasks, timestamps, dependencies |
|
||||
|
||||
This protocol ensures:
|
||||
- Consistent documentation across the codebase
|
||||
- Clear status visibility for humans and agents
|
||||
- Traceable progress through the project lifecycle
|
||||
|
||||
## File Templates
|
||||
|
||||
### README.md Structure
|
||||
|
||||
```markdown
|
||||
# Directory Name
|
||||
|
||||
> One-line purpose description
|
||||
|
||||
## Overview
|
||||
Brief explanation of what this directory contains.
|
||||
|
||||
## Key Files
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `main.py` | Primary module |
|
||||
| `config.yaml` | Configuration |
|
||||
|
||||
## Interfaces / APIs
|
||||
Document any CLI commands, APIs, or interfaces.
|
||||
|
||||
## Status
|
||||
**✅ COMPLETE** (or 🚧 IN_PROGRESS, ❗ BLOCKED, etc.)
|
||||
|
||||
See [STATUS.md](./STATUS.md) for detailed progress.
|
||||
|
||||
## Architecture Reference
|
||||
Part of the [Agent Governance System](/opt/agent-governance/docs/ARCHITECTURE.md).
|
||||
Parent: [Parent Directory](../)
|
||||
```
|
||||
|
||||
### STATUS.md Structure
|
||||
|
||||
```markdown
|
||||
# Status: Directory Name
|
||||
|
||||
## Current Phase
|
||||
**🚧 IN_PROGRESS**
|
||||
|
||||
## Tasks
|
||||
| Status | Task | Updated |
|
||||
|--------|------|---------|
|
||||
| ✓ | Initial implementation | 2026-01-20 |
|
||||
| ☐ | Add tests | - |
|
||||
|
||||
## Dependencies
|
||||
- Requires `pipeline/core.py`
|
||||
- Depends on DragonflyDB service
|
||||
|
||||
## Issues / Blockers
|
||||
- Waiting on schema finalization
|
||||
|
||||
## Activity Log
|
||||
### 2026-01-23 10:30:00 UTC
|
||||
- **Phase**: IN_PROGRESS
|
||||
- **Action**: Added chaos testing
|
||||
- **Details**: Implemented error injection framework
|
||||
```
|
||||
|
||||
## Status Phases
|
||||
|
||||
| Phase | Icon | Description |
|
||||
|-------|------|-------------|
|
||||
| `complete` | ✅ | Work is finished and verified |
|
||||
| `in_progress` | 🚧 | Active development underway |
|
||||
| `blocked` | ❗ | Waiting on external dependencies |
|
||||
| `needs_review` | ⚠️ | Requires attention or review |
|
||||
| `not_started` | ⬜ | No work has begun |
|
||||
|
||||
## CLI Commands
|
||||
|
||||
The `status` command provides management tools:
|
||||
|
||||
```bash
|
||||
# Check all directories for missing/outdated files
|
||||
status sweep
|
||||
|
||||
# Auto-create missing README/STATUS files
|
||||
status sweep --fix
|
||||
|
||||
# Update a directory's status
|
||||
status update ./pipeline --phase complete
|
||||
status update ./tests --phase in_progress --task "Add integration tests"
|
||||
|
||||
# Initialize files for a new directory
|
||||
status init ./new-module
|
||||
|
||||
# Show project-wide dashboard
|
||||
status dashboard
|
||||
|
||||
# View templates
|
||||
status template readme
|
||||
status template status
|
||||
```
|
||||
|
||||
## Agent Workflow Integration
|
||||
|
||||
### When Entering a Directory
|
||||
|
||||
Agents should:
|
||||
1. Read `README.md` to understand the directory's purpose
|
||||
2. Read `STATUS.md` to see current progress and any blockers
|
||||
3. Check dependencies before starting work
|
||||
|
||||
### While Working
|
||||
|
||||
Agents should:
|
||||
1. Update `STATUS.md` when starting a significant task
|
||||
2. Log activity entries for notable changes
|
||||
3. Update task checkboxes as work completes
|
||||
|
||||
### Before Leaving a Directory
|
||||
|
||||
Agents should:
|
||||
1. Update `STATUS.md` with final state
|
||||
2. Mark completed tasks as done
|
||||
3. Add any new issues discovered
|
||||
4. Update the timestamp
|
||||
|
||||
### Example Agent Workflow
|
||||
|
||||
```python
|
||||
# At start of work
|
||||
async def enter_directory(dir_path: str):
|
||||
# Read context
|
||||
readme = await read_file(f"{dir_path}/README.md")
|
||||
status = await read_file(f"{dir_path}/STATUS.md")
|
||||
|
||||
# Log entry
|
||||
logger.info(f"Entering {dir_path}, phase: {parse_phase(status)}")
|
||||
|
||||
# During work
|
||||
async def log_progress(dir_path: str, task: str):
|
||||
await run_command(f"status update {dir_path} --task '{task}'")
|
||||
|
||||
# At end of work
|
||||
async def exit_directory(dir_path: str, phase: str):
|
||||
await run_command(f"status update {dir_path} --phase {phase}")
|
||||
```
|
||||
|
||||
## Diagnostics
|
||||
|
||||
### Finding Incomplete Directories
|
||||
|
||||
```bash
|
||||
# Show all non-complete directories
|
||||
status dashboard
|
||||
|
||||
# Example output:
|
||||
# ❗ BLOCKED:
|
||||
# integrations/slack (updated 3d ago)
|
||||
# 🚧 IN_PROGRESS:
|
||||
# tests/multi-agent-chaos (updated today)
|
||||
# pipeline (updated 1d ago)
|
||||
```
|
||||
|
||||
### Automated Sweeps
|
||||
|
||||
Run regular sweeps to catch missing documentation:
|
||||
|
||||
```bash
|
||||
# Quick check
|
||||
status sweep
|
||||
|
||||
# Auto-fix missing files
|
||||
status sweep --fix
|
||||
```
|
||||
|
||||
## Checkpoint Integration
|
||||
|
||||
The status system is integrated with the checkpoint system for unified state management.
|
||||
|
||||
### How They Work Together
|
||||
|
||||
1. **When `checkpoint now` runs**: The checkpoint captures an aggregate snapshot of all directory statuses, including:
|
||||
- Status summary (counts by phase)
|
||||
- List of all directories with their current phase
|
||||
- Timestamps and task counts
|
||||
|
||||
2. **When `status update` runs**: A lightweight checkpoint delta is automatically created to record the change (unless `--no-checkpoint` is specified).
|
||||
|
||||
3. **Unified reporting**: Use `checkpoint report` to see both checkpoint metadata and directory statuses side-by-side.
|
||||
|
||||
### Key Commands
|
||||
|
||||
```bash
|
||||
# Create checkpoint with directory status snapshot
|
||||
checkpoint now --notes "Completed pipeline work"
|
||||
|
||||
# View combined checkpoint + status report
|
||||
checkpoint report
|
||||
|
||||
# View timeline of checkpoints with status changes
|
||||
checkpoint timeline
|
||||
|
||||
# Update status (auto-creates checkpoint delta)
|
||||
status update ./pipeline --phase complete
|
||||
|
||||
# Update status without checkpoint
|
||||
status update ./tests --phase in_progress --no-checkpoint
|
||||
```
|
||||
|
||||
### Recovery After Reset
|
||||
|
||||
After a context reset or session restart:
|
||||
|
||||
1. **Load the latest checkpoint**:
|
||||
```bash
|
||||
checkpoint load
|
||||
```
|
||||
|
||||
2. **Inspect per-directory statuses**:
|
||||
```bash
|
||||
checkpoint report
|
||||
# or
|
||||
status dashboard
|
||||
```
|
||||
|
||||
3. **Resume work** by finding active directories:
|
||||
```bash
|
||||
checkpoint report --phase in_progress
|
||||
```
|
||||
|
||||
4. **Read local STATUS.md** in the target directory for task-level context.
|
||||
|
||||
### Example: Full Recovery Workflow
|
||||
|
||||
```bash
|
||||
# 1. Load checkpoint to get context
|
||||
checkpoint load
|
||||
# Shows: Phase, Dependencies, Status Summary
|
||||
|
||||
# 2. See what's active
|
||||
checkpoint report --phase in_progress
|
||||
# Lists: Active directories with their progress
|
||||
|
||||
# 3. Pick up work in a directory
|
||||
cd /opt/agent-governance/pipeline
|
||||
cat STATUS.md
|
||||
# Read activity log, pending tasks
|
||||
|
||||
# 4. Continue work and update status
|
||||
status update . --task "Resumed: adding validation" --phase in_progress
|
||||
# Auto-creates checkpoint delta
|
||||
|
||||
# 5. When done, mark complete
|
||||
status update . --phase complete
|
||||
# Creates final checkpoint
|
||||
```
|
||||
|
||||
## Directory Exclusions
|
||||
|
||||
The following directories are excluded from status tracking:
|
||||
|
||||
- `__pycache__` - Python cache
|
||||
- `node_modules` - Node.js dependencies
|
||||
- `.git` - Version control
|
||||
- `logs` - Runtime logs
|
||||
- `storage` - Generated data
|
||||
- `credentials` - Sensitive data
|
||||
- `workspace` - Agent workspaces
|
||||
- `dragonfly-data` - Database files
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep README.md Stable**: Update only when structure changes
|
||||
2. **Update STATUS.md Frequently**: Log all significant activity
|
||||
3. **Use Clear Task Descriptions**: Be specific about what needs doing
|
||||
4. **Document Blockers**: Help others understand dependencies
|
||||
5. **Timestamp Everything**: Enable tracking of staleness
|
||||
6. **Link to Architecture**: Reference parent docs for context
|
||||
|
||||
## Integration with Checkpoints
|
||||
|
||||
The status system integrates with the checkpoint system:
|
||||
|
||||
```bash
|
||||
# After major changes, create checkpoint
|
||||
checkpoint now --notes "Updated pipeline status to complete"
|
||||
|
||||
# Status can inform checkpoint decisions
|
||||
status dashboard # See what's in progress
|
||||
checkpoint now # Save current state
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Part of the [Agent Governance System](/opt/agent-governance/docs/ARCHITECTURE.md)*
|
||||
@@ -0,0 +1,59 @@
|
||||
# Tier 0 Agent Guide
|
||||
|
||||
## Overview
|
||||
|
||||
You are a **Tier 0 Observer Agent**. Your role is to learn the environment,
|
||||
understand patterns, and generate compliant plans without executing them.
|
||||
|
||||
## What You CAN Do
|
||||
|
||||
1. **Read Documentation**
|
||||
- Access all files in `/opt/agent-governance/docs/`
|
||||
- Understand system architecture and patterns
|
||||
|
||||
2. **Read Inventory**
|
||||
- View available targets in `/opt/agent-governance/inventory/`
|
||||
- Understand which hosts you could potentially manage
|
||||
|
||||
3. **Generate Plans**
|
||||
- Create Terraform plans (plan only, no apply)
|
||||
- Create Ansible playbooks with check mode
|
||||
- Write plans to `/opt/agent-governance/agents/tier0-agent/plans/`
|
||||
|
||||
4. **Request Reviews**
|
||||
- Submit plans for human review
|
||||
- Ask questions about requirements
|
||||
|
||||
## What You CANNOT Do
|
||||
|
||||
- Execute any commands
|
||||
- Write files outside your workspace
|
||||
- Access secrets or credentials
|
||||
- SSH to any hosts
|
||||
- Make API calls to infrastructure
|
||||
- Apply Terraform changes
|
||||
- Run Ansible playbooks (even in check mode)
|
||||
|
||||
## Promotion Path
|
||||
|
||||
To be promoted to Tier 1:
|
||||
1. Generate 5 compliant plans
|
||||
2. Maintain 3 consecutive compliant runs
|
||||
3. Zero violations in 30 days
|
||||
4. Plans must demonstrate correct scoping and rollback awareness
|
||||
|
||||
## Example: Generating a Plan
|
||||
|
||||
```
|
||||
# Good: Generate a plan file
|
||||
Write plan to: /opt/agent-governance/agents/tier0-agent/plans/my-plan.md
|
||||
|
||||
# Bad: Try to execute
|
||||
terraform apply # FORBIDDEN - will trigger violation
|
||||
```
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Read other docs in this directory
|
||||
- Check the inventory for available targets
|
||||
- Ask questions via the request_review action
|
||||
Reference in New Issue
Block a user