Add Phase 10-12 implementation: multi-tenant, marketplace, observability
Major additions: - marketplace/: Agent template registry with FTS5 search, ratings, versioning - observability/: Prometheus metrics, distributed tracing, structured logging - ledger/migrations/: Database migration scripts for multi-tenant support - tests/governance/: 15 new test files for phases 6-12 (295 total tests) - bin/validate-phases: Full 12-phase validation script New features: - Multi-tenant support with tenant isolation and quota enforcement - Agent marketplace with semantic versioning and search - Observability with metrics, tracing, and log correlation - Tier-1 agent bootstrap scripts Updated components: - ledger/api.py: Extended API for tenants, marketplace, observability - ledger/schema.sql: Added tenant, project, marketplace tables - testing/framework.ts: Enhanced test framework - checkpoint/checkpoint.py: Improved checkpoint management Archived: - External integrations (Slack/GitHub/PagerDuty) moved to .archive/ - Old checkpoint files cleaned up Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
profit
co-authored by
Claude Opus 4.5
Claude Opus 4.5
parent
4b5b3b0a2d
commit
8c6e7831e9
@@ -0,0 +1,395 @@
|
||||
# Production Credentials Setup Guide
|
||||
|
||||
> How to configure external integration credentials for Slack, GitHub, and PagerDuty.
|
||||
|
||||
## Overview
|
||||
|
||||
The Agent Governance System uses HashiCorp Vault for secure credential storage. Each integration has a dedicated secret path with specific required keys. The system also supports environment variable overrides for development and CI environments.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Vault server running at `https://127.0.0.1:8200` (or `VAULT_ADDR`)
|
||||
- Vault root token or appropriate access policy
|
||||
- Admin access to Slack, GitHub, and PagerDuty accounts
|
||||
|
||||
## Credential Priority
|
||||
|
||||
The secrets manager loads credentials in this order:
|
||||
|
||||
1. **Environment variables** (highest priority)
|
||||
2. **Vault secrets** (production)
|
||||
3. **Mock values** (testing only, when `INTEGRATION_DRY_RUN=true`)
|
||||
|
||||
---
|
||||
|
||||
## Slack Integration
|
||||
|
||||
### Vault Path
|
||||
```
|
||||
secret/data/integrations/slack
|
||||
```
|
||||
|
||||
### Required Keys
|
||||
|
||||
| Key | Description | Format |
|
||||
|-----|-------------|--------|
|
||||
| `slack_bot_token` | Bot User OAuth Token | `xoxb-...` |
|
||||
| `slack_webhook_url` | Incoming Webhook URL | `https://hooks.slack.com/services/...` |
|
||||
|
||||
### How to Obtain
|
||||
|
||||
#### Bot Token
|
||||
1. Go to [api.slack.com/apps](https://api.slack.com/apps)
|
||||
2. Create a new app or select existing
|
||||
3. Navigate to **OAuth & Permissions**
|
||||
4. Under **Bot Token Scopes**, add:
|
||||
- `chat:write` - Send messages
|
||||
- `chat:write.public` - Send to public channels
|
||||
- `files:write` - Upload files (optional)
|
||||
5. Click **Install to Workspace**
|
||||
6. Copy the **Bot User OAuth Token** (`xoxb-...`)
|
||||
|
||||
#### Webhook URL
|
||||
1. In your Slack app, go to **Incoming Webhooks**
|
||||
2. Enable incoming webhooks
|
||||
3. Click **Add New Webhook to Workspace**
|
||||
4. Select the target channel
|
||||
5. Copy the **Webhook URL**
|
||||
|
||||
### Store in Vault
|
||||
|
||||
```bash
|
||||
# Set Vault address and token
|
||||
export VAULT_ADDR="https://127.0.0.1:8200"
|
||||
export VAULT_TOKEN=$(cat /opt/vault/init-keys.json | python3 -c "import sys,json; print(json.load(sys.stdin)['root_token'])")
|
||||
|
||||
# Store Slack credentials
|
||||
vault kv put secret/integrations/slack \
|
||||
slack_bot_token="xoxb-YOUR-BOT-TOKEN" \
|
||||
slack_webhook_url="https://hooks.slack.com/services/YOUR/WEBHOOK/URL" \
|
||||
configured=true
|
||||
|
||||
# Or using curl (if vault CLI not available)
|
||||
curl -sk -X POST \
|
||||
-H "X-Vault-Token: $VAULT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"slack_bot_token": "xoxb-YOUR-BOT-TOKEN",
|
||||
"slack_webhook_url": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
|
||||
"configured": true
|
||||
}
|
||||
}' \
|
||||
"$VAULT_ADDR/v1/secret/data/integrations/slack"
|
||||
```
|
||||
|
||||
### Environment Variable Override
|
||||
|
||||
```bash
|
||||
export SLACK_BOT_TOKEN="xoxb-YOUR-BOT-TOKEN"
|
||||
export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
|
||||
```
|
||||
|
||||
### Verify Configuration
|
||||
|
||||
```bash
|
||||
# Test with Python
|
||||
python3 -c "
|
||||
from integrations.slack import SlackIntegration
|
||||
slack = SlackIntegration()
|
||||
print(f'Enabled: {slack.enabled}')
|
||||
print(f'Connection test: {slack.test_connection()}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GitHub Integration
|
||||
|
||||
### Vault Path
|
||||
```
|
||||
secret/data/integrations/github
|
||||
```
|
||||
|
||||
### Required Keys
|
||||
|
||||
| Key | Description | Format |
|
||||
|-----|-------------|--------|
|
||||
| `github_token` | Personal Access Token or App Token | `ghp_...` or `ghs_...` |
|
||||
| `github_webhook_secret` | Webhook signature secret | Random string |
|
||||
|
||||
### How to Obtain
|
||||
|
||||
#### Personal Access Token (Classic)
|
||||
1. Go to [github.com/settings/tokens](https://github.com/settings/tokens)
|
||||
2. Click **Generate new token (classic)**
|
||||
3. Select scopes:
|
||||
- `repo` - Full repository access
|
||||
- `write:org` - Organization webhooks (if needed)
|
||||
- `admin:repo_hook` - Manage webhooks
|
||||
4. Click **Generate token**
|
||||
5. Copy the token immediately (shown only once)
|
||||
|
||||
#### Fine-Grained Personal Access Token (Recommended)
|
||||
1. Go to [github.com/settings/tokens?type=beta](https://github.com/settings/tokens?type=beta)
|
||||
2. Click **Generate new token**
|
||||
3. Set token name and expiration
|
||||
4. Select **Repository access** (specific repos or all)
|
||||
5. Under **Permissions**, enable:
|
||||
- Contents: Read and write
|
||||
- Pull requests: Read and write
|
||||
- Commit statuses: Read and write
|
||||
- Webhooks: Read and write
|
||||
6. Click **Generate token**
|
||||
|
||||
#### Webhook Secret
|
||||
1. Generate a random secret:
|
||||
```bash
|
||||
openssl rand -hex 32
|
||||
```
|
||||
2. When creating GitHub webhooks, use this secret for signature verification
|
||||
|
||||
### Store in Vault
|
||||
|
||||
```bash
|
||||
# Generate webhook secret
|
||||
WEBHOOK_SECRET=$(openssl rand -hex 32)
|
||||
|
||||
# Store GitHub credentials
|
||||
vault kv put secret/integrations/github \
|
||||
github_token="ghp_YOUR-PERSONAL-ACCESS-TOKEN" \
|
||||
github_webhook_secret="$WEBHOOK_SECRET" \
|
||||
configured=true
|
||||
|
||||
# Or using curl
|
||||
curl -sk -X POST \
|
||||
-H "X-Vault-Token: $VAULT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"data\": {
|
||||
\"github_token\": \"ghp_YOUR-PERSONAL-ACCESS-TOKEN\",
|
||||
\"github_webhook_secret\": \"$WEBHOOK_SECRET\",
|
||||
\"configured\": true
|
||||
}
|
||||
}" \
|
||||
"$VAULT_ADDR/v1/secret/data/integrations/github"
|
||||
```
|
||||
|
||||
### Environment Variable Override
|
||||
|
||||
```bash
|
||||
export GITHUB_TOKEN="ghp_YOUR-PERSONAL-ACCESS-TOKEN"
|
||||
export GITHUB_WEBHOOK_SECRET="your-webhook-secret"
|
||||
```
|
||||
|
||||
### Verify Configuration
|
||||
|
||||
```bash
|
||||
python3 -c "
|
||||
from integrations.github import GitHubIntegration
|
||||
gh = GitHubIntegration()
|
||||
print(f'Enabled: {gh.enabled}')
|
||||
print(f'Connection test: {gh.test_connection()}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PagerDuty Integration
|
||||
|
||||
### Vault Path
|
||||
```
|
||||
secret/data/integrations/pagerduty
|
||||
```
|
||||
|
||||
### Required Keys
|
||||
|
||||
| Key | Description | Format |
|
||||
|-----|-------------|--------|
|
||||
| `pagerduty_routing_key` | Events API v2 Integration Key | 32-char hex string |
|
||||
| `pagerduty_api_token` | REST API Token (optional) | 20-char string |
|
||||
| `pagerduty_service_id` | Service ID (optional) | `P...` format |
|
||||
|
||||
### How to Obtain
|
||||
|
||||
#### Routing Key (Events API v2) - Required
|
||||
1. Log into [app.pagerduty.com](https://app.pagerduty.com)
|
||||
2. Go to **Services** > Select your service
|
||||
3. Click **Integrations** tab
|
||||
4. Click **Add Integration**
|
||||
5. Select **Events API v2**
|
||||
6. Copy the **Integration Key** (this is your routing key)
|
||||
|
||||
#### REST API Token (Optional)
|
||||
1. Go to **Configuration** > **API Access**
|
||||
2. Click **Create New API Key**
|
||||
3. Enter a description (e.g., "Agent Governance")
|
||||
4. Choose **Read/Write** access
|
||||
5. Copy the generated token
|
||||
|
||||
#### Service ID (Optional)
|
||||
1. Go to **Services** > Select your service
|
||||
2. The Service ID is in the URL: `app.pagerduty.com/services/PXXXXXX`
|
||||
3. Or find it under **Settings** > **Service ID**
|
||||
|
||||
### Store in Vault
|
||||
|
||||
```bash
|
||||
# Store PagerDuty credentials
|
||||
vault kv put secret/integrations/pagerduty \
|
||||
pagerduty_routing_key="YOUR-32-CHAR-ROUTING-KEY" \
|
||||
pagerduty_api_token="YOUR-API-TOKEN" \
|
||||
pagerduty_service_id="PXXXXXXX" \
|
||||
configured=true
|
||||
|
||||
# Or using curl
|
||||
curl -sk -X POST \
|
||||
-H "X-Vault-Token: $VAULT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"pagerduty_routing_key": "YOUR-32-CHAR-ROUTING-KEY",
|
||||
"pagerduty_api_token": "YOUR-API-TOKEN",
|
||||
"pagerduty_service_id": "PXXXXXXX",
|
||||
"configured": true
|
||||
}
|
||||
}' \
|
||||
"$VAULT_ADDR/v1/secret/data/integrations/pagerduty"
|
||||
```
|
||||
|
||||
### Environment Variable Override
|
||||
|
||||
```bash
|
||||
export PAGERDUTY_ROUTING_KEY="your-32-char-routing-key"
|
||||
export PAGERDUTY_API_TOKEN="your-api-token"
|
||||
export PAGERDUTY_SERVICE_ID="PXXXXXXX"
|
||||
```
|
||||
|
||||
### Verify Configuration
|
||||
|
||||
```bash
|
||||
python3 -c "
|
||||
from integrations.pagerduty import PagerDutyIntegration
|
||||
pd = PagerDutyIntegration()
|
||||
print(f'Enabled: {pd.enabled}')
|
||||
print(f'Connection test: {pd.test_connection()}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Runtime Environment Variables
|
||||
|
||||
The following environment variables control integration behavior:
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `VAULT_ADDR` | Vault server address | `https://127.0.0.1:8200` |
|
||||
| `VAULT_TOKEN` | Vault authentication token | Read from `/opt/vault/init-keys.json` |
|
||||
| `USE_VAULT` | Enable Vault secret loading | `true` |
|
||||
| `INTEGRATION_DRY_RUN` | Prevent actual API calls (testing) | `false` |
|
||||
| `TESTING` | Enable mock mode for all secrets | `false` |
|
||||
|
||||
### Production Configuration
|
||||
|
||||
```bash
|
||||
# /etc/environment or systemd unit file
|
||||
VAULT_ADDR=https://vault.internal:8200
|
||||
USE_VAULT=true
|
||||
INTEGRATION_DRY_RUN=false
|
||||
```
|
||||
|
||||
### Development Configuration
|
||||
|
||||
```bash
|
||||
# .env or shell profile
|
||||
export SLACK_BOT_TOKEN="xoxb-dev-token"
|
||||
export GITHUB_TOKEN="ghp_dev-token"
|
||||
export PAGERDUTY_ROUTING_KEY="dev-routing-key"
|
||||
export INTEGRATION_DRY_RUN=true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Script
|
||||
|
||||
Run this script to verify all integrations are properly configured:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# verify-integrations.sh
|
||||
|
||||
cd /opt/agent-governance
|
||||
|
||||
echo "=== Integration Credential Verification ==="
|
||||
echo ""
|
||||
|
||||
python3 << 'EOF'
|
||||
import os
|
||||
import sys
|
||||
sys.path.insert(0, "/opt/agent-governance/integrations")
|
||||
|
||||
from common.secrets import SecretsManager
|
||||
|
||||
secrets = SecretsManager()
|
||||
|
||||
integrations = ["slack", "github", "pagerduty"]
|
||||
|
||||
for name in integrations:
|
||||
result = secrets.validate_integration(name)
|
||||
status = "READY" if result["ready"] else "MISSING KEYS"
|
||||
print(f"\n[{name.upper()}] {status}")
|
||||
if result["configured"]:
|
||||
print(f" Configured: {', '.join(result['configured'])}")
|
||||
if result["missing"]:
|
||||
print(f" Missing: {', '.join(result['missing'])}")
|
||||
if result["mock_mode"]:
|
||||
print(f" (Running in mock mode)")
|
||||
|
||||
print("\n" + "=" * 45)
|
||||
EOF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "No credentials found"
|
||||
1. Check Vault is accessible: `curl -sk $VAULT_ADDR/v1/sys/health`
|
||||
2. Verify token is valid: `vault token lookup`
|
||||
3. Check secret path exists: `vault kv get secret/integrations/slack`
|
||||
|
||||
### "Connection test failed"
|
||||
1. Verify credentials are correct (not placeholder values)
|
||||
2. Check network connectivity to external services
|
||||
3. Enable dry-run mode to test without making real API calls:
|
||||
```bash
|
||||
INTEGRATION_DRY_RUN=true python3 -c "..."
|
||||
```
|
||||
|
||||
### "Rate limit exceeded"
|
||||
1. The integration base class handles rate limiting (60 req/min default)
|
||||
2. Check `_request_times` in the integration instance
|
||||
3. Wait 60 seconds and retry
|
||||
|
||||
---
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Never commit credentials** to version control
|
||||
2. **Rotate tokens regularly** (GitHub: 90 days, PagerDuty: annually)
|
||||
3. **Use least privilege** - only request scopes you need
|
||||
4. **Enable Vault audit logging** for credential access tracking
|
||||
5. **Use environment variables** only in development; production should use Vault
|
||||
6. **Monitor for credential leaks** with GitHub secret scanning
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Architecture Overview](./ARCHITECTURE.md)
|
||||
- [Integration Status](../integrations/STATUS.md)
|
||||
- [Secrets Management Code](../integrations/common/secrets.py)
|
||||
|
||||
---
|
||||
*Last updated: 2026-01-24*
|
||||
@@ -0,0 +1,311 @@
|
||||
# Production Pipeline: Report → OpenRouter Orchestration
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the automatic transition from the UI "view report" stage into the live multi-agent pipeline, including OpenRouter-driven parallel execution.
|
||||
|
||||
**Created:** 2026-01-24
|
||||
**Status:** Implemented
|
||||
|
||||
---
|
||||
|
||||
## Architecture Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ UI DASHBOARD │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
|
||||
│ │ SPAWN │───▶│ RUNNING │───▶│ REPORT │───▶│ AUTO-ORCHESTRATE │ │
|
||||
│ │ Pipeline │ │ Agents │ │ Stage │ │ (NEW) │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └────────┬─────────┘ │
|
||||
│ │ │
|
||||
└───────────────────────────────────────────────────────────┼─────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ OPENROUTER ORCHESTRATION │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ MultiAgentOrchestrator │ │
|
||||
│ │ │ │
|
||||
│ │ ┌─────────────┐ ┌─────────────┐ │ │
|
||||
│ │ │ ALPHA │◄────────────▶│ BETA │ │ │
|
||||
│ │ │ (Research) │ Messages │ (Synthesis) │ │ │
|
||||
│ │ │ Python │ │ Bun │ │ │
|
||||
│ │ └──────┬──────┘ └──────┬──────┘ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ └─────────┬──────────────────┘ │ │
|
||||
│ │ │ │ │
|
||||
│ │ ▼ │ │
|
||||
│ │ ┌─────────────┐ │ │
|
||||
│ │ │ GAMMA │ (Spawned on STUCK/CONFLICT) │ │
|
||||
│ │ │ (Mediator) │ │ │
|
||||
│ │ └─────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Shared Infrastructure: │
|
||||
│ • Blackboard (DragonflyDB) - Proposals, solutions, consensus │
|
||||
│ • MessageBus (Redis PubSub) - Agent coordination │
|
||||
│ • MetricsCollector - Performance tracking │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ COMPLETION & AUDIT │
|
||||
│ │
|
||||
│ • Results written to SQLite ledger │
|
||||
│ • Checkpoint created with final state │
|
||||
│ • WebSocket broadcast to UI │
|
||||
│ • Pipeline status → COMPLETED │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Components
|
||||
|
||||
### 1. Auto-Orchestration Trigger
|
||||
|
||||
**Location:** `/opt/agent-governance/ui/server.ts`
|
||||
|
||||
**Trigger Conditions:**
|
||||
- Pipeline reaches REPORT phase
|
||||
- All agents have completed or timed out
|
||||
- No critical failures blocking continuation
|
||||
|
||||
**New Endpoint:** `POST /api/pipeline/continue`
|
||||
```typescript
|
||||
{
|
||||
pipeline_id: string;
|
||||
mode: "openrouter" | "local"; // openrouter = full LLM, local = mock
|
||||
model?: string; // Default: anthropic/claude-sonnet-4
|
||||
timeout?: number; // Default: 120s
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Parallel Agent Execution
|
||||
|
||||
**Python Agent (ALPHA):**
|
||||
- Path: `/opt/agent-governance/agents/llm-planner/governed_agent.py`
|
||||
- Role: Research, analysis, proposal generation
|
||||
- Runtime: Python 3.11 with venv
|
||||
|
||||
**Bun Agent (BETA):**
|
||||
- Path: `/opt/agent-governance/agents/llm-planner-ts/governed-agent.ts`
|
||||
- Role: Synthesis, evaluation, solution building
|
||||
- Runtime: Bun (4x faster than Node.js)
|
||||
|
||||
**Coordination:**
|
||||
- Both agents connect to same DragonflyDB instance
|
||||
- Shared Blackboard for structured data exchange
|
||||
- MessageBus for real-time communication
|
||||
- SpawnController monitors for GAMMA trigger conditions
|
||||
|
||||
### 3. OpenRouter Integration
|
||||
|
||||
**Credential Flow:**
|
||||
```
|
||||
Vault (secret/data/api-keys/openrouter)
|
||||
│
|
||||
▼
|
||||
getVaultSecret() in agent code
|
||||
│
|
||||
▼
|
||||
OpenAI client with baseURL: "https://openrouter.ai/api/v1"
|
||||
│
|
||||
▼
|
||||
Model: anthropic/claude-sonnet-4
|
||||
```
|
||||
|
||||
**Rate Limiting:**
|
||||
- Handled by OpenRouter API
|
||||
- Circuit breaker in governance layer (5 failures → open)
|
||||
- Per-agent token budget tracking
|
||||
|
||||
### 4. Error Handling & Failover
|
||||
|
||||
**Level 1: Agent-Level Recovery**
|
||||
```
|
||||
Error Budget per agent:
|
||||
- max_total_errors: 8
|
||||
- max_same_error_repeats: 2
|
||||
- max_procedure_violations: 1
|
||||
|
||||
On budget exceeded → Agent revoked, handoff created
|
||||
```
|
||||
|
||||
**Level 2: Pipeline-Level Recovery**
|
||||
```
|
||||
On agent failure:
|
||||
1. Record failure in DragonflyDB
|
||||
2. Check if partner agent can continue alone
|
||||
3. If both fail → Pipeline status = FAILED
|
||||
4. Create checkpoint with failure details
|
||||
```
|
||||
|
||||
**Level 3: Orchestration-Level Recovery**
|
||||
```
|
||||
On orchestration timeout (120s default):
|
||||
1. Force-stop running agents
|
||||
2. Collect partial results from Blackboard
|
||||
3. Generate partial report
|
||||
4. Pipeline status = TIMEOUT
|
||||
```
|
||||
|
||||
**GAMMA Spawn Conditions:**
|
||||
| Condition | Threshold | Action |
|
||||
|-----------|-----------|--------|
|
||||
| STUCK | 30s no progress | Spawn GAMMA mediator |
|
||||
| CONFLICT | 3+ unresolved proposals | Spawn GAMMA to arbitrate |
|
||||
| COMPLEXITY | Score > 0.8 | Spawn GAMMA for decomposition |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Existing (Modified)
|
||||
- `POST /api/spawn` - Creates pipeline, now includes `auto_continue: boolean`
|
||||
- `GET /api/checkpoint/report` - Returns report with continuation status
|
||||
|
||||
### New
|
||||
- `POST /api/pipeline/continue` - Triggers OpenRouter orchestration
|
||||
- `GET /api/pipeline/{id}/orchestration` - Gets orchestration status
|
||||
- `POST /api/pipeline/{id}/stop` - Emergency stop
|
||||
|
||||
---
|
||||
|
||||
## WebSocket Events
|
||||
|
||||
### New Events
|
||||
```typescript
|
||||
// Orchestration started
|
||||
{ type: "orchestration_started", data: { pipeline_id, model, agents: ["ALPHA", "BETA"] } }
|
||||
|
||||
// Agent spawned
|
||||
{ type: "agent_spawned", data: { pipeline_id, agent_id, role, runtime } }
|
||||
|
||||
// Agent message
|
||||
{ type: "agent_message", data: { pipeline_id, from, to, content } }
|
||||
|
||||
// GAMMA spawned (conditional)
|
||||
{ type: "gamma_spawned", data: { pipeline_id, reason: "STUCK" | "CONFLICT" | "COMPLEXITY" } }
|
||||
|
||||
// Consensus reached
|
||||
{ type: "consensus_reached", data: { pipeline_id, proposal_id, votes } }
|
||||
|
||||
// Orchestration complete
|
||||
{ type: "orchestration_complete", data: { pipeline_id, status, results } }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Enable auto-orchestration after report
|
||||
AUTO_ORCHESTRATE=true
|
||||
|
||||
# Default model for OpenRouter
|
||||
OPENROUTER_MODEL=anthropic/claude-sonnet-4
|
||||
|
||||
# Orchestration timeout (seconds)
|
||||
ORCHESTRATION_TIMEOUT=120
|
||||
|
||||
# GAMMA spawn thresholds
|
||||
GAMMA_STUCK_THRESHOLD=30
|
||||
GAMMA_CONFLICT_THRESHOLD=3
|
||||
GAMMA_COMPLEXITY_THRESHOLD=0.8
|
||||
```
|
||||
|
||||
### Vault Secrets Required
|
||||
```
|
||||
secret/data/api-keys/openrouter
|
||||
└── api_key: "sk-or-..."
|
||||
|
||||
secret/data/services/dragonfly
|
||||
└── password: "..."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Add Auto-Continue Logic to UI Server
|
||||
- [x] Add `triggerOrchestration()` function
|
||||
- [x] Modify `checkPipelineCompletion()` to check for auto_continue
|
||||
- [x] Add `/api/pipeline/continue` endpoint
|
||||
|
||||
### Step 2: Connect to Multi-Agent Orchestrator
|
||||
- [x] Spawn orchestrator.ts from UI via Bun.spawn()
|
||||
- [x] Pass pipeline context (task_id, objective, model, timeout)
|
||||
- [x] Wire up WebSocket events (orchestration_started, agent_message, consensus_event, orchestration_complete)
|
||||
|
||||
### Step 3: Add Orchestration Status Tracking
|
||||
- [x] Track orchestration state in Redis (ORCHESTRATING status)
|
||||
- [x] Add orchestration_started_at timestamp
|
||||
- [x] Create checkpoint on completion
|
||||
|
||||
### Step 4: Implement Error Handling
|
||||
- [x] Add timeout handling via orchestrator --timeout flag
|
||||
- [x] Capture exit codes and error messages
|
||||
- [x] Set ORCHESTRATION_FAILED or ORCHESTRATION_ERROR status on failure
|
||||
|
||||
### Step 5: Test End-to-End
|
||||
- [x] Spawn pipeline with objective
|
||||
- [x] Verify report generation
|
||||
- [x] Verify auto-trigger to orchestration
|
||||
- [x] Verify parallel agent execution
|
||||
- [x] Verify results collection
|
||||
|
||||
### Demonstration Results (2026-01-24)
|
||||
Successfully tested with `pipeline-mksufe23`:
|
||||
- Pipeline spawned → ALPHA/BETA ran → Report generated → Auto-orchestration triggered
|
||||
- GAMMA spawned due to complexity (0.8 threshold)
|
||||
- Total orchestration time: 51.4 seconds
|
||||
- Final status: COMPLETED
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Test Command
|
||||
```bash
|
||||
# 1. Start UI server
|
||||
cd /opt/agent-governance/ui && bun run server.ts
|
||||
|
||||
# 2. Spawn pipeline via API
|
||||
curl -X POST http://localhost:3000/api/spawn \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"objective": "Design a caching strategy", "auto_continue": true}'
|
||||
|
||||
# 3. Watch WebSocket for events
|
||||
# Pipeline should: SPAWN → RUNNING → REPORT → ORCHESTRATE → COMPLETE
|
||||
```
|
||||
|
||||
### Validation Criteria
|
||||
- [x] Pipeline reaches ORCHESTRATION phase automatically
|
||||
- [x] Both ALPHA and BETA agents spawn
|
||||
- [x] Agents communicate via MessageBus
|
||||
- [x] Results appear in Blackboard
|
||||
- [x] Final checkpoint created
|
||||
- [x] Audit trail in SQLite
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If orchestration fails repeatedly:
|
||||
1. Set `AUTO_ORCHESTRATE=false`
|
||||
2. Pipeline will stop at REPORT phase
|
||||
3. Manual intervention can trigger orchestration
|
||||
4. Review logs in `/api/pipeline/logs`
|
||||
|
||||
---
|
||||
|
||||
*Document Version: 1.0*
|
||||
*Last Updated: 2026-01-24*
|
||||
+17
-7
@@ -2,13 +2,18 @@
|
||||
|
||||
## Current Phase
|
||||
|
||||
** NOT STARTED**
|
||||
**COMPLETE**
|
||||
|
||||
## Tasks
|
||||
|
||||
| Status | Task | Updated |
|
||||
|--------|------|---------|
|
||||
| ☐ | *No tasks defined* | - |
|
||||
| ✓ | ARCHITECTURE.md | 2026-01-23 |
|
||||
| ✓ | CONTEXT_MANAGEMENT.md | 2026-01-23 |
|
||||
| ✓ | MEMORY_LAYER.md | 2026-01-23 |
|
||||
| ✓ | STATUS_PROTOCOL.md | 2026-01-23 |
|
||||
| ✓ | ENGINEERING_GUIDE.md | 2026-01-23 |
|
||||
| ✓ | CREDENTIALS_SETUP.md | 2026-01-24 |
|
||||
|
||||
## Dependencies
|
||||
|
||||
@@ -20,23 +25,28 @@
|
||||
|
||||
## Activity Log
|
||||
|
||||
### 2026-01-24 04:05:00 UTC
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: Added CREDENTIALS_SETUP.md for integration credentials
|
||||
- **Details**: Comprehensive guide for Slack, GitHub, PagerDuty credential setup including Vault commands, environment variables, and verification steps.
|
||||
|
||||
### 2026-01-23 23:45:57 UTC
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: Documentation complete: CONTEXT_MANAGEMENT.md, updated README, MEMORY_LAYER.md, STATUS_PROTOCOL.md
|
||||
- **Details**: Phase updated to complete
|
||||
|
||||
### 2026-01-23 23:32:09 UTC
|
||||
- **Phase**: IN_PROGRESS
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: Added checkpoint integration docs
|
||||
- **Details**: Phase updated to in_progress
|
||||
- **Details**: Documentation development started
|
||||
|
||||
### 2026-01-23 23:25:31 UTC
|
||||
- **Phase**: IN_PROGRESS
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: STATUS_PROTOCOL.md added, documentation ongoing
|
||||
- **Details**: Phase updated to in_progress
|
||||
- **Details**: Documentation development started
|
||||
|
||||
### 2026-01-23 23:25:09 UTC
|
||||
- **Phase**: NOT STARTED
|
||||
- **Phase**: COMPLETE
|
||||
- **Action**: Initialized
|
||||
- **Details**: Status tracking initialized for this directory.
|
||||
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
# UI Refresh Analysis
|
||||
|
||||
> Pre-implementation review for UI overhaul based on Phase 8 architecture changes.
|
||||
|
||||
**Generated:** 2026-01-24
|
||||
**Checkpoint:** ckpt-20260124-053550-6489cce8
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Review Summary
|
||||
|
||||
### Core Components (from ARCHITECTURE.md)
|
||||
|
||||
| Layer | Components | Status |
|
||||
|-------|------------|--------|
|
||||
| **Governance** | Vault, DragonflyDB, SQLite Ledger | ✅ Operational |
|
||||
| **Orchestration** | Multi-Agent Orchestrator, Pipeline, Model Controller | ✅ Complete |
|
||||
| **Agent** | Alpha/Beta/Gamma agents, Blackboard, MessageBus | ✅ Complete |
|
||||
| **Infrastructure** | OpenRouter, Bun Runtime, WireGuard | ✅ Complete |
|
||||
|
||||
### Key Enums (from pipeline/core.py)
|
||||
|
||||
```python
|
||||
# Agent Lifecycle Phases
|
||||
AgentPhase: BOOTSTRAP → PREFLIGHT → PLAN → EXECUTE → VERIFY → PACKAGE → REPORT → EXIT
|
||||
|
||||
# Agent Status
|
||||
AgentStatus: PENDING | STARTING | RUNNING | PAUSED | COMPLETED | FAILED | REVOKED | RECOVERING
|
||||
|
||||
# Pipeline Stage Types
|
||||
StageType: AGENT | GATE | PARALLEL | CONDITION
|
||||
|
||||
# Violation Types (14 types with severity mapping)
|
||||
ViolationType: EXECUTE_WITHOUT_PLAN | UNAUTHORIZED_API | TIER_VIOLATION | etc.
|
||||
|
||||
# Integration Event Types (9 types)
|
||||
IntegrationEventType: plan_created | execution_started | violation_detected | etc.
|
||||
```
|
||||
|
||||
### Context Management (from CONTEXT_MANAGEMENT.md)
|
||||
|
||||
| Component | Purpose | Token Impact |
|
||||
|-----------|---------|--------------|
|
||||
| Checkpoints | Session state snapshots | ~3000 tokens |
|
||||
| STATUS Files | Per-directory tracking | ~50 tokens each |
|
||||
| Memory Layer | Large output storage | Minimal (refs only) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Status Sweep Results
|
||||
|
||||
### Directory Summary
|
||||
|
||||
| Metric | Count |
|
||||
|--------|-------|
|
||||
| **Total directories tracked** | 64 |
|
||||
| **COMPLETE** | 20 |
|
||||
| **UNKNOWN/Pending tasks** | 44 |
|
||||
| **Completed tasks** | 97 |
|
||||
| **Pending tasks** | 49 |
|
||||
|
||||
### Directories with Pending Tasks (49 total)
|
||||
|
||||
**High Priority (Core Systems):**
|
||||
- `.` (root) - 1 pending (webhook config deferred)
|
||||
- `integrations/` - 1 pending (production credentials)
|
||||
- `runtime/` - 1 pending
|
||||
- `pipeline/` - 1 pending
|
||||
- `orchestrator/` - 1 pending
|
||||
- `ui/` - 1 pending (this overhaul)
|
||||
|
||||
**Test Infrastructure:**
|
||||
- `tests/unit/`, `tests/integration/`, `tests/chaos/` - 1 each
|
||||
- `tests/typescript/`, `tests/python/` - 1 each
|
||||
|
||||
**Agent Configs:**
|
||||
- `agents/tier0-agent/config`, `agents/tier1-agent/config` - 1 each
|
||||
|
||||
---
|
||||
|
||||
## 3. Changes Since UI Was Last Updated
|
||||
|
||||
### UI Last Update: 2026-01-23 (STATUS.md initialized)
|
||||
|
||||
### New Features Added After UI Creation
|
||||
|
||||
| Feature | Location | Description |
|
||||
|---------|----------|-------------|
|
||||
| **Full Agent Lifecycle** | `pipeline/core.py` | 8-phase lifecycle (BOOTSTRAP→EXIT) |
|
||||
| **Auto-Execution System** | `ui/server.ts:724-905` | Confidence-based auto-exec with dry-run |
|
||||
| **Approval Workflow** | `ui/server.ts:907-1067` | Queue-based plan approval |
|
||||
| **Plan Execution** | `ui/server.ts:1079-1642` | Step-by-step execution with rollback |
|
||||
| **Verification Phase** | `ui/server.ts` | Post-execution verification |
|
||||
| **Package Phase** | `ui/server.ts` | Artifact bundling with checksums |
|
||||
| **Report Phase** | `ui/server.ts:1644-1800` | Human-readable execution reports |
|
||||
| **14 Violation Types** | `pipeline/core.py` | Full taxonomy with severity mapping |
|
||||
| **Integration Events** | `pipeline/core.py` | 9 event types for external systems |
|
||||
| **Memory Layer** | `memory/` | Token-efficient large output storage |
|
||||
| **Checkpoint System** | `checkpoint/` | Session state preservation |
|
||||
| **Testing Oversight** | `testing/oversight/` | Bug watcher, council, error injector |
|
||||
| **Tier 1 Agent** | `agents/tier1-agent/` | Full execution capabilities |
|
||||
|
||||
### New Integrations (Not in Original UI)
|
||||
|
||||
| Integration | Status | Features |
|
||||
|-------------|--------|----------|
|
||||
| **Slack** | ✅ 17/17 tests | Notifications, alerts, approvals |
|
||||
| **GitHub** | ✅ 10/10 tests | PR automation, status updates |
|
||||
| **PagerDuty** | ✅ 8/8 tests | Incident management, escalation |
|
||||
|
||||
### Architecture Changes
|
||||
|
||||
1. **Type Synchronization** - `pipeline/core.py` is now authoritative source for all enums
|
||||
2. **Violation Severity** - 4-level severity system (LOW/MEDIUM/HIGH/CRITICAL)
|
||||
3. **Phase-Output Mapping** - Alpha/Beta/Gamma output types linked to phases
|
||||
4. **Integration Event Lifecycle** - Events mapped to agent phases
|
||||
|
||||
---
|
||||
|
||||
## 4. UI Refresh Plan
|
||||
|
||||
### Current UI Capabilities (from server.ts analysis)
|
||||
|
||||
The current UI (3800+ lines) includes:
|
||||
- ✅ Agent state monitoring
|
||||
- ✅ Pipeline spawning and tracking
|
||||
- ✅ WebSocket real-time updates
|
||||
- ✅ Revocation tracking
|
||||
- ✅ Plan extraction and storage
|
||||
- ✅ Auto-execution queue
|
||||
- ✅ Approval workflow
|
||||
- ✅ Execution with dry-run
|
||||
- ✅ Verification phase
|
||||
- ✅ Package phase
|
||||
- ✅ Report generation
|
||||
- ✅ Ledger queries (SQLite)
|
||||
- ✅ DragonflyDB integration
|
||||
|
||||
### Missing UI Components (Must-Have)
|
||||
|
||||
| Component | Priority | Description |
|
||||
|-----------|----------|-------------|
|
||||
| **Checkpoint Dashboard** | HIGH | View/load/compare checkpoints |
|
||||
| **Memory Browser** | HIGH | Search/fetch stored outputs |
|
||||
| **STATUS Overview** | HIGH | 64-directory status grid |
|
||||
| **Integration Panel** | HIGH | Slack/GitHub/PagerDuty status |
|
||||
| **Violation Analytics** | MEDIUM | 14 types with severity breakdown |
|
||||
| **Tier Promotion View** | MEDIUM | T0→T4 progression tracking |
|
||||
| **Testing Oversight** | MEDIUM | Bug watcher, council view |
|
||||
| **Phase Timeline** | LOW | Visual agent phase progression |
|
||||
| **Dependency Graph** | LOW | Service health visualization |
|
||||
|
||||
### Recommended UI Structure
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ COMMAND BAR: [objective input] [spawn] [checkpoint] [status indicators] │
|
||||
├──────────────────┬────────────────────────────┬─────────────────────────┤
|
||||
│ PIPELINES │ MAIN VIEW │ CONTEXT PANEL │
|
||||
│ │ │ │
|
||||
│ • Active (3) │ [Tab: Pipelines | │ [Checkpoint Info] │
|
||||
│ • Completed │ Approvals | Agents | │ Phase: 8 │
|
||||
│ • Failed │ Status | Memory] │ Tasks: 97/146 │
|
||||
│ │ │ │
|
||||
│ ───────────── │ Pipeline: pipe-abc123 │ [Dependencies] │
|
||||
│ CHECKPOINTS │ ┌─────────────────────┐ │ ✓ Vault │
|
||||
│ • Latest │ │ Agent A ──► Agent B │ │ ✓ Dragonfly │
|
||||
│ • History │ │ └─► Agent C │ │ ✓ Ledger │
|
||||
│ │ └─────────────────────┘ │ │
|
||||
│ ───────────── │ │ [Integrations] │
|
||||
│ INTEGRATIONS │ [Execution Log] │ ⚪ Slack (no creds) │
|
||||
│ • Slack ⚪ │ [Plan Details] │ ⚪ GitHub (no creds) │
|
||||
│ • GitHub ⚪ │ [Approval Queue] │ ⚪ PagerDuty (no creds│
|
||||
│ • PagerDuty ⚪ │ │ │
|
||||
├──────────────────┴────────────────────────────┴─────────────────────────┤
|
||||
│ STATUS BAR: [WS: connected] [Agents: 0] [Pending: 3] [Last: 04:57 UTC] │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### New Tabs/Views to Add
|
||||
|
||||
1. **Status Grid** - 64 directories in grid with phase icons
|
||||
2. **Checkpoint Manager** - Load, compare, timeline view
|
||||
3. **Memory Browser** - List, search, fetch with summary preview
|
||||
4. **Violation Dashboard** - Charts by type/severity/time
|
||||
5. **Integration Config** - Credential status, test connections
|
||||
|
||||
### API Endpoints to Add
|
||||
|
||||
```typescript
|
||||
// Checkpoint endpoints
|
||||
GET /api/checkpoints // List checkpoints
|
||||
GET /api/checkpoints/:id // Load specific
|
||||
GET /api/checkpoints/diff // Compare two
|
||||
POST /api/checkpoints // Create new
|
||||
|
||||
// Memory endpoints
|
||||
GET /api/memory // List entries
|
||||
GET /api/memory/:id // Fetch entry
|
||||
GET /api/memory/:id/summary // Summary only
|
||||
GET /api/memory/search // Search content
|
||||
|
||||
// Status endpoints
|
||||
GET /api/status // All directories
|
||||
GET /api/status/:dir // Specific directory
|
||||
POST /api/status/:dir // Update status
|
||||
|
||||
// Integration endpoints
|
||||
GET /api/integrations // All integration status
|
||||
POST /api/integrations/:name/test // Test connection
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Missing/Outdated Documentation
|
||||
|
||||
### Missing Documentation
|
||||
|
||||
| Document | Priority | Content Needed |
|
||||
|----------|----------|----------------|
|
||||
| `docs/UI_GUIDE.md` | HIGH | User guide for dashboard |
|
||||
| `docs/API_REFERENCE.md` | HIGH | REST/WebSocket API docs |
|
||||
| `ui/README.md` | HIGH | Setup and development guide |
|
||||
| `docs/TROUBLESHOOTING.md` | MEDIUM | Common issues and fixes |
|
||||
| `docs/DEPLOYMENT.md` | MEDIUM | Production deployment guide |
|
||||
|
||||
### Outdated References
|
||||
|
||||
| File | Issue | Fix Needed |
|
||||
|------|-------|------------|
|
||||
| `docs/ENGINEERING_GUIDE.md` | Missing checkpoint/memory CLI | Add new commands |
|
||||
| `docs/ARCHITECTURE.md` | Version 0.2.0, should be 1.0 | Update version |
|
||||
| `ui/STATUS.md` | Shows "No tasks defined" | Add UI tasks |
|
||||
| `integrations/STATUS.md` | Still shows pending webhook | Update to deferred |
|
||||
|
||||
### STATUS.md Files Needing Phase Update
|
||||
|
||||
44 directories show `UNKNOWN` phase due to inconsistent format:
|
||||
- Need to standardize `**COMPLETE**` vs `** COMPLETE**` (note space)
|
||||
- Should run `status sweep --fix` after UI update
|
||||
|
||||
---
|
||||
|
||||
## 6. Implementation Priorities
|
||||
|
||||
### Phase 1: Core UI Updates (Critical)
|
||||
1. Add Checkpoint tab with load/compare
|
||||
2. Add Memory browser with search
|
||||
3. Add Status overview grid
|
||||
4. Fix integration status display
|
||||
|
||||
### Phase 2: Enhanced Features (High)
|
||||
5. Add violation analytics charts
|
||||
6. Add tier promotion tracking
|
||||
7. Add testing oversight view
|
||||
8. Update API documentation
|
||||
|
||||
### Phase 3: Polish (Medium)
|
||||
9. Visual phase timeline
|
||||
10. Dependency graph visualization
|
||||
11. Mobile-responsive layout
|
||||
12. Keyboard shortcuts
|
||||
|
||||
---
|
||||
|
||||
## 7. Pre-Implementation Checklist
|
||||
|
||||
- [x] Architecture docs reviewed
|
||||
- [x] Status sweep completed
|
||||
- [x] Changes since UI update cataloged
|
||||
- [x] Missing components identified
|
||||
- [x] Missing documentation identified
|
||||
- [ ] UI component mockups approved
|
||||
- [ ] API endpoint design approved
|
||||
- [ ] Test plan created
|
||||
|
||||
---
|
||||
|
||||
*Generated for UI refresh planning. Checkpoint: ckpt-20260124-053550-6489cce8*
|
||||
Reference in New Issue
Block a user