profit/golangLAKEHOUSE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

H: observerd — autonomous-iteration witness loop (SPEC §2 port)

Browse Source 
Port of the load-bearing pieces of mcp-server/observer.ts (Rust
system, 852 lines TS) per SPEC §2's named target. Implements PRD
loop 3 ("Observer loop — watches each run, refines configs").

Routes (all under /v1/observer/* via gateway):
  GET  /observer/health   — liveness
  GET  /observer/stats    — total / successes / failures /
                             by_source / recent_scenario_ops
                             (matches Rust JSON shape exactly)
  POST /observer/event    — record one ObservedOp; auto-defaults
                             timestamp + source, validates required
                             fields (endpoint), persists to JSONL,
                             appends to ring buffer

Architecture:
  - internal/observer/types.go — ObservedOp model + Source taxonomy
    (mcp / scenario / langfuse / overseer_correction). Mirrors the
    Rust shape so JSON round-trips during cutover.
  - internal/observer/store.go — Store + Persistor. Ring buffer cap
    matches Rust's 2000; recent_scenarios cap matches Rust's 10.
    Same persist-then-apply order as pathwayd; same corruption-
    tolerant replay (skip malformed lines + warn).
  - cmd/observerd — :3219 HTTP service, fronted by gateway as
    /v1/observer/*.
  - lakehouse.toml + DefaultConfig — [observerd] block matches the
    pathwayd pattern (Bind + PersistPath; empty path = ephemeral).

Tests + smoke (all PASS):
  - 7 unit tests in store_test.go: validation, default fields,
    stats aggregation, recent-scenarios cap + ordering, ring-buffer
    rollover at cap, JSONL round-trip persistence, corruption-
    tolerant replay (1 valid + 1 corrupt + 1 valid → 2 applied)
  - scripts/observer_smoke.sh: 4 assertions through gateway —
    record 5 events (3 ok / 2 fail across 2 sources), stats
    aggregates correctly, empty-endpoint→400, kill+restart preserves
    via JSONL replay (5 ops, 3 ok, 2 err survive)

Deferred (named in package + cmd doc, not in this commit):
  - POST /observer/review (cloud-LLM hand-review fall-back). The
    heuristic-only path could land cheaply but the productized
    cloud path (qwen3-coder fall-back) is multi-day port.
  - Background loops: analyzeErrors, consolidatePlaybooks,
    tailOverseerCorrections (read overseer_corrections.jsonl into
    the ring buffer once per cycle).
  - escalateFailureClusterToLLMTeam (failure clustering trigger
    that posts to LLM Team's /api/run with code_review mode).

/relevance is NOT duplicated — already ported in 9588bd8 to
internal/matrix/relevance.go (component 3 of SPEC §3.4).

16-smoke regression all green (D1-D6, G1, G1P, G2, storaged_cap,
pathway, matrix, relevance, downgrade, playbook, observer).
13 binaries now: gateway, storaged, catalogd, ingestd, queryd,
vectord, embedd, pathwayd, matrixd, observerd, mcpd, fake_ollama
(plus catalogd-only test build).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-29 20:18:02 -05:00
co-authored by Claude Opus 4.7
parent 6392772f41
commit bc9ab93afe
8 changed files with 891 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/observer/types.go
+131
View File
@@ -0,0 +1,131 @@
// Package observer is the Go port of mcp-server/observer.ts (Rust
// system, 852 lines TS) — the "third-party witness" loop that records
// every observed operation, surfaces failures, and feeds learnings
// back into the substrate.
//
// What this package owns (this commit):
// - ObservedOp data model + ring buffer + JSONL persistence
// - Stats aggregation (total / successes / failures / by_source)
// - Source taxonomy (mcp / scenario / langfuse / overseer_correction)
//
// What's deferred to follow-up commits:
// - /review endpoint with cloud-LLM hand-review (the heuristic
// plus qwen3-coder fall-back path)
// - tailOverseerCorrections (background loop reading
// overseer_corrections.jsonl)
// - analyzeErrors / consolidatePlaybooks periodic loops
// - escalateFailureClusterToLLMTeam (failure clustering trigger)
//
// /relevance was already ported in 9588bd8 (component 3 of SPEC §3.4)
// and lives in internal/matrix/relevance.go; the observer package
// doesn't re-implement it.
package observer
import (
"errors"
"time"
)
// Source is the provenance of an observed op. Empty string defaults
// to SourceMCP for back-compat with Phase 24 callers.
type Source string
const (
SourceMCP Source = "mcp"
SourceScenario Source = "scenario"
SourceLangfuse Source = "langfuse"
SourceOverseerCorrection Source = "overseer_correction"
)
// ObservedOp is one entry in the observer's ring buffer (and JSONL
// log when persistence is configured). Mirrors the Rust ObservedOp
// shape exactly so the on-wire JSON round-trips between the two
// implementations during the Rust→Go cutover.
//
// Optional fields use omitempty so absent values don't bloat the
// JSONL file. Numeric zero values are intentionally treated as
// "not set" by the JSON layer; if a real zero needs to be
// persisted, future schema-version bump can switch to pointers.
type ObservedOp struct {
Timestamp string `json:"timestamp"` // ISO 8601
Endpoint string `json:"endpoint"`
InputSummary string `json:"input_summary"`
Success bool `json:"success"`
DurationMs int64 `json:"duration_ms"`
OutputSummary string `json:"output_summary"`
Error string `json:"error,omitempty"`
Source Source `json:"source,omitempty"`
StafferID string `json:"staffer_id,omitempty"`
SigHash string `json:"sig_hash,omitempty"`
EventKind string `json:"event_kind,omitempty"`
Role string `json:"role,omitempty"`
City string `json:"city,omitempty"`
State string `json:"state,omitempty"`
Count int `json:"count,omitempty"`
RescueAttempted bool `json:"rescue_attempted,omitempty"`
RescueSucceeded bool `json:"rescue_succeeded,omitempty"`
TaskClass string `json:"task_class,omitempty"`
Correction string `json:"correction,omitempty"`
AppliedAtTurn int `json:"applied_at_turn,omitempty"`
}
// Stats is the aggregated view of the ring buffer — useful for
// dashboards and the GET /stats endpoint. RecentScenarios holds the
// most recent N source=scenario ops (default cap 10) so operators
// can see what the staffing scenarios are emitting at a glance.
type Stats struct {
Total int `json:"total"`
Successes int `json:"successes"`
Failures int `json:"failures"`
BySource map[string]int `json:"by_source"`
RecentScenarios []ScenarioOpDigest `json:"recent_scenario_ops"`
}
// ScenarioOpDigest is the slim per-op shape returned in
// Stats.RecentScenarios — matches the TS digest exactly:
// {ts, ok, staffer, kind, role}.
type ScenarioOpDigest struct {
TS string `json:"ts"`
OK bool `json:"ok"`
Staffer string `json:"staffer"`
Kind string `json:"kind"`
Role string `json:"role"`
}
// Errors surfaced to HTTP handlers.
var (
ErrInvalidOp = errors.New("observer: invalid op (timestamp + endpoint required)")
)
// Validate returns an error if required fields are missing. Called
// by Record before the op is added to the ring buffer.
func (op ObservedOp) Validate() error {
if op.Timestamp == "" {
return ErrInvalidOp
}
if op.Endpoint == "" {
return ErrInvalidOp
}
return nil
}
// EnsureTimestamp populates Timestamp with the current UTC ISO 8601
// time if it's empty. Useful for HTTP handlers that take the body
// as authoritative but need to default the timestamp when absent.
func (op *ObservedOp) EnsureTimestamp() {
if op.Timestamp == "" {
op.Timestamp = time.Now().UTC().Format(time.RFC3339)
}
}
// DefaultSource sets Source to SourceMCP if empty. Mirrors the Rust
// `op.source ?? "mcp"` pattern in recordExternalOp.
func (op *ObservedOp) DefaultSource() {
if op.Source == "" {
op.Source = SourceMCP
}
}