Compare commits

...
Author SHA1 Message Date
profitandClaude Opus 4.7 6e39d8778f Cohesion: Python inventory + integration plan + Phase A verdict indexing
lakehouse/auditor all checks passed (3 findings, all info)
Three artifacts in one PR:

1. docs/PYTHON_INVENTORY.md — every .py file in the repo classified:
   Production (sidecar routers + 3 systemd services), Documented
   (kb_measure, kb_staffer_report), Manual (one-off tools), Dead
   (sidecar/sidecar/lab_ui.py + pipeline_lab.py are genuinely
   not imported anywhere).

2. docs/COHESION_INTEGRATION_PLAN.md — the "smarter DB" loop J
   called out as missing. Six phases A-F. Phase A ships here; B-F
   are named + sequenced for follow-up PRs. Each phase adds ONE
   wire of the loop; no single PR does them all.

3. Phase A wire (auditor verdicts → observer + KB):
   - auditor/audit.ts: after assembleVerdict, fire-and-forget POST
     to :3800/event with source="auditor" AND append to
     data/_kb/outcomes.jsonl with kind="audit". Errors log + drop
     — the verdict is still on disk at _auditor/verdicts/.
   - mcp-server/observer.ts: extend source union to include
     "auditor" | "bot" (was "mcp" | "scenario" only, which silently
     coerced my first auditor POST to source="scenario"). Accept
     body.ok OR body.success. Accept body.audit_duration_ms as a
     fallback for duration_ms. Uses body.one_liner as
     output_summary when set.

Live-verified after observer restart:
   re-audit PR #6 → verdict=request_changes, 4 findings (1 warn)
     observer: by_source={'auditor': 1}  (previously coerced to 'scenario')
     _kb/outcomes.jsonl tail: kind=audit sig=pr6-7fe47bab
       pr=6 overall=request_changes

The shape of the loop is now visible to downstream consumers. Phase
B (auditor's kb_query check reads these audit rows for history)
lands in a follow-up PR. Phase C-F similar.

NOT in this PR:
- Actually deleting lab_ui.py + pipeline_lab.py (operator decision,
  called out in the inventory doc)
- Cleaning up the 5 overlapping Python scripts (same)
- Phases B-F of the cohesion plan (separate PRs per wire)
- Integration test that asserts "smarter DB" across runs (Phase F)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 17:22:42 -05:00
profit 6d7b251607 Merge pull request 'Phase 45 slice 3: doc_drift check + resolve endpoints' (#5) from phase/45-slice-3 into main 2026-04-22 19:14:11 +00:00
profitandClaude Opus 4.7 8bacd43465 Phase 45 slice 3: doc_drift check + resolve endpoints
lakehouse/auditor cloud: claim not backed — "Previously the hybrid fixture honestly reported layer 5 as 404/unimplemented. With this PR it flips "
Closes the last open loop of Phase 45. Previously, playbooks could
carry doc_refs (slice 1) and the context7 bridge could report drift
(slice 2) — but nothing tied them together. An operator had no way
to say "check this playbook against its doc sources and flag it if
the docs moved." This slice wires that.

Ships:
- crates/vectord/src/doc_drift.rs — thin context7 bridge client.
  No cache (bridge has its own 5-min TTL). No retry (transient
  failure = Unknown outcome, caller decides).
- PlaybookMemory::flag_doc_drift(id) — stamps doc_drift_flagged_at
  idempotently. Once flagged, compute_boost_for_filtered_with_role
  excludes the entry from both the non-geo and geo-indexed boost
  paths until resolved.
- PlaybookMemory::resolve_doc_drift(id) — human re-admission.
  Stamps doc_drift_reviewed_at which clears the boost exclusion.
- PlaybookMemory::get_entry(id) — new read-only accessor the
  handler uses to read doc_refs without exposing the state lock.
- POST /vectors/playbook_memory/doc_drift/check/{id}
- POST /vectors/playbook_memory/doc_drift/resolve/{id}

Design call: Unknown outcomes from the bridge (bridge down, tool
not in context7, no snippet_hash recorded) are NEVER enough to
flag. Only a positive drifted=true from the bridge flips the flag.
A down bridge doesn't silently drift-flag every playbook.

Tests (5 new, in upsert_tests mod):
- flag_doc_drift_stamps_timestamp_and_persists
- flag_doc_drift_is_idempotent_on_already_flagged
- resolve_doc_drift_clears_flag_admission_gate
- boost_excludes_flagged_unreviewed_entries
- boost_re_admits_resolved_entries
14/14 upsert tests pass (9 pre-existing + 5 new).

Live end-to-end — hybrid fixture on auditor/scaffold (merged to
main at b6d69b2) now shows:

  overall: PASS
  shipped: [38, 40, 45.1, 45.2, 45.3]
  placeholder: [—]
  ✓ Phase 38    /v1/chat              4039ms
  ✓ Phase 40    Langfuse trace          11ms
  ✓ Phase 45.1  seed + doc_refs        748ms
  ✓ Phase 45.2  bridge diff            563ms
  ✓ Phase 45.3  drift-check endpoint   116ms ← was a 404 before this

First time the fixture reports overall=PASS with zero placeholder
layers. The honest "not built" signal on layer 5 is now honestly
"built and working."

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:12:57 -05:00
profit e57ab8ad01 Merge pull request 'ops: systemd units for auditor + context7 bridge' (#4) from ops/auditor-systemd-units into main 2026-04-22 09:17:09 +00:00
profit c85c55006d ops: systemd units for auditor + context7 bridge
lakehouse/auditor 3 warnings — see review
Promotes two previously manual-start Bun services to systemd
so they survive restarts + run continuously.

- ops/systemd/lakehouse-auditor.service — polls Gitea every 90s,
  runs 4 audit checks per PR head SHA, posts commit status + review
  comment. Runs as root to match existing lakehouse-* service
  conventions on this host; can read /home/profit/.git-credentials
  (0600 profit:profit).
- ops/systemd/lakehouse-context7-bridge.service — HTTP wrapper on
  :3900 for Phase 45 doc-drift detection. Decoupled from gateway;
  runs independently.
- ops/systemd/install.sh — idempotent installer (copy → daemon-reload
  → enable --now). Prints post-install active/enabled status.
- ops/systemd/README.md — run/stop/logs/pause docs.

Pause control stays per-service (bot.paused / auditor.paused files
at repo root). Not wired to branch protection yet — the auditor's
commit status is currently advisory, not enforcing. Flip via Gitea
branch_protections API when confident.
2026-04-22 04:15:58 -05:00
profit b6d69b2e82 Merge pull request 'Auditor: PR-claim hard-block reviewer (scaffold)' (#1) from auditor/scaffold into main 2026-04-22 09:13:34 +00:00
profit b82caa9971 Merge pull request 'Fix: UPDATE branch of upsert_entry dropped doc_refs + valid_until' (#3) from fix/upsert-outcome-update-merge into main 2026-04-22 09:11:15 +00:00
profit 1270e167fe Post-merge: update test pattern matches for struct-like UpsertOutcome
After merging main (with the UpsertOutcome struct-like enum shape
from PR #2), the 4 new upsert tests needed pattern-match updates:
  UpsertOutcome::Added(_) → UpsertOutcome::Added { .. }

9/9 upsert tests pass.
2026-04-22 04:11:13 -05:00
profit 4dca2a6705 Merge branch 'main' of https://git.agentview.dev/profit/lakehouse into fix/upsert-outcome-update-merge 2026-04-22 04:10:27 -05:00
profit b667fdeff1 Fix: UpsertOutcome newtype serde panic (silent since Phase 26)
Auditor found this via hybrid fixture 2026-04-22. Blocks the serde-tag-newtype shape by converting to struct-like variants. See PR #2 body for full context.

Manual merge: auditor commit status was failure due to 1 false-positive inference finding on a commit-message reference; underlying fix is verified (curl against live gateway confirmed all 3 upsert paths return valid JSON). Proceeding per human review.
2026-04-22 09:10:07 +00:00
profitandClaude Opus 4.7 320009ddf4 Fix: UPDATE branch of upsert_entry dropped doc_refs + valid_until
lakehouse/auditor all checks passed (3 findings, all info)
The auditor's hybrid fixture (branch auditor/scaffold) surfaced this
on 2026-04-22. A re-seed of the same (operation, day) pair with new
endorsed_names merged the names but silently discarded the incoming
doc_refs and valid_until fields. schema_fingerprint was partially
handled (set-if-Some) but doc_refs and valid_until weren't touched.

Root cause: the UPDATE arm of upsert_entry at playbook_memory.rs:609
only covered:
  - endorsed_names (union-merge)
  - timestamp
  - embedding (if Some)
  - schema_fingerprint (if Some)

Fix:
  - valid_until — refresh if caller provides one
  - doc_refs — merge by tool (case-insensitive). Same-tool new entry
    supersedes older one; different-tool refs are appended. Empty
    incoming doc_refs preserves existing (don't wipe on partial seed).

4 new regression tests under upsert_tests:
  - update_merges_doc_refs_with_existing_ones
  - update_same_tool_supersedes_older_version
  - update_preserves_existing_doc_refs_when_new_entry_has_none
  - update_refreshes_valid_until_when_caller_provides_one

Test result: 9/9 upsert tests pass (4 new + 5 pre-existing).

Branch basis note: this branch is off main, so the UpsertOutcome enum
here still has the newtype variants Added(String) / Noop(String). PR
#2 (fix/upsert-outcome-serde) changes that enum to struct-like. When
PR #2 merges first this branch needs a trivial rebase; the UPDATE
arm logic is untouched by that change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 04:06:54 -05:00
profit f0a3ed6832 Fix: UpsertOutcome newtype variants panicked serde from Phase 26
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "Verified live after gateway restart:"
playbook_memory.rs:257 — UpsertOutcome had two newtype variants
carrying a bare String:
  Added(String)
  Noop(String)
under #[serde(tag = "mode")]. serde cannot tag newtype variants of
primitive types, so every serialization threw:
  "cannot serialize tagged newtype variant UpsertOutcome::Added
   containing a string"
This caused gateway /vectors/playbook_memory/seed to panic the
tokio worker on EVERY call that reached Added or Noop, returning
an empty socket close to the client. The bug was silent from commit
640db8c (Phase 26, 2026-04-21) until 2026-04-22 when the auditor's
hybrid fixture (auditor/fixtures/hybrid_38_40_45.ts on the
auditor/scaffold branch) exercised the endpoint live and gateway
logs showed the panic.

Fix — convert both newtype variants to struct-like:
  Added { playbook_id: String }
  Noop { playbook_id: String }
Updated all 7 construction + pattern-match sites. Updated rustdoc
on the enum explaining why the shape is what it is.

JSON wire format is now uniform across all three variants:
  {"mode":"added","playbook_id":"pb-..."}
  {"mode":"updated","playbook_id":"pb-...","merged_names":[...]}
  {"mode":"noop","playbook_id":"pb-..."}

Verified live after gateway restart:
  curl /seed new payload               → mode=added, playbook 860231f5
  curl /seed new payload + doc_refs    → mode=added, playbook 11d348d9
  curl /seed identical re-submit       → mode=noop,  same id 860231f5,
                                         entries_after unchanged (Mem0
                                         contract intact)

Tests: 51/51 vectord lib tests green. Release build clean.

This is a follow-up bug fix landed in its own branch
(fix/upsert-outcome-serde) rather than commingled with other work.
The auditor's hybrid fixture on the auditor/scaffold branch will
now light up layer 3 (phase45_seed_with_doc_refs) as a pass once
this merges — previously it failed here with an empty socket close.
2026-04-22 03:48:05 -05:00
13 changed files with 1045 additions and 18 deletions
+72
View File
@@ -80,6 +80,16 @@ export async function auditPr(pr: PrSnapshot, opts: AuditOptions = {}): Promise<
await persistVerdict(verdict); await persistVerdict(verdict);
// Phase A of the cohesion plan (docs/COHESION_INTEGRATION_PLAN.md):
// make every audit verdict visible to the observer + KB. Enables
// future Phase B (kb_query sees prior audit history) without a
// separate backfill. Fire-and-forget: observer/KB failures don't
// block the Gitea post.
indexVerdictToObserver(verdict).catch(e =>
console.error(`[auditor] observer indexing failed: ${(e as Error).message}`));
appendVerdictToKbOutcomes(verdict).catch(e =>
console.error(`[auditor] kb outcomes append failed: ${(e as Error).message}`));
if (!opts.dry_run) { if (!opts.dry_run) {
await postToGitea(verdict); await postToGitea(verdict);
} }
@@ -87,6 +97,68 @@ export async function auditPr(pr: PrSnapshot, opts: AuditOptions = {}): Promise<
return verdict; return verdict;
} }
// Phase A — verdict indexing.
//
// Two destinations, both append-only + non-blocking:
// 1. observer :3800/event — ring buffer + data/_observer/ops.jsonl
// 2. data/_kb/outcomes.jsonl — same file scenarios write to, with
// kind:"audit" so readers can filter
//
// Errors log + drop. The verdict is still on disk at
// _auditor/verdicts/{pr}-{sha}.json; observer + KB are a convenience
// surface, not a source of truth.
const OBSERVER_URL = process.env.LH_OBSERVER_URL ?? "http://localhost:3800";
const KB_OUTCOMES = "/home/profit/lakehouse/data/_kb/outcomes.jsonl";
async function indexVerdictToObserver(v: Verdict): Promise<void> {
const payload = {
source: "auditor",
event_kind: "audit",
ok: v.overall === "approve",
sig_hash: `pr${v.pr_number}-${v.head_sha.slice(0, 8)}`,
pr_number: v.pr_number,
head_sha: v.head_sha,
overall: v.overall,
one_liner: v.one_liner,
findings_block: v.metrics.findings_block,
findings_warn: v.metrics.findings_warn,
audit_duration_ms: v.metrics.audit_duration_ms,
audited_at: v.audited_at,
};
const r = await fetch(`${OBSERVER_URL}/event`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(payload),
signal: AbortSignal.timeout(3000),
});
if (!r.ok) throw new Error(`observer ${r.status}: ${await r.text()}`);
}
async function appendVerdictToKbOutcomes(v: Verdict): Promise<void> {
const { appendFile, mkdir } = await import("node:fs/promises");
const { dirname } = await import("node:path");
await mkdir(dirname(KB_OUTCOMES), { recursive: true });
const row = {
kind: "audit",
sig_hash: `pr${v.pr_number}-${v.head_sha.slice(0, 8)}`,
audited_at: v.audited_at,
pr_number: v.pr_number,
head_sha: v.head_sha,
overall: v.overall,
one_liner: v.one_liner,
ok_events: v.overall === "approve" ? 1 : 0,
total_events: 1,
findings: {
block: v.metrics.findings_block,
warn: v.metrics.findings_warn,
info: v.metrics.findings_info,
},
elapsed_secs: (v.metrics.audit_duration_ms ?? 0) / 1000,
};
await appendFile(KB_OUTCOMES, JSON.stringify(row) + "\n");
}
async function persistVerdict(v: Verdict): Promise<void> { async function persistVerdict(v: Verdict): Promise<void> {
await mkdir(VERDICTS_DIR, { recursive: true }); await mkdir(VERDICTS_DIR, { recursive: true });
const filename = `${v.pr_number}-${v.head_sha.slice(0, 12)}.json`; const filename = `${v.pr_number}-${v.head_sha.slice(0, 12)}.json`;
+1
View File
@@ -26,3 +26,4 @@ chrono = { workspace = true }
instant-distance = { workspace = true } instant-distance = { workspace = true }
uuid = { workspace = true } uuid = { workspace = true }
sha2 = { workspace = true } sha2 = { workspace = true }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
+166
View File
@@ -0,0 +1,166 @@
// Phase 45 slice 3 — context7 bridge client for doc-drift detection.
//
// Calls the Bun context7 bridge on :3900 (mcp-server/context7_bridge.ts)
// which itself wraps context7's public API. For each DocRef on a
// playbook, queries /docs/:tool/diff?since=<snippet_hash> and parses
// the response.
//
// Kept deliberately thin: no caching here (bridge caches for 5 min),
// no retry logic (transient fail = report as "unknown" drift status,
// don't flag). The handler decides whether to flag the playbook based
// on whether ANY tool came back drifted=true.
use serde::Deserialize;
use std::time::Duration;
use crate::playbook_memory::DocRef;
const DEFAULT_BRIDGE_URL: &str = "http://localhost:3900";
const CALL_TIMEOUT_SECS: u64 = 15;
#[derive(Debug, Clone)]
pub struct DriftCheck {
pub tool: String,
pub version_seen: String,
pub outcome: DriftOutcome,
}
#[derive(Debug, Clone)]
pub enum DriftOutcome {
/// Bridge reports drift (current snippet_hash != previous).
Drifted { current_snippet_hash: String, source_url: Option<String> },
/// Bridge reports no drift — the stored snippet_hash still matches
/// the current context7 docs.
Unchanged,
/// Bridge unreachable, 404 on the tool, or returned unparseable
/// data. We deliberately don't flag on these — a down bridge
/// shouldn't silently mark every playbook drift-flagged.
Unknown { reason: String },
}
#[derive(Debug, Clone, Deserialize)]
struct BridgeDiffResponse {
drifted: bool,
current_snippet_hash: Option<String>,
source_url: Option<String>,
}
pub struct DriftCheckerConfig {
pub bridge_url: String,
}
impl Default for DriftCheckerConfig {
fn default() -> Self {
Self {
bridge_url: std::env::var("LH_BRIDGE_URL")
.unwrap_or_else(|_| DEFAULT_BRIDGE_URL.to_string()),
}
}
}
/// For every doc_ref, ask the bridge whether it drifted against the
/// recorded snippet_hash. Returns per-tool outcomes.
pub async fn check_all_refs(
cfg: &DriftCheckerConfig,
doc_refs: &[DocRef],
) -> Vec<DriftCheck> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(CALL_TIMEOUT_SECS))
.build()
.expect("reqwest client build");
let mut out = Vec::with_capacity(doc_refs.len());
for r in doc_refs {
let hash = r.snippet_hash.as_deref().unwrap_or("");
if hash.is_empty() {
// No hash to compare against — can't detect drift. Report
// unknown so the caller isn't forced to flag.
out.push(DriftCheck {
tool: r.tool.clone(),
version_seen: r.version_seen.clone(),
outcome: DriftOutcome::Unknown {
reason: "no snippet_hash recorded on doc_ref".into(),
},
});
continue;
}
let url = format!(
"{}/docs/{}/diff?since={}",
cfg.bridge_url.trim_end_matches('/'),
urlencoding_minimal(&r.tool),
urlencoding_minimal(hash),
);
let outcome = match client.get(&url).send().await {
Err(e) => DriftOutcome::Unknown {
reason: format!("bridge unreachable: {}", e),
},
Ok(resp) => {
if resp.status() == reqwest::StatusCode::NOT_FOUND {
DriftOutcome::Unknown {
reason: format!("bridge 404 — no context7 library for tool '{}'", r.tool),
}
} else if !resp.status().is_success() {
let status = resp.status();
DriftOutcome::Unknown {
reason: format!("bridge {}: {}", status, resp.text().await.unwrap_or_default()),
}
} else {
match resp.json::<BridgeDiffResponse>().await {
Err(e) => DriftOutcome::Unknown {
reason: format!("bridge response parse: {}", e),
},
Ok(body) => {
if body.drifted {
DriftOutcome::Drifted {
current_snippet_hash: body.current_snippet_hash.unwrap_or_default(),
source_url: body.source_url,
}
} else {
DriftOutcome::Unchanged
}
}
}
}
}
};
out.push(DriftCheck {
tool: r.tool.clone(),
version_seen: r.version_seen.clone(),
outcome,
});
}
out
}
/// Minimal URL path+query encoder for ASCII tool names. We're not
/// pulling in the `urlencoding` crate for this single use — tool
/// names are short alphanumeric (docker/terraform/react), snippet
/// hashes are hex. Handles space + `/` + `?` + `&` defensively so
/// unusual tool names don't corrupt the URL.
fn urlencoding_minimal(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for b in s.bytes() {
match b {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => out.push(b as char),
_ => out.push_str(&format!("%{:02X}", b)),
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn urlencoding_handles_ascii_safe_chars_passthrough() {
assert_eq!(urlencoding_minimal("docker"), "docker");
assert_eq!(urlencoding_minimal("next.js"), "next.js");
}
#[test]
fn urlencoding_encodes_slash_and_space() {
assert_eq!(urlencoding_minimal("foo/bar"), "foo%2Fbar");
assert_eq!(urlencoding_minimal("foo bar"), "foo%20bar");
}
}
+1
View File
@@ -8,6 +8,7 @@ pub mod hnsw;
pub mod index_registry; pub mod index_registry;
pub mod jobs; pub mod jobs;
pub mod playbook_memory; pub mod playbook_memory;
pub mod doc_drift;
pub mod promotion; pub mod promotion;
pub mod refresh; pub mod refresh;
pub mod store; pub mod store;
+315 -12
View File
@@ -252,11 +252,21 @@ pub struct BoostEntry {
/// Phase 26 — what happened during an upsert. The seed endpoint /// Phase 26 — what happened during an upsert. The seed endpoint
/// returns this shape so the caller sees whether its write was a new /// returns this shape so the caller sees whether its write was a new
/// entry, a merge, or a dedup'd no-op. /// entry, a merge, or a dedup'd no-op.
///
/// SHAPE NOTE: `#[serde(tag = "mode")]` requires struct-like variants —
/// bare `Added(String)` and `Noop(String)` newtype variants would
/// panic serialization at runtime. That bug silently 500-ed every
/// /seed call from Phase 26 (commit 640db8c) until 2026-04-22 when the
/// auditor's hybrid fixture surfaced it. All three variants are now
/// struct-like, producing uniform JSON:
/// {"mode":"added","playbook_id":"pb-..."}
/// {"mode":"updated","playbook_id":"pb-...","merged_names":[...]}
/// {"mode":"noop","playbook_id":"pb-..."}
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
#[serde(tag = "mode", rename_all = "lowercase")] #[serde(tag = "mode", rename_all = "lowercase")]
pub enum UpsertOutcome { pub enum UpsertOutcome {
/// New playbook appended. Carries the new playbook_id. /// New playbook appended. Carries the new playbook_id.
Added(String), Added { playbook_id: String },
/// Existing same-day entry updated. Playbook_id unchanged; names /// Existing same-day entry updated. Playbook_id unchanged; names
/// merged (union, original order preserved, new names appended). /// merged (union, original order preserved, new names appended).
Updated { Updated {
@@ -265,7 +275,7 @@ pub enum UpsertOutcome {
}, },
/// Identical same-day entry already exists; nothing changed. /// Identical same-day entry already exists; nothing changed.
/// Returns the stable playbook_id so caller still has a reference. /// Returns the stable playbook_id so caller still has a reference.
Noop(String), Noop { playbook_id: String },
} }
/// Phase 27 — shape returned from `revise_entry`. Reports both ends of /// Phase 27 — shape returned from `revise_entry`. Reports both ends of
@@ -388,6 +398,64 @@ impl PlaybookMemory {
Ok(touched) Ok(touched)
} }
/// Phase 45 slice 3 — stamp `doc_drift_flagged_at` on a playbook.
/// Idempotent: if already flagged and not yet reviewed, this is a
/// no-op. Callers should check the return value: Ok(true) means we
/// made a new flag, Ok(false) means already flagged (unreviewed) or
/// playbook not found, Err means persist failed.
pub async fn flag_doc_drift(&self, playbook_id: &str) -> Result<bool, String> {
let mut touched = false;
{
let mut state = self.state.write().await;
for e in state.entries.iter_mut() {
if e.playbook_id == playbook_id {
// Skip if already flagged + unreviewed (avoid
// churning the timestamp on every re-check).
if e.doc_drift_flagged_at.is_some() && e.doc_drift_reviewed_at.is_none() {
break;
}
e.doc_drift_flagged_at = Some(chrono::Utc::now().to_rfc3339());
e.doc_drift_reviewed_at = None; // fresh flag clears any stale review
touched = true;
break;
}
}
}
if touched {
self.persist().await?;
}
Ok(touched)
}
/// Phase 45 slice 3 — stamp `doc_drift_reviewed_at` on a playbook,
/// re-admitting it to boost calculation. Idempotent.
pub async fn resolve_doc_drift(&self, playbook_id: &str) -> Result<bool, String> {
let mut touched = false;
{
let mut state = self.state.write().await;
for e in state.entries.iter_mut() {
if e.playbook_id == playbook_id && e.doc_drift_flagged_at.is_some()
&& e.doc_drift_reviewed_at.is_none()
{
e.doc_drift_reviewed_at = Some(chrono::Utc::now().to_rfc3339());
touched = true;
break;
}
}
}
if touched {
self.persist().await?;
}
Ok(touched)
}
/// Read-only: get a playbook entry by id. Used by the drift-check
/// handler to read doc_refs without exposing the full state lock.
pub async fn get_entry(&self, playbook_id: &str) -> Option<PlaybookEntry> {
let state = self.state.read().await;
state.entries.iter().find(|e| e.playbook_id == playbook_id).cloned()
}
/// Phase 25 — retire every entry matching (city, state) whose /// Phase 25 — retire every entry matching (city, state) whose
/// schema_fingerprint doesn't match the current one. Entries with /// schema_fingerprint doesn't match the current one. Entries with
/// no fingerprint (legacy) are skipped — caller can use /// no fingerprint (legacy) are skipped — caller can use
@@ -597,7 +665,7 @@ impl PlaybookMemory {
drop(state); drop(state);
self.persist().await?; self.persist().await?;
self.rebuild_geo_index().await; self.rebuild_geo_index().await;
Ok(UpsertOutcome::Added(pid)) Ok(UpsertOutcome::Added { playbook_id: pid })
} }
Some(i) => { Some(i) => {
let mut existing_names_sorted = state.entries[i].endorsed_names.clone(); let mut existing_names_sorted = state.entries[i].endorsed_names.clone();
@@ -605,7 +673,7 @@ impl PlaybookMemory {
if existing_names_sorted == new_names_sorted { if existing_names_sorted == new_names_sorted {
// NOOP — identical data, just report the existing id // NOOP — identical data, just report the existing id
let pid = state.entries[i].playbook_id.clone(); let pid = state.entries[i].playbook_id.clone();
Ok(UpsertOutcome::Noop(pid)) Ok(UpsertOutcome::Noop { playbook_id: pid })
} else { } else {
// UPDATE — merge names (union, stable order). // UPDATE — merge names (union, stable order).
let existing = state.entries.get_mut(i).ok_or("index invalidated")?; let existing = state.entries.get_mut(i).ok_or("index invalidated")?;
@@ -615,15 +683,45 @@ impl PlaybookMemory {
} }
existing.endorsed_names = merged.clone(); existing.endorsed_names = merged.clone();
existing.timestamp = new_entry.timestamp.clone(); existing.timestamp = new_entry.timestamp.clone();
// Keep original playbook_id + embedding + schema fingerprint. // Keep original playbook_id. Refresh embedding only
// Refresh embedding only if the caller passed a non-None // if the caller passed a non-None one (indicates
// one (indicates the text shape changed). // the text shape changed).
if new_entry.embedding.is_some() { if new_entry.embedding.is_some() {
existing.embedding = new_entry.embedding.clone(); existing.embedding = new_entry.embedding.clone();
} }
// Lifecycle / drift-signal fields — refresh only
// when the caller provides a new value. Each was
// previously either ignored entirely (valid_until,
// doc_refs) or only partially handled. Caught by
// the auditor's hybrid fixture on 2026-04-22 when
// re-seeding with fresh doc_refs silently dropped
// them on the UPDATE path.
if new_entry.schema_fingerprint.is_some() { if new_entry.schema_fingerprint.is_some() {
existing.schema_fingerprint = new_entry.schema_fingerprint.clone(); existing.schema_fingerprint = new_entry.schema_fingerprint.clone();
} }
if new_entry.valid_until.is_some() {
existing.valid_until = new_entry.valid_until.clone();
}
if !new_entry.doc_refs.is_empty() {
// Merge by tool (case-insensitive). A new ref
// for the same tool supersedes the old one —
// caller is asserting "this is what I used
// now." Refs for tools the existing entry
// didn't track are appended. This keeps the
// cumulative tool history without letting a
// single re-seed erase it.
let mut merged_refs = existing.doc_refs.clone();
for new_ref in &new_entry.doc_refs {
let hit = merged_refs.iter().position(|r| {
r.tool.eq_ignore_ascii_case(&new_ref.tool)
});
match hit {
Some(idx) => merged_refs[idx] = new_ref.clone(),
None => merged_refs.push(new_ref.clone()),
}
}
existing.doc_refs = merged_refs;
}
let pid = existing.playbook_id.clone(); let pid = existing.playbook_id.clone();
drop(state); drop(state);
self.persist().await?; self.persist().await?;
@@ -745,6 +843,13 @@ impl PlaybookMemory {
.filter(|e| { .filter(|e| {
if e.retired_at.is_some() { return false; } if e.retired_at.is_some() { return false; }
if e.superseded_at.is_some() { return false; } if e.superseded_at.is_some() { return false; }
// Phase 45 slice 3 — entries flagged for doc drift and
// not yet human-reviewed are excluded from boost. The
// flag is the same strength of exclusion as retirement
// for this purpose; reviewing (via /resolve) re-admits.
if e.doc_drift_flagged_at.is_some() && e.doc_drift_reviewed_at.is_none() {
return false;
}
if let Some(vu) = &e.valid_until { if let Some(vu) = &e.valid_until {
if let Ok(parsed) = chrono::DateTime::parse_from_rfc3339(vu) { if let Ok(parsed) = chrono::DateTime::parse_from_rfc3339(vu) {
if now > parsed.with_timezone(&chrono::Utc) { return false; } if now > parsed.with_timezone(&chrono::Utc) { return false; }
@@ -775,6 +880,13 @@ impl PlaybookMemory {
.filter(|e| { .filter(|e| {
if e.retired_at.is_some() { return false; } if e.retired_at.is_some() { return false; }
if e.superseded_at.is_some() { return false; } if e.superseded_at.is_some() { return false; }
// Phase 45 slice 3 — same drift exclusion as the
// non-geo path above. Keeps the two filter paths
// consistent so a flagged entry is invisible to
// BOTH geo-targeted and global boost queries.
if e.doc_drift_flagged_at.is_some() && e.doc_drift_reviewed_at.is_none() {
return false;
}
if let Some(vu) = &e.valid_until { if let Some(vu) = &e.valid_until {
if let Ok(parsed) = chrono::DateTime::parse_from_rfc3339(vu) { if let Ok(parsed) = chrono::DateTime::parse_from_rfc3339(vu) {
if now > parsed.with_timezone(&chrono::Utc) { return false; } if now > parsed.with_timezone(&chrono::Utc) { return false; }
@@ -1600,7 +1712,7 @@ mod upsert_tests {
let pm = PlaybookMemory::new(Arc::new(InMemory::new())); let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let e = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith"]); let e = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith"]);
match pm.upsert_entry(e).await.unwrap() { match pm.upsert_entry(e).await.unwrap() {
UpsertOutcome::Added(_) => {} UpsertOutcome::Added { .. } => {}
other => panic!("expected Added, got {:?}", other), other => panic!("expected Added, got {:?}", other),
} }
assert_eq!(pm.entry_count().await, 1); assert_eq!(pm.entry_count().await, 1);
@@ -1613,7 +1725,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]); let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]);
pm.upsert_entry(e1).await.unwrap(); pm.upsert_entry(e1).await.unwrap();
let outcome = pm.upsert_entry(e2).await.unwrap(); let outcome = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(outcome, UpsertOutcome::Noop(_))); assert!(matches!(outcome, UpsertOutcome::Noop { .. }));
// Still exactly one entry, no duplicate from the re-seed. // Still exactly one entry, no duplicate from the re-seed.
assert_eq!(pm.entry_count().await, 1); assert_eq!(pm.entry_count().await, 1);
} }
@@ -1625,7 +1737,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]); let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]);
let o1 = pm.upsert_entry(e1).await.unwrap(); let o1 = pm.upsert_entry(e1).await.unwrap();
let pid = match o1 { let pid = match o1 {
UpsertOutcome::Added(p) => p, UpsertOutcome::Added { playbook_id: p } => p,
other => panic!("expected Added, got {:?}", other), other => panic!("expected Added, got {:?}", other),
}; };
let o2 = pm.upsert_entry(e2).await.unwrap(); let o2 = pm.upsert_entry(e2).await.unwrap();
@@ -1646,7 +1758,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-22", &["Alice Smith"]); let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-22", &["Alice Smith"]);
pm.upsert_entry(e1).await.unwrap(); pm.upsert_entry(e1).await.unwrap();
let o2 = pm.upsert_entry(e2).await.unwrap(); let o2 = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o2, UpsertOutcome::Added(_)), "different day → fresh ADD"); assert!(matches!(o2, UpsertOutcome::Added { .. }), "different day → fresh ADD");
assert_eq!(pm.entry_count().await, 2); assert_eq!(pm.entry_count().await, 2);
} }
@@ -1659,9 +1771,200 @@ mod upsert_tests {
// A new seed on same day should ADD, not merge into the retired one. // A new seed on same day should ADD, not merge into the retired one.
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Carol Davis"]); let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Carol Davis"]);
let o = pm.upsert_entry(e2).await.unwrap(); let o = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o, UpsertOutcome::Added(_))); assert!(matches!(o, UpsertOutcome::Added { .. }));
assert_eq!(pm.entry_count().await, 2); assert_eq!(pm.entry_count().await, 2);
} }
// Regression test for the auditor-discovered bug: UPDATE branch
// silently dropped doc_refs, valid_until, and (partially)
// schema_fingerprint from the incoming entry.
//
// Before the fix: a re-seed with different endorsed_names AND new
// doc_refs would merge names but discard the new doc_refs. After:
// both sets are unioned (doc_refs merged by tool, same-tool
// supersedes).
fn docref(tool: &str, version: &str) -> DocRef {
DocRef {
tool: tool.into(),
version_seen: version.into(),
snippet_hash: Some(format!("hash-{tool}-{version}")),
source_url: None,
seen_at: "2026-04-22T00:00:00Z".into(),
}
}
fn mk_with_docs(op: &str, day: &str, names: &[&str], docs: Vec<DocRef>) -> PlaybookEntry {
let mut e = mk(op, day, names);
e.doc_refs = docs;
e
}
#[tokio::test]
async fn update_merges_doc_refs_with_existing_ones() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let day = "2026-04-22";
let op = "fill: Welder x2 in Nashville, TN";
// First seed: Alice + Docker doc ref
let e1 = mk_with_docs(op, day, &["Alice"], vec![docref("docker", "24.0.7")]);
let o1 = pm.upsert_entry(e1).await.unwrap();
assert!(matches!(o1, UpsertOutcome::Added { .. }));
// Second seed on same (op, day) but new names AND new tool ref
let e2 = mk_with_docs(op, day, &["Bob"], vec![docref("terraform", "1.5.0")]);
let o2 = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o2, UpsertOutcome::Updated { .. }));
assert_eq!(pm.entry_count().await, 1); // still one entry, merged
let snap = pm.snapshot().await;
assert_eq!(snap.len(), 1);
let merged = &snap[0];
// Names unioned
assert!(merged.endorsed_names.contains(&"Alice".to_string()));
assert!(merged.endorsed_names.contains(&"Bob".to_string()));
// Both doc_refs preserved (different tools → union)
assert_eq!(merged.doc_refs.len(), 2);
let tools: Vec<_> = merged.doc_refs.iter().map(|r| r.tool.as_str()).collect();
assert!(tools.contains(&"docker"));
assert!(tools.contains(&"terraform"));
}
#[tokio::test]
async fn update_same_tool_supersedes_older_version() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let day = "2026-04-22";
let op = "fill: Welder x2 in Nashville, TN";
let e1 = mk_with_docs(op, day, &["Alice"], vec![docref("docker", "24.0.7")]);
pm.upsert_entry(e1).await.unwrap();
// Same day, different name, SAME tool newer version
let e2 = mk_with_docs(op, day, &["Bob"], vec![docref("docker", "25.0.1")]);
let o2 = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o2, UpsertOutcome::Updated { .. }));
let snap = pm.snapshot().await;
assert_eq!(snap[0].doc_refs.len(), 1, "same tool should supersede, not duplicate");
assert_eq!(snap[0].doc_refs[0].version_seen, "25.0.1");
}
#[tokio::test]
async fn update_preserves_existing_doc_refs_when_new_entry_has_none() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let day = "2026-04-22";
let op = "fill: Welder x2 in Nashville, TN";
let e1 = mk_with_docs(op, day, &["Alice"], vec![docref("docker", "24.0.7")]);
pm.upsert_entry(e1).await.unwrap();
// Second seed doesn't carry doc_refs — shouldn't wipe the existing
let e2 = mk(op, day, &["Bob"]); // no doc_refs
pm.upsert_entry(e2).await.unwrap();
let snap = pm.snapshot().await;
assert_eq!(snap[0].doc_refs.len(), 1, "empty new doc_refs shouldn't wipe existing");
assert_eq!(snap[0].doc_refs[0].tool, "docker");
}
// ─── Phase 45 slice 3 — doc_drift flag / resolve / boost exclusion ───
#[tokio::test]
async fn flag_doc_drift_stamps_timestamp_and_persists() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let e = mk("fill: Welder x1 in Nashville, TN", "2026-04-22", &["Alice"]);
pm.set_entries(vec![e.clone()]).await.unwrap();
let touched = pm.flag_doc_drift(&e.playbook_id).await.unwrap();
assert!(touched, "first flag should return true");
let fetched = pm.get_entry(&e.playbook_id).await.unwrap();
assert!(fetched.doc_drift_flagged_at.is_some(), "flag_doc_drift should stamp the timestamp");
assert!(fetched.doc_drift_reviewed_at.is_none(), "reviewed_at starts None on fresh flag");
}
#[tokio::test]
async fn flag_doc_drift_is_idempotent_on_already_flagged() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let mut e = mk("fill: Welder x1 in Nashville, TN", "2026-04-22", &["Alice"]);
e.doc_drift_flagged_at = Some("2026-04-01T00:00:00Z".into());
pm.set_entries(vec![e.clone()]).await.unwrap();
let touched = pm.flag_doc_drift(&e.playbook_id).await.unwrap();
assert!(!touched, "already-flagged, unreviewed entry shouldn't re-stamp");
}
#[tokio::test]
async fn resolve_doc_drift_clears_flag_admission_gate() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let mut e = mk("fill: Welder x1 in Nashville, TN", "2026-04-22", &["Alice"]);
e.doc_drift_flagged_at = Some("2026-04-21T00:00:00Z".into());
pm.set_entries(vec![e.clone()]).await.unwrap();
let resolved = pm.resolve_doc_drift(&e.playbook_id).await.unwrap();
assert!(resolved);
let fetched = pm.get_entry(&e.playbook_id).await.unwrap();
assert!(fetched.doc_drift_reviewed_at.is_some());
// Second resolve is a no-op
let again = pm.resolve_doc_drift(&e.playbook_id).await.unwrap();
assert!(!again);
}
#[tokio::test]
async fn boost_excludes_flagged_unreviewed_entries() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let e_clean = mk("fill: Welder x1 in Nashville, TN", "2026-04-22", &["Alice"]);
let mut e_flagged = mk("fill: Welder x1 in Nashville, TN", "2026-04-21", &["Bob"]);
e_flagged.doc_drift_flagged_at = Some("2026-04-22T00:00:00Z".into());
e_flagged.doc_drift_reviewed_at = None;
pm.set_entries(vec![e_clean, e_flagged]).await.unwrap();
let boosts = pm
.compute_boost_for_filtered_with_role(
&[1.0, 0.0, 0.0], 10, 0.5,
Some(("Nashville", "TN")), None,
).await;
let keys: Vec<_> = boosts.keys().map(|(_, _, n)| n.clone()).collect();
assert!(keys.contains(&"Alice".to_string()));
assert!(!keys.contains(&"Bob".to_string()), "flagged+unreviewed entry leaked into boost");
}
#[tokio::test]
async fn boost_re_admits_resolved_entries() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let mut e = mk("fill: Welder x1 in Nashville, TN", "2026-04-22", &["Alice"]);
e.doc_drift_flagged_at = Some("2026-04-22T00:00:00Z".into());
e.doc_drift_reviewed_at = Some("2026-04-22T00:01:00Z".into()); // human reviewed
pm.set_entries(vec![e]).await.unwrap();
let boosts = pm
.compute_boost_for_filtered_with_role(
&[1.0, 0.0, 0.0], 10, 0.5,
Some(("Nashville", "TN")), None,
).await;
let keys: Vec<_> = boosts.keys().map(|(_, _, n)| n.clone()).collect();
assert!(keys.contains(&"Alice".to_string()), "resolved entry should re-enter boost pool");
}
#[tokio::test]
async fn update_refreshes_valid_until_when_caller_provides_one() {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let day = "2026-04-22";
let op = "fill: Welder x2 in Nashville, TN";
// First seed with no valid_until
let e1 = mk(op, day, &["Alice"]);
pm.upsert_entry(e1).await.unwrap();
// Second seed with a valid_until deadline
let mut e2 = mk(op, day, &["Bob"]);
e2.valid_until = Some("2026-05-01T00:00:00Z".into());
pm.upsert_entry(e2).await.unwrap();
let snap = pm.snapshot().await;
assert_eq!(snap[0].valid_until.as_deref(), Some("2026-05-01T00:00:00Z"));
}
} }
#[cfg(test)] #[cfg(test)]
+91
View File
@@ -133,6 +133,9 @@ pub fn router(state: VectorState) -> Router {
.route("/playbook_memory/revise", post(revise_playbook_memory)) .route("/playbook_memory/revise", post(revise_playbook_memory))
.route("/playbook_memory/history/{id}", get(playbook_memory_history)) .route("/playbook_memory/history/{id}", get(playbook_memory_history))
.route("/playbook_memory/status", get(playbook_memory_status)) .route("/playbook_memory/status", get(playbook_memory_status))
// Phase 45 slice 3 — doc drift detection + human re-admission.
.route("/playbook_memory/doc_drift/check/{id}", post(check_doc_drift))
.route("/playbook_memory/doc_drift/resolve/{id}", post(resolve_doc_drift))
.with_state(state) .with_state(state)
} }
@@ -2500,6 +2503,94 @@ async fn retire_playbook_memory(
"supply either {playbook_id, reason} or {city, state, current_schema_fingerprint, reason}".into())) "supply either {playbook_id, reason} or {city, state, current_schema_fingerprint, reason}".into()))
} }
/// Phase 45 slice 3 — POST /playbook_memory/doc_drift/check/{id}
///
/// Iterates the playbook's `doc_refs`, asks the context7 bridge whether
/// each one drifted against the recorded snippet_hash. If any tool
/// returned `drifted: true`, stamps `doc_drift_flagged_at` on the
/// entry — which excludes it from boost (via the filter in
/// `compute_boost_for_filtered_with_role`) until a human reviews and
/// resolves.
///
/// Unknown outcomes (bridge down, tool not in context7, no snippet
/// hash) are explicitly NOT enough to flag. Only a positive drifted=true
/// from the bridge flips the flag.
async fn check_doc_drift(
State(state): State<VectorState>,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
use crate::doc_drift::{check_all_refs, DriftCheckerConfig, DriftOutcome};
let entry = state.playbook_memory.get_entry(&id).await
.ok_or((StatusCode::NOT_FOUND, format!("playbook not found: {id}")))?;
if entry.doc_refs.is_empty() {
return Ok(Json(serde_json::json!({
"playbook_id": id,
"checked_tools": [],
"drifted": false,
"flagged": false,
"reason": "entry has no doc_refs — nothing to check",
})));
}
let cfg = DriftCheckerConfig::default();
let results = check_all_refs(&cfg, &entry.doc_refs).await;
let per_tool: Vec<serde_json::Value> = results.iter().map(|r| {
let (drifted, current, src, reason) = match &r.outcome {
DriftOutcome::Drifted { current_snippet_hash, source_url } =>
(true, Some(current_snippet_hash.clone()), source_url.clone(), None),
DriftOutcome::Unchanged =>
(false, None, None, None),
DriftOutcome::Unknown { reason } =>
(false, None, None, Some(reason.clone())),
};
serde_json::json!({
"tool": r.tool,
"version_seen": r.version_seen,
"drifted": drifted,
"current_snippet_hash": current,
"source_url": src,
"unknown_reason": reason,
})
}).collect();
let any_drifted = results.iter().any(|r| matches!(r.outcome, DriftOutcome::Drifted { .. }));
let flagged = if any_drifted {
state.playbook_memory.flag_doc_drift(&id).await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, format!("flag: {e}")))?
} else {
false
};
Ok(Json(serde_json::json!({
"playbook_id": id,
"checked_tools": results.iter().map(|r| &r.tool).collect::<Vec<_>>(),
"drifted": any_drifted,
"flagged": flagged,
"per_tool": per_tool,
})))
}
/// Phase 45 slice 3 — POST /playbook_memory/doc_drift/resolve/{id}
///
/// Human-in-the-loop re-admission. Stamps `doc_drift_reviewed_at`.
/// Idempotent: returns `resolved: false` if nothing changed (entry
/// wasn't flagged, already reviewed, or doesn't exist).
async fn resolve_doc_drift(
State(state): State<VectorState>,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let resolved = state.playbook_memory.resolve_doc_drift(&id).await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, format!("resolve: {e}")))?;
Ok(Json(serde_json::json!({
"playbook_id": id,
"resolved": resolved,
})))
}
/// Phase 27 — request body for `POST /playbook_memory/revise`. Same /// Phase 27 — request body for `POST /playbook_memory/revise`. Same
/// shape as a seed request minus `append` (revise is always /// shape as a seed request minus `append` (revise is always
/// append-semantics for a specific parent) plus `parent_id`. The new /// append-semantics for a specific parent) plus `parent_id`. The new
+126
View File
@@ -0,0 +1,126 @@
# Cohesion integration plan — the "smarter DB" loop
**Written:** 2026-04-22, after J flagged that the system has good parts but they don't compose into the self-improving loop promised in the control-plane thesis (`project_control_plane_thesis.md` memory: 0→85% via hyperfocus-then-escalate).
## The gap
Each piece works in isolation. What's not wired:
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ observer │──✓──│ data/_kb/ │──?──│ auditor │
│ :3800 │ │ outcomes. │ │ kb_query │
│ │ │ jsonl │ │ check │
└─────────────┘ └─────────────┘ └─────────────┘
▲ │
│? ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ auditor │─────│ data/_audit │ │ cloud │
│ verdicts │ │or/verdicts/ │ │ inference │
│ │ │ │ │ │
└─────────────┘ └─────────────┘ └─────────────┘
│?
▼
┌─────────────┐
│ hybrid_srch │
│ playbook_mem│
│ context7 │
└─────────────┘
```
- **✓** solid: observer captures scenarios, writes `_kb/outcomes.jsonl`.
- **?** missing:
- auditor verdicts go to `_auditor/verdicts/` — not to observer or KB
- auditor kb_query reads `_kb/outcomes.jsonl` but doesn't query hybrid_search, playbook_memory, or context7
- auditor inference sends claim+diff to cloud — but WITHOUT KB neighbors, WITHOUT drift context, WITHOUT tool-aware retrieval
- Result: every audit is stateless-ish. The DB doesn't get smarter across audits.
## The target loop
```
PR opened
↓
auditor fetches diff + claims
↓
auditor enriches with:
— KB neighbors: past verdicts on similar claim-hashes
— hybrid_search: playbooks for task classes the diff touches
— context7 drift status: for any tool named in commit messages
— MCP tools: agent-style queries if relevant
↓
cloud inference sees: diff + claims + enrichment
↓
verdict posted to Gitea
↓
verdict ALSO persisted to:
— observer :3800/event (source=auditor)
— _kb/outcomes.jsonl (kind=audit, sig_hash=pr+sha)
↓
next audit on similar PR:
— KB neighbors find this prior verdict
— cloud sees "similar PRs ended in request_changes for reason X"
— verdict is more calibrated
```
## Phases of closure
Building this in order from least-invasive:
### Phase A — Verdict indexing (shipped in this PR)
After every `auditPr()` completes, in addition to persisting to `_auditor/verdicts/`:
1. POST the verdict to observer `:3800/event` with `source: "auditor"`, `ok: verdict === "approve"`, `event_kind: "audit"`, `sig_hash: <stable hash of pr_number + head_sha>`.
2. Append a simplified outcome to `data/_kb/outcomes.jsonl` with a special `kind: "audit"` row so the KB surface treats audits alongside scenarios.
Minimal surgery. Doesn't change the verdict itself. Just makes it visible to downstream consumers.
**Test:** after this PR lands, re-run the auditor once; observer stats should show `by_source.auditor > 0`; `_kb/outcomes.jsonl` should have one new row per audited SHA.
### Phase B — KB query sees auditor history
Extend `auditor/checks/kb_query.ts` to:
1. Read `_kb/outcomes.jsonl`, filter `kind === "audit"`, find prior audits with matching `sig_hash` (same PR across SHAs) or similar claim hashes.
2. Emit finding: "prior N audits on this PR ended in [approve, block, request_changes], last reason was X."
Gives the auditor memory across re-audits of the same PR.
### Phase C — Hybrid search in kb_query
When a PR touches `crates/vectord/src/playbook_memory.rs` (task-class matching), auditor calls `POST /vectors/hybrid` to find playbooks semantically related to the diff. Surfaces as "N playbooks in production rely on this code path, consider backward-compat."
Requires:
- Task-class extractor from diff paths
- Cloud-free hybrid_search call (we have local Ollama for embeddings)
### Phase D — Context7 drift awareness in auditor
If the diff / commit message names a tool that's in any playbook's `doc_refs`, auditor calls the context7 bridge to check current drift status. Surfaces as "tool X has drifted since N playbooks referenced it; this PR may need to update those."
### Phase E — Inference sees enrichment
The cloud inference check currently sends `diff + claims`. Extend to send `diff + claims + kb_neighbors + drift_context + hybrid_search_matches`. The prompt becomes context-rich — exactly what makes the cloud model competent (per the control-plane thesis).
### Phase F — Full integration test
An auditor self-test that:
1. Creates a synthetic PR with known-good and known-bad claims
2. Runs the enriched auditor
3. Asserts the verdict found the planted issues
4. Runs a second identical-claim PR
5. Asserts the SECOND verdict references the FIRST audit (via KB neighbor retrieval)
6. "Smarter DB" proof: two runs, measurable context gain.
## Sequence
Phase A lands in this PR alongside the inventory. Phases B-F are follow-up PRs, each with their own auditor gate. Order matters: A → B (reads A's output) → C+D (in parallel) → E (consumes B/C/D) → F (asserts E's behavior).
## Not in this plan (deliberate)
- **Rewriting the auditor to use a different architecture.** Current 4-check model stays; we just enrich the checks.
- **Tuning cloud inference precision.** The false-positive rate is a prompt-engineering concern; this plan is about context enrichment, which is separate.
- **Branch protection enforcement.** Stays off until Phase F passes.
The overall bet: this is the "putting it all together coherently" J said was a real problem. Six phases over however many PRs it takes. Each one ships one wire of the loop; no single PR tries to do them all.
+75
View File
@@ -0,0 +1,75 @@
# Python inventory — what's wired vs dead
Generated 2026-04-22. Enumerates every `.py` file in the repo and classifies each by actual integration status. Goal: surface the "ambiguous names that don't hook in" J flagged.
## Classification
- **🟢 Production** — running as a systemd service, or imported by code that does
- **🟡 Documented** — referenced in `docs/PRD.md` / `docs/PHASES.md` / `docs/CONTROL_PLANE_PRD.md` or called from a shell/TS orchestrator
- **🟠 Manual** — runnable as a one-off utility but not in any automated pipeline
- **🔴 Dead** — not imported, not referenced, not called anywhere
## Results
### Sidecar (`sidecar/sidecar/`)
The Python FastAPI sidecar on `:3200`. Runs as `lakehouse-sidecar.service`.
| File | Status | Evidence |
|---|---|---|
| `main.py` | 🟢 Production | systemd ExecStart: `uvicorn sidecar.sidecar.main:app` |
| `admin.py` | 🟢 Production | imported by `main.py` as `admin_router` |
| `embed.py` | 🟢 Production | imported by `main.py` as `embed_router` |
| `generate.py` | 🟢 Production | imported by `main.py` as `generate_router` |
| `rerank.py` | 🟢 Production | imported by `main.py` as `rerank_router` |
| `ollama.py` | 🟢 Production | imported transitively by embed/generate/rerank |
| `__init__.py` | 🟢 Production | package marker |
| `lab_ui.py` | 🔴 **Dead** | **not imported by any file in `sidecar/`. `scripts/serve_lab.py` only uses stdlib — doesn't import this.** Mtime 2026-04-16 (pre-session). Safe to delete. |
| `pipeline_lab.py` | 🔴 **Dead** | **same — not imported anywhere. Mtime 2026-04-16.** Docstring says "iterative embedding/LLM pipeline experimentation" — looks like an abandoned prototype. Safe to delete. |
### Scripts (`scripts/`)
17 files. Only **2** are production (behind systemd), **3** are documented in phase/PRD docs, **12** are unreferenced.
| File | Status | Evidence |
|---|---|---|
| `serve_ui.py` | 🟢 Production | `lakehouse-ui.service` ExecStart |
| `serve_imagegen.py` | 🟢 Production | `imagegen.service` ExecStart |
| `serve_lab.py` | 🟢 Production | `pipeline-lab.service` ExecStart |
| `kb_measure.py` | 🟡 Documented | referenced in `docs/PHASES.md` (Phase 22 aggregator) |
| `kb_staffer_report.py` | 🟡 Documented | referenced in `docs/PHASES.md` + `docs/PRD.md` (Phase 23), also called by `scripts/run_staffer_demo.sh` |
| `autonomous_agent.py` | 🟠 Manual | "Autonomous stress-test agent" — no docs refs, no callers. Mtime 2026-04-17. |
| `copilot.py` | 🟠 Manual | "Staffing Co-Pilot — the anticipation layer" — no docs refs. Writes briefing to `/tmp/copilot_briefing.json` on run. |
| `generate_demo.py` | 🟠 Manual | "realistic demo datasets" — mtime 2026-03-27, never touched this session |
| `generate_workers.py` | 🟠 Manual | "worker profiles at scale" — usage example in docstring: `python3 generate_workers.py 100000 > /tmp/workers_100k.csv`. Still useful, just not auto-run. |
| `lance_tune.py` | 🟠 Manual | "IVF_PQ parameter sweep" — one-shot experiment tool. |
| `qwen3_plan.py` | 🟠 Manual | "Qwen 3 agent plan — structured test with playbook building" |
| `quality_eval.py` | 🟠 Manual | "Quality evaluation pipeline — tests whether the system gives CORRECT" |
| `scale_test.py` | 🟠 Manual | called by `scripts/scale_10m_test.sh` (not in systemd). 2.5M row test. |
| `staffing_day.py` | 🟠 Manual | "Real-world staffing agency day simulation" — no docs refs. Overlaps conceptually with `tests/multi-agent/scenario.ts`. |
| `staffing_demo.py` | 🟠 Manual | "Realistic staffing company data generator" — demo fixture. |
| `staffing_simulation.py` | 🟠 Manual | "multi-agent stress test" — overlaps with `tests/multi-agent/scenario.ts`. |
| `stress_test.py` | 🟠 Manual | "prove this architecture is sound or find where it breaks" — overlaps with `tests/multi-agent/run_stress.ts`. |
## Overlap with TypeScript / Rust equivalents
Several Python scripts overlap conceptually with newer TS/Rust code. This is the "ambiguous names that don't hook in" pattern — the Python originals were superseded but not removed.
| Python script | Newer equivalent | Integration drift |
|---|---|---|
| `scripts/staffing_simulation.py` | `tests/multi-agent/scenario.ts` | scenario.ts ships + observer-integrated (Phase 24); Python version is unwired. |
| `scripts/staffing_day.py` | `tests/multi-agent/scenario.ts` | same. |
| `scripts/stress_test.py` | `tests/multi-agent/run_stress.ts` | TS version is the one the auditor fixture exercises. |
| `scripts/autonomous_agent.py` | `bot/` + `auditor/` Bun sub-agents | new Bun pattern supersedes. Python version predates the control-plane pivot. |
| `scripts/qwen3_plan.py` | `tests/multi-agent/agent.ts` | TS agent is the one with Phase 21 continuation + cloud routing. |
**Recommended cleanup actions (not in this PR — separate decision):**
1. **Delete the two confirmed dead sidecar files:** `sidecar/sidecar/lab_ui.py`, `sidecar/sidecar/pipeline_lab.py`. Nothing calls them; removal is safe.
2. **Archive the 5 overlapping scripts** (staffing_simulation, staffing_day, stress_test, autonomous_agent, qwen3_plan) — either move to `scripts/legacy/` with a README explaining they predate the TS versions, or delete if J confirms no future use.
3. **Keep the one-off utilities** (generate_demo, generate_workers, lance_tune, quality_eval, copilot, scale_test, staffing_demo) — useful as manual tools. Maybe add a `scripts/README.md` documenting what each does and when to run.
4. **Promote `kb_measure.py` and `kb_staffer_report.py`** to be actually callable from the KB flow — they're documented but only run manually after scenario batches.
## Honest next step
The cleanup above isn't this session's work — it's a subsequent PR. This doc lands first so the state is visible and operator-auditable.
+25 -6
View File
@@ -37,7 +37,11 @@ interface ObservedOp {
// Phase 24 — optional provenance so error analyzer and playbook // Phase 24 — optional provenance so error analyzer and playbook
// builder can differentiate MCP-layer ops from scenario-sourced // builder can differentiate MCP-layer ops from scenario-sourced
// events. Scenarios set source="scenario" + staffer_id + sig_hash. // events. Scenarios set source="scenario" + staffer_id + sig_hash.
source?: "mcp" | "scenario"; // Phase A (cohesion plan 2026-04-22) — auditor verdicts also post
// to :3800/event with source="auditor"; stats endpoint counts them
// separately so KB readers can see audit cadence alongside
// scenario cadence.
source?: "mcp" | "scenario" | "auditor" | "bot";
staffer_id?: string; staffer_id?: string;
sig_hash?: string; sig_hash?: string;
event_kind?: string; event_kind?: string;
@@ -253,15 +257,30 @@ function startHttpListener() {
} }
if (req.method === "POST" && url.pathname === "/event") { if (req.method === "POST" && url.pathname === "/event") {
return req.json().then((body: any) => { return req.json().then((body: any) => {
// Accept caller-provided source when present and valid.
// Unrecognized sources fall back to "scenario" for
// backward-compat with older callers.
const allowedSource = ["mcp", "scenario", "auditor", "bot"] as const;
const bodySrc = typeof body.source === "string" ? body.source : "";
const source = (allowedSource as readonly string[]).includes(bodySrc)
? (bodySrc as typeof allowedSource[number])
: "scenario";
// Phase A: auditor+bot POST with ok=true|false (not success)
// for symmetry with the kind_of of their domain. Accept either.
const success = typeof body.success === "boolean" ? body.success
: typeof body.ok === "boolean" ? body.ok
: false;
const op: ObservedOp = { const op: ObservedOp = {
timestamp: body.timestamp ?? new Date().toISOString(), timestamp: body.timestamp ?? new Date().toISOString(),
endpoint: body.endpoint ?? "scenario:fill", endpoint: body.endpoint ?? (source === "auditor" ? "auditor:verdict"
: source === "bot" ? "bot:cycle"
: "scenario:fill"),
input_summary: body.input_summary ?? `${body.event_kind ?? "?"} ${body.role ?? "?"}×${body.count ?? "?"} in ${body.city ?? "?"}, ${body.state ?? "?"}`, input_summary: body.input_summary ?? `${body.event_kind ?? "?"} ${body.role ?? "?"}×${body.count ?? "?"} in ${body.city ?? "?"}, ${body.state ?? "?"}`,
success: !!body.success, success,
duration_ms: Number(body.duration_ms ?? 0), duration_ms: Number(body.duration_ms ?? body.audit_duration_ms ?? 0),
output_summary: body.output_summary ?? (body.success ? "filled" : (body.error ?? "failed")), output_summary: body.output_summary ?? body.one_liner ?? (success ? "ok" : (body.error ?? "failed")),
error: body.error, error: body.error,
source: "scenario", source,
staffer_id: body.staffer_id, staffer_id: body.staffer_id,
sig_hash: body.sig_hash, sig_hash: body.sig_hash,
event_kind: body.event_kind, event_kind: body.event_kind,
+57
View File
@@ -0,0 +1,57 @@
# Lakehouse systemd units
Service definitions for long-running Lakehouse sidecars that aren't the
Rust gateway itself. The gateway has its own pre-existing unit
(`lakehouse.service`) that was configured at initial deploy time and
isn't tracked here.
## Units
| File | Service | Port | Purpose |
|---|---|---|---|
| `lakehouse-auditor.service` | `lakehouse-auditor` | n/a | Polls Gitea for open PRs, runs four checks (static / dynamic / inference / KB query), posts commit-status + review comment. Hard-blocks merges when claims aren't backed. |
| `lakehouse-context7-bridge.service` | `lakehouse-context7-bridge` | `:3900` | HTTP wrapper around context7's public API for Phase 45 doc-drift detection. |
## Install
```bash
sudo bash ops/systemd/install.sh
```
Idempotent. Copies units to `/etc/systemd/system/`, reloads, enables + (re)starts both services.
## Operate
```bash
# Status
systemctl status lakehouse-auditor
systemctl status lakehouse-context7-bridge
# Live logs
journalctl -u lakehouse-auditor -f
# Restart
systemctl restart lakehouse-auditor
# Stop (won't restart until enable + start again)
systemctl stop lakehouse-auditor
```
## Pause the auditor without stopping
```bash
touch /home/profit/lakehouse/auditor.paused # skip cycles until removed
rm /home/profit/lakehouse/auditor.paused # resume
```
## Env toggles on the auditor (edit the unit file, `systemctl daemon-reload`, restart)
```
LH_AUDITOR_RUN_DYNAMIC=1 # include the hybrid fixture on every audit
# default off — fixture mutates live playbook state
LH_AUDITOR_SKIP_INFERENCE=1 # skip cloud inference for fast/cheap runs
```
## Why both services run as root
To match the existing `lakehouse.service` + `mcp-server` + `observer` conventions on this host. Hardening to a dedicated unprivileged user is a follow-up: would need PATH adjustment for `bun`, credential file accessibility (the auditor reads `/home/profit/.git-credentials` which is `0600 profit:profit` — root reads fine, a non-profit non-root user wouldn't).
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Install the lakehouse-auditor + lakehouse-context7-bridge systemd units.
# Idempotent: re-running just reloads + restarts.
#
# Usage (as root):
# bash ops/systemd/install.sh
#
# What it does:
# 1. Copies *.service to /etc/systemd/system/
# 2. systemctl daemon-reload
# 3. systemctl enable --now both services
# 4. Prints post-install status
set -euo pipefail
UNIT_DIR="$(dirname "$(readlink -f "$0")")"
TARGET_DIR=/etc/systemd/system
UNITS=(
lakehouse-auditor.service
lakehouse-context7-bridge.service
)
if [[ $EUID -ne 0 ]]; then
echo "install.sh: must run as root (writes to $TARGET_DIR)" >&2
exit 1
fi
for unit in "${UNITS[@]}"; do
src="$UNIT_DIR/$unit"
dst="$TARGET_DIR/$unit"
if [[ ! -f "$src" ]]; then
echo "install.sh: missing source $src" >&2
exit 1
fi
echo "→ copy $unit"
install -m 0644 "$src" "$dst"
done
echo "→ systemctl daemon-reload"
systemctl daemon-reload
for unit in "${UNITS[@]}"; do
echo "→ enable + (re)start $unit"
systemctl enable "$unit" >/dev/null
systemctl restart "$unit"
done
echo ""
echo "─── post-install status ───"
for unit in "${UNITS[@]}"; do
active=$(systemctl is-active "$unit" 2>/dev/null || true)
enabled=$(systemctl is-enabled "$unit" 2>/dev/null || true)
printf " %-40s active=%s enabled=%s\n" "$unit" "$active" "$enabled"
done
echo ""
echo "Live logs: journalctl -u lakehouse-auditor.service -f"
echo "Pause: touch /home/profit/lakehouse/auditor.paused"
echo "Resume: rm /home/profit/lakehouse/auditor.paused"
+34
View File
@@ -0,0 +1,34 @@
[Unit]
Description=Lakehouse Claim Auditor — polls Gitea for open PRs + hard-blocks placeholder merges
Documentation=file:///home/profit/lakehouse/auditor/README.md
After=network.target lakehouse.service
Wants=lakehouse.service
[Service]
Type=simple
WorkingDirectory=/home/profit/lakehouse
# Runs as root to match the other lakehouse-* services on this host
# (gateway, mcp-server, observer). The auditor reads the git PAT out
# of /home/profit/.git-credentials which is 0600 profit:profit —
# root can read it, which is why the service runs as root.
# Alternative: run as `profit` and ensure bun binary is on PATH;
# left for a follow-up hardening PR.
ExecStart=/home/profit/.bun/bin/bun run /home/profit/lakehouse/auditor/index.ts
Restart=on-failure
RestartSec=30
# Stop responds to SIGTERM cleanly — no in-flight cycle survives a
# restart; the poller is idempotent so a mid-cycle restart just
# re-audits from state.json on next start.
KillSignal=SIGTERM
TimeoutStopSec=10
# Optional env toggles documented in auditor/index.ts:
# LH_AUDITOR_RUN_DYNAMIC=1 — include hybrid fixture on every audit
# (default off — mutates live playbook)
# LH_AUDITOR_SKIP_INFERENCE=1 — skip cloud inference for fast runs
# Pause file — operator can `touch /home/profit/lakehouse/auditor.paused`
# to skip the next cycle without stopping the service.
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,23 @@
[Unit]
Description=Lakehouse context7 HTTP bridge — doc-version lookups for Phase 45 drift detection
Documentation=file:///home/profit/lakehouse/mcp-server/context7_bridge.ts
After=network.target
# No hard dependency on gateway — the bridge is self-contained and
# talks to context7.com directly. Gateway calls INTO the bridge
# during drift checks but they're decoupled.
[Service]
Type=simple
WorkingDirectory=/home/profit/lakehouse
ExecStart=/home/profit/.bun/bin/bun run /home/profit/lakehouse/mcp-server/context7_bridge.ts
Restart=on-failure
RestartSec=15
# Default port :3900 (overridable via CONTEXT7_BRIDGE_PORT env).
# If the user moves the bridge to a different port, update the
# LH_BRIDGE_URL env on lakehouse.service + auditor.service too.
Environment=CONTEXT7_BRIDGE_PORT=3900
KillSignal=SIGTERM
TimeoutStopSec=5
[Install]
WantedBy=multi-user.target