observer + scrum: gold-standard answer corpus for compounding context
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts

The compose-don't-add discipline applied to the original ask: when big
models produce good results (scrum reviews + observer escalations),
save them into the matrix indexer so future small-model handlers can
retrieve them as scaffolding. Local model gets near-paid quality from
a fraction of the cost.

New: scripts/build_answers_corpus.ts indexes lakehouse_answers_v1
from data/_kb/scrum_reviews.jsonl + data/_kb/observer_escalations.jsonl.
doc_id prefixes ('review:' vs 'escalation:') let consumers same-file-
gate the prior-reviews case while keeping escalations broad.

observer.ts: buildKbPreamble adds lakehouse_answers_v1 as a third
retrieval source alongside pathway/bug_fingerprints + lakehouse_arch_v1.
qwen3.5:latest synthesis now compresses three lenses into a single
briefing for the cloud reviewer.

scrum_master_pipeline.ts: epilogue dispatches a fire-and-forget rebuild
of lakehouse_answers_v1 after each run so this run's accepted reviews
are retrievable within ~30s. LH_SCRUM_SKIP_ANSWERS_REBUILD=1 disables.

Verified live: kb_preamble grew 416 → 727 chars after wiring third
source; qwen3.5:latest synthesis (702 → 128 tokens) compresses
correctly; deepseek-v3.1-terminus diagnosis (301 → 148 tokens) is
sharper, citing architectural patterns (circuit breaker, adapter
files) instead of generic timeouts. Total cost per escalation
unchanged at ~$0.0002.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-26 18:49:36 -05:00
co-authored by Claude Opus 4.7
parent 340fca2427
commit 0844206660
3 changed files with 181 additions and 4 deletions
+19
View File
@@ -2208,6 +2208,25 @@ async function main() {
log("");
log(`report: ${OUT_DIR}/scrum_report.md`);
// Auto-rebuild lakehouse_answers_v1 so this run's reviews are
// retrievable by future scrum/observer enrichment paths within ~30s.
// Best-effort, fire-and-forget — don't fail the pipeline on rebuild
// hiccups. Toggle off via LH_SCRUM_SKIP_ANSWERS_REBUILD=1.
if (process.env.LH_SCRUM_SKIP_ANSWERS_REBUILD !== "1") {
try {
const { spawn } = await import("node:child_process");
const child = spawn("bun", ["run", "scripts/build_answers_corpus.ts"], {
cwd: process.cwd(),
stdio: "inherit",
detached: true,
});
child.unref();
log("answers corpus rebuild dispatched (bun run scripts/build_answers_corpus.ts)");
} catch (e) {
log(`answers corpus rebuild skipped: ${(e as Error).message}`);
}
}
process.exit(summary.resolved === summary.target_count ? 0 : 1);
}