observer: add /relevance heuristic filter for adjacency pollution
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts

Matrix retrieval often surfaces high-cosine chunks that are about
symbols the focus file IMPORTS but doesn't define. The reviewer LLM
then hallucinates those imported-crate internals as in-file content
("I see main.rs does X" when X lives in queryd::context).

mcp-server/relevance.ts — pure scorer with five signals:
  path_match      +1.0  chunk source/doc_id encodes focus path
  defined_match   +0.6  chunk text mentions focus.defined_symbols
  token_overlap   +0.4  jaccard of non-stopword tokens
  prefix_match    +0.3  shared first-2-segment prefix
  import_only    -0.5  mentions only imported symbols (pollution)

Default threshold 0.3 — tuned empirically on the gateway/main.rs case.

Also fixes a regex bug in the import extractor: the character class
was lowercase-only, so `use catalogd::Registry;` silently never
matched (regex backed off when it hit the uppercase R). Caught by
the test suite.

observer.ts — POST /relevance endpoint wraps filterChunks().
scrum_master_pipeline.ts — fetchMatrixContext gains optional
focusContent param; calls /relevance after collecting allHits and
before sort+top. Opt-out via LH_RELEVANCE_FILTER=0; threshold via
LH_RELEVANCE_THRESHOLD. Fall-open on observer failure.

9 unit tests, all green. Live probe on real shape correctly drops
a 0.7-cosine adjacency-pollution chunk while keeping in-focus hits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-25 23:51:45 -05:00
co-authored by Claude Opus 4.7
parent 54689d523c
commit 0115a60072
4 changed files with 438 additions and 3 deletions
+20
View File
@@ -17,6 +17,8 @@
* agents do and helps them not repeat mistakes.
*/
import { filterChunks } from "./relevance";
const GATEWAY = process.env.GATEWAY_URL || "http://localhost:3700";
const LAKEHOUSE = process.env.LAKEHOUSE_URL || "http://localhost:3100";
const CYCLE_SECS = parseInt(process.env.OBSERVER_CYCLE || "30");
@@ -578,6 +580,24 @@ function startHttpListener() {
}).catch((e: Error) =>
new Response(JSON.stringify({ error: e.message }), { status: 400 }));
}
// ─── Relevance filter (2026-04-25) ───
// Drops "adjacency pollution" from matrix-retrieved chunks before
// they reach the reviewer LLM. Caller (scrum/agent) posts the
// focus file + candidate chunks; observer scores via heuristic
// (path/symbol/token signals) and returns kept + dropped lists.
// Pure function — no I/O, safe to call hot.
if (req.method === "POST" && url.pathname === "/relevance") {
return req.json().then((body: any) => {
const focus = body.focus_file ?? body.focus ?? {};
const chunks = body.chunks ?? [];
const threshold = typeof body.threshold === "number" ? body.threshold : 0.3;
const result = filterChunks(focus, chunks, threshold);
return new Response(JSON.stringify(result), {
headers: { "content-type": "application/json" },
});
}).catch((e: Error) =>
new Response(JSON.stringify({ error: e.message }), { status: 400 }));
}
return new Response("not found", { status: 404 });
},
});