observer: add /relevance heuristic filter for adjacency pollution
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Matrix retrieval often surfaces high-cosine chunks that are about
symbols the focus file IMPORTS but doesn't define. The reviewer LLM
then hallucinates those imported-crate internals as in-file content
("I see main.rs does X" when X lives in queryd::context).
mcp-server/relevance.ts — pure scorer with five signals:
path_match +1.0 chunk source/doc_id encodes focus path
defined_match +0.6 chunk text mentions focus.defined_symbols
token_overlap +0.4 jaccard of non-stopword tokens
prefix_match +0.3 shared first-2-segment prefix
import_only -0.5 mentions only imported symbols (pollution)
Default threshold 0.3 — tuned empirically on the gateway/main.rs case.
Also fixes a regex bug in the import extractor: the character class
was lowercase-only, so `use catalogd::Registry;` silently never
matched (regex backed off when it hit the uppercase R). Caught by
the test suite.
observer.ts — POST /relevance endpoint wraps filterChunks().
scrum_master_pipeline.ts — fetchMatrixContext gains optional
focusContent param; calls /relevance after collecting allHits and
before sort+top. Opt-out via LH_RELEVANCE_FILTER=0; threshold via
LH_RELEVANCE_THRESHOLD. Fall-open on observer failure.
9 unit tests, all green. Live probe on real shape correctly drops
a 0.7-cosine adjacency-pollution chunk while keeping in-focus hits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,8 @@
|
||||
* agents do and helps them not repeat mistakes.
|
||||
*/
|
||||
|
||||
import { filterChunks } from "./relevance";
|
||||
|
||||
const GATEWAY = process.env.GATEWAY_URL || "http://localhost:3700";
|
||||
const LAKEHOUSE = process.env.LAKEHOUSE_URL || "http://localhost:3100";
|
||||
const CYCLE_SECS = parseInt(process.env.OBSERVER_CYCLE || "30");
|
||||
@@ -578,6 +580,24 @@ function startHttpListener() {
|
||||
}).catch((e: Error) =>
|
||||
new Response(JSON.stringify({ error: e.message }), { status: 400 }));
|
||||
}
|
||||
// ─── Relevance filter (2026-04-25) ───
|
||||
// Drops "adjacency pollution" from matrix-retrieved chunks before
|
||||
// they reach the reviewer LLM. Caller (scrum/agent) posts the
|
||||
// focus file + candidate chunks; observer scores via heuristic
|
||||
// (path/symbol/token signals) and returns kept + dropped lists.
|
||||
// Pure function — no I/O, safe to call hot.
|
||||
if (req.method === "POST" && url.pathname === "/relevance") {
|
||||
return req.json().then((body: any) => {
|
||||
const focus = body.focus_file ?? body.focus ?? {};
|
||||
const chunks = body.chunks ?? [];
|
||||
const threshold = typeof body.threshold === "number" ? body.threshold : 0.3;
|
||||
const result = filterChunks(focus, chunks, threshold);
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}).catch((e: Error) =>
|
||||
new Response(JSON.stringify({ error: e.message }), { status: 400 }));
|
||||
}
|
||||
return new Response("not found", { status: 404 });
|
||||
},
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user