matrix: judge-gated Shape B inject — closes lift-suite tail issues
Lift suite run #004 left two unresolved tail issues: - Q6 ("Forklift loader") ↔ Q7 ("Hazmat warehouse, cold storage") swap recordings as warm top-1 because their embeddings are within 0.20 cosine of each other. Distance gate can't tell them apart. - Q9 + Q15 lose paraphrase recovery when qwen2.5 rephrases past the 0.20 threshold. Distance says "drift too far"; sometimes the drift is real (skip), sometimes the paraphrase is still on-domain (don't want to skip). Multi-coord run #008's judge re-rating proved the LLM can distinguish: Q3 crane case landed at distance 0.23 (looks tight) but rating 1 (irrelevant). The judge sees domain mismatch the embedder doesn't. This commit lifts that pattern into the matrix substrate. Shape B inject now optionally routes every candidate through a judge gate before the rank insert lands. Distance + judge BOTH have to approve. internal/matrix/playbook.go: - InjectPlaybookMisses signature gains a query string + an optional InjectGate. nil gate preserves pre-judge-gating behavior (current tests already pass with nil). - New InjectGate interface + InjectGateFunc adapter for tests and non-LLM callers. - Per-candidate gate.Approve(query, hit) call inserted between the dedup and the inject. Rejected candidates skip silently; injected count reflects post-gate decision. internal/matrix/judge.go (new, ~140 lines): - LLMJudgeGate calls an Ollama-shape /api/chat endpoint with the same 1-5 staffing-rubric prompt that worked in multi_coord run #008. fail-closed on HTTP/JSON errors (don't inject if judge can't speak — better miss than wrong-domain). - NewLLMJudgeGate returns nil when URL or Model is empty, matching InjectGate's nil-means-no-judge semantics. internal/matrix/retrieve.go: - SearchRequest gains JudgeURL, JudgeModel, JudgeMinRating fields. Run() builds an LLMJudgeGate when set; passes nil otherwise. Backward compatible — existing callers see no behavior change. Tests: - TestInjectPlaybookMisses_GateRejectsCandidate (rejectAll → 0 injected, even with tight distance) - TestInjectPlaybookMisses_GateApprovesCandidate (approveAll → same as nil-gate behavior) - TestInjectPlaybookMisses_GateSeesCorrectQuery (gate receives CURRENT query + RECORDED query separately so it can score the (current, candidate) pair) - All 5 existing inject tests updated to new signature go test ./internal/matrix → all 8 inject tests pass. go test ./internal/matrix ./internal/shared ./cmd/{matrixd, queryd,pathwayd,observerd} → all green. STATE_OF_PLAY: - OPEN item #1 (judge-gated injection) closed. - DO NOT RELITIGATE adds the substrate-level judge-gate lock. - OPEN list now 5 rows (was 6). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -202,7 +202,24 @@ type PlaybookHit struct {
|
||||
// playbook-corpus cosine distance exceeds it are skipped (the boost
|
||||
// path may still re-rank them in place). Pass 0 (or any non-positive
|
||||
// value) to use DefaultPlaybookMaxInjectDistance.
|
||||
func InjectPlaybookMisses(results []Result, hits []PlaybookHit, maxInjectDist float32) ([]Result, int) {
|
||||
//
|
||||
// gate is an optional approval callback called once per CANDIDATE
|
||||
// (post-distance-filter, post-dedup) before injection. Returning
|
||||
// false rejects that candidate. Use nil for the historical "all
|
||||
// distance-eligible candidates inject" behavior.
|
||||
//
|
||||
// Multi-coord run #008's judge re-rating proved that distance + LLM
|
||||
// rating disagree often enough to matter (Q3 crane: dist 0.23 looks
|
||||
// confident, judge says 1/5 = irrelevant). Lift-suite tail issues
|
||||
// (Q6↔Q7 swap, Q9/Q15 paraphrase drift) are exactly this shape —
|
||||
// embedding-tight but wrong-domain. The gate parameter lets callers
|
||||
// route those candidates through a judge before the inject lands.
|
||||
//
|
||||
// query is the current search's query text — passed to the gate so
|
||||
// it can score (query, candidate) pairs without re-deriving from
|
||||
// SearchRequest. Empty when the caller doesn't have it (gate
|
||||
// implementations should treat empty query as "skip judge, allow").
|
||||
func InjectPlaybookMisses(query string, results []Result, hits []PlaybookHit, maxInjectDist float32, gate InjectGate) ([]Result, int) {
|
||||
if len(hits) == 0 {
|
||||
return results, 0
|
||||
}
|
||||
@@ -235,7 +252,16 @@ func InjectPlaybookMisses(results []Result, hits []PlaybookHit, maxInjectDist fl
|
||||
}
|
||||
}
|
||||
|
||||
injected := 0
|
||||
for _, h := range bestForKey {
|
||||
// Judge gate (per OPEN item #1, closed by this commit):
|
||||
// post-distance-filter, ask the gate whether the candidate
|
||||
// actually fits the current query before letting it inject.
|
||||
// Closes the lift-suite tail issues where embedding said
|
||||
// "tight" but a judge said "wrong domain."
|
||||
if gate != nil && !gate.Approve(query, h) {
|
||||
continue
|
||||
}
|
||||
injectedDist := h.Distance * float32(h.Entry.BoostFactor())
|
||||
// Synthesize metadata that flags the injection so callers
|
||||
// (driver/UI/observer) can distinguish "regular retrieval"
|
||||
@@ -256,11 +282,34 @@ func InjectPlaybookMisses(results []Result, hits []PlaybookHit, maxInjectDist fl
|
||||
Distance: injectedDist,
|
||||
Metadata: meta,
|
||||
})
|
||||
injected++
|
||||
}
|
||||
|
||||
return results, len(bestForKey)
|
||||
return results, injected
|
||||
}
|
||||
|
||||
// InjectGate is the optional approval callback for Shape B inject.
|
||||
// Called once per candidate (after distance filter, after dedup).
|
||||
// Returning false rejects that candidate. Implementations:
|
||||
// - LLMJudgeGate (this package, see judge.go): Ollama LLM rates the
|
||||
// (query, candidate) pair against a 1-5 rubric.
|
||||
// - InjectGateFunc (this package): zero-deps adapter for arbitrary
|
||||
// caller logic — useful in tests + when callers want non-LLM
|
||||
// gating (e.g. metadata-only filters).
|
||||
//
|
||||
// nil InjectGate = pre-judge-gating behavior (all distance-eligible
|
||||
// candidates inject); preserves backward compatibility.
|
||||
type InjectGate interface {
|
||||
Approve(query string, hit PlaybookHit) bool
|
||||
}
|
||||
|
||||
// InjectGateFunc adapts a plain function to the InjectGate interface.
|
||||
// Used heavily in tests; production callers usually use LLMJudgeGate.
|
||||
type InjectGateFunc func(query string, hit PlaybookHit) bool
|
||||
|
||||
// Approve makes InjectGateFunc satisfy InjectGate.
|
||||
func (f InjectGateFunc) Approve(q string, h PlaybookHit) bool { return f(q, h) }
|
||||
|
||||
// ApplyPlaybookBoost re-ranks results in place using matched
|
||||
// playbook hits. For each hit whose (AnswerID, AnswerCorpus)
|
||||
// matches a result, multiply that result's distance by the hit's
|
||||
|
||||
Reference in New Issue
Block a user