matrix: cross-role playbook gate — closes real_001 bleed (OPEN #1)
real_001 surfaced same-client+city queries bleeding across roles:
Q#2 (Forklift Operator @ Beacon Freight Detroit) recorded e-6193
in the playbook corpus. Q#5 (Pickers same client+city) and Q#10
(CNC Operator same client+city) embedded within 0.13-0.18 cosine of
Q#2's query — well inside the 0.20 inject threshold — so e-6193
injected on both, demoting the cold-pass-correct workers.
Root cause: the inject distance threshold isn't tight enough on
the same-client+city cluster. Cosine collapses queries that share
city + client + count-token + time-token regardless of role. The
existing judge gate is per-injection at record time and doesn't
fire at retrieve time.
Fix: structural role gate in front of both Shape A boost and
Shape B inject. PlaybookEntry gains Role; SearchRequest gains
QueryRole. When both are non-empty and differ under roleEqual's
case+plural normalization, the entry is rejected before BoostFactor
or judge-gate logic runs.
Backward-compat: empty role on either side disables the gate —
preserves behavior for the lift suite's free-form multi-constraint
queries that have no clean single role. Caller-supplied (not
inferred), so existing recordings unaffected.
Wire-through:
- internal/matrix/playbook.go: Role field, NewPlaybookEntryWithRole,
roleEqual helper with plural+case normalization
- internal/matrix/retrieve.go: QueryRole on SearchRequest, threaded
to both ApplyPlaybookBoost + InjectPlaybookMisses
- cmd/matrixd/main.go: role on POST /matrix/playbooks/record + bulk
- scripts/playbook_lift/main.go: extractRoleFromNeed regex pulls
role from "Need N {role}{s} in" queries (the fill_events shape);
free-form queries fall back to empty (gate disabled)
Tests (5 new):
- TestInjectPlaybookMisses_RoleGateRejectsCrossRole: exact Q#10
scenario (distance 0.135, recorded "Forklift Operator", query
"CNC Operator") — locks the bleed at unit level
- TestInjectPlaybookMisses_RoleGateAllowsSameRole: Forklift Operator
recording fires on Forklift Operators query (plural normalization)
- TestInjectPlaybookMisses_RoleGateBackwardCompat: empty Role on
either side = gate disabled, preserves current behavior
- TestApplyPlaybookBoost_RoleGateRejectsCrossRole: Shape A defense
in depth — boost doesn't fire on cross-role even when answer is
in cold top-K
- TestRoleEqual_PluralAndCase: case + -s + -es plural normalization
Verification (real_002, same query set as real_001):
- Q#5 Pickers @ Beacon Freight: e-6193 → e-8499 (no bleed)
- Q#10 CNC Operator @ Beacon Freight: e-6193 → w-2404 (no bleed)
- Discoveries + lifts unchanged at 2 each (same-role lift still fires)
- Mean Δdist tightens from -0.127 to -0.040 (boosts no longer
pulling distances through the floor on cross-role mismatches)
Findings: reports/reality-tests/real_002_findings.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -92,6 +92,14 @@ type SearchRequest struct {
|
||||
JudgeURL string `json:"judge_url,omitempty"`
|
||||
JudgeModel string `json:"judge_model,omitempty"`
|
||||
JudgeMinRating int `json:"judge_min_rating,omitempty"`
|
||||
// QueryRole is an optional role hint that gates Shape A boost +
|
||||
// Shape B inject on role match (real_001 cross-role bleed fix).
|
||||
// When set, a recorded playbook entry only fires for queries with
|
||||
// the same role under roleEqual's normalization. Empty here OR
|
||||
// empty on the recorded entry = gate disabled (preserves behavior
|
||||
// for callers that don't supply role — e.g. lift-suite multi-
|
||||
// constraint queries that have no clean single role).
|
||||
QueryRole string `json:"query_role,omitempty"`
|
||||
MetadataFilter map[string]any `json:"metadata_filter,omitempty"`
|
||||
// ExcludeIDs filters out specific worker IDs post-retrieval.
|
||||
// Real-world driver: a coordinator places 200 workers at a
|
||||
@@ -292,7 +300,7 @@ func (r *Retriever) Search(ctx context.Context, req SearchRequest) (*SearchRespo
|
||||
}
|
||||
hits = keptHits
|
||||
}
|
||||
resp.PlaybookBoosted = ApplyPlaybookBoost(resp.Results, hits)
|
||||
resp.PlaybookBoosted = ApplyPlaybookBoost(req.QueryRole, resp.Results, hits)
|
||||
maxInjectDist := float32(req.PlaybookMaxInjectDistance)
|
||||
if maxInjectDist <= 0 {
|
||||
maxInjectDist = float32(DefaultPlaybookMaxInjectDistance)
|
||||
@@ -304,7 +312,7 @@ func (r *Retriever) Search(ctx context.Context, req SearchRequest) (*SearchRespo
|
||||
gate = g
|
||||
}
|
||||
var injected int
|
||||
resp.Results, injected = InjectPlaybookMisses(req.QueryText, resp.Results, hits, maxInjectDist, gate)
|
||||
resp.Results, injected = InjectPlaybookMisses(req.QueryText, req.QueryRole, resp.Results, hits, maxInjectDist, gate)
|
||||
resp.PlaybookInjected = injected
|
||||
if injected > 0 {
|
||||
// Re-sort + truncate after injection. ApplyPlaybookBoost
|
||||
|
||||
Reference in New Issue
Block a user