matrix: cross-role playbook gate — closes real_001 bleed (OPEN #1)

real_001 surfaced same-client+city queries bleeding across roles:
Q#2 (Forklift Operator @ Beacon Freight Detroit) recorded e-6193
in the playbook corpus. Q#5 (Pickers same client+city) and Q#10
(CNC Operator same client+city) embedded within 0.13-0.18 cosine of
Q#2's query — well inside the 0.20 inject threshold — so e-6193
injected on both, demoting the cold-pass-correct workers.

Root cause: the inject distance threshold isn't tight enough on
the same-client+city cluster. Cosine collapses queries that share
city + client + count-token + time-token regardless of role. The
existing judge gate is per-injection at record time and doesn't
fire at retrieve time.

Fix: structural role gate in front of both Shape A boost and
Shape B inject. PlaybookEntry gains Role; SearchRequest gains
QueryRole. When both are non-empty and differ under roleEqual's
case+plural normalization, the entry is rejected before BoostFactor
or judge-gate logic runs.

Backward-compat: empty role on either side disables the gate —
preserves behavior for the lift suite's free-form multi-constraint
queries that have no clean single role. Caller-supplied (not
inferred), so existing recordings unaffected.

Wire-through:
- internal/matrix/playbook.go: Role field, NewPlaybookEntryWithRole,
  roleEqual helper with plural+case normalization
- internal/matrix/retrieve.go: QueryRole on SearchRequest, threaded
  to both ApplyPlaybookBoost + InjectPlaybookMisses
- cmd/matrixd/main.go: role on POST /matrix/playbooks/record + bulk
- scripts/playbook_lift/main.go: extractRoleFromNeed regex pulls
  role from "Need N {role}{s} in" queries (the fill_events shape);
  free-form queries fall back to empty (gate disabled)

Tests (5 new):
- TestInjectPlaybookMisses_RoleGateRejectsCrossRole: exact Q#10
  scenario (distance 0.135, recorded "Forklift Operator", query
  "CNC Operator") — locks the bleed at unit level
- TestInjectPlaybookMisses_RoleGateAllowsSameRole: Forklift Operator
  recording fires on Forklift Operators query (plural normalization)
- TestInjectPlaybookMisses_RoleGateBackwardCompat: empty Role on
  either side = gate disabled, preserves current behavior
- TestApplyPlaybookBoost_RoleGateRejectsCrossRole: Shape A defense
  in depth — boost doesn't fire on cross-role even when answer is
  in cold top-K
- TestRoleEqual_PluralAndCase: case + -s + -es plural normalization

Verification (real_002, same query set as real_001):
- Q#5 Pickers @ Beacon Freight: e-6193 → e-8499 (no bleed)
- Q#10 CNC Operator @ Beacon Freight: e-6193 → w-2404 (no bleed)
- Discoveries + lifts unchanged at 2 each (same-role lift still fires)
- Mean Δdist tightens from -0.127 to -0.040 (boosts no longer
  pulling distances through the floor on cross-role mismatches)

Findings: reports/reality-tests/real_002_findings.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-30 20:34:10 -05:00
co-authored by Claude Opus 4.7
parent 7f2f112e6a
commit 997527be4d
8 changed files with 513 additions and 24 deletions
+36
View File
@@ -37,6 +37,7 @@ import (
"log"
"net/http"
"os"
"regexp"
"sort"
"strings"
"time"
@@ -481,6 +482,13 @@ func matrixSearch(hc *http.Client, gw, query string, corpora []string, k int, us
"per_corpus_k": k,
"use_playbook": usePlaybook,
}
// Mechanical role extraction (real_001 cross-role bleed fix).
// Empty when the query doesn't match the "Need N {role}{s} in"
// pattern — gate stays disabled and the harness behaves as before
// for the lift suite's free-form queries.
if role := extractRoleFromNeed(query); role != "" {
body["query_role"] = role
}
bs, _ := json.Marshal(body)
req, _ := http.NewRequest("POST", gw+"/v1/matrix/search", bytes.NewReader(bs))
req.Header.Set("Content-Type", "application/json")
@@ -508,6 +516,11 @@ func playbookRecord(hc *http.Client, gw, query, answerID, answerCorpus string, s
"score": score,
"tags": []string{"reality-test", "playbook-lift-001"},
}
// Same mechanical role extraction as matrixSearch — recorded role
// lets retrieve-time gate fire on cross-role queries (real_001 fix).
if role := extractRoleFromNeed(query); role != "" {
body["role"] = role
}
bs, _ := json.Marshal(body)
req, _ := http.NewRequest("POST", gw+"/v1/matrix/playbooks/record", bytes.NewReader(bs))
req.Header.Set("Content-Type", "application/json")
@@ -617,3 +630,26 @@ func appendNote(existing, add string) string {
// Suppress unused-import warning when sort isn't used in a future
// refactor; harmless for now.
var _ = sort.Slice
// extractRoleFromNeed pulls the role out of "Need N {role}{s} in {city}"
// shape queries — the fill_events-derived form used by real_NNN runs.
// Returns "" for any query that doesn't match (free-form lift-suite
// queries fall back to empty, leaving the cross-role gate disabled).
//
// The pattern is permissive: the count can be any digits, and the
// role is everything between the count and " in ". This catches
// "Need 5 Warehouse Associates in Kansas City" → "Warehouse Associates";
// roleEqual on the matrix side handles plurals + case.
//
// Lives here (not in internal/matrix) because role extraction from
// free-form text is a caller concern; matrix only consumes the
// already-resolved Role string. A future LLM-based extractor would
// replace this regex without changing matrix's gate logic.
func extractRoleFromNeed(query string) string {
re := regexp.MustCompile(`(?i)^Need\s+\d+\s+(.+?)\s+in\s+`)
m := re.FindStringSubmatch(query)
if len(m) < 2 {
return ""
}
return strings.TrimSpace(m[1])
}