multi-coord stress Phase 2: ExcludeIDs + fresh-resume + 200-worker swap
Three Phase 2 additions land in this commit: 1. matrix.SearchRequest gains ExcludeIDs ([]string) — filters specific worker IDs out of results post-retrieval, AND skips them at the playbook boost+inject step (so excluded answers can't sneak back via Shape B). Real-world driver: coordinator placed N workers, client asks for replacements, system needs alternatives, not the same N. Threaded through retrieve.go after merge but before metadata filter so excluded IDs don't waste post-filter top-K slots. 2. New harness phase 2b: 200-worker swap simulation. Captures the top-K from alpha's warehouse query, then re-issues with exclude_ids=<placed>. Result Jaccard(orig, swap) measures whether the substrate finds genuine alternatives. 3. New harness phase 1b: fresh-resume mid-run injection. Three new workers ingested via /v1/embed + /v1/vectors/index/workers/add, then verified findable via semantic queries matching resume content. Plus Hour labels on every event (operational narrative: 0/6/12/18/ 24/30/36/42/48) and a refactor of captureEvent to take hour as a param. Run #003 + #004 results (5K workers + 10K ethereal): Diversity (#004): Same-role-across-contracts Jaccard = 0.080 (n=9) Different-roles-same-contract Jaccard = 0.013 (n=18) Determinism: 1.000 (#004 unchanged) Verbatim handover: 4/4 = 100% Paraphrase handover: 4/4 = 100% Phase 2b — 200-worker swap (Jaccard 0.000): 8 originally-placed workers fully replaced by 8 alternatives. ExcludeIDs substrate change works end-to-end — boost AND inject both honor the exclusion, so excluded workers don't return via the playbook either. Phase 1b — fresh-resume injection: REAL PRODUCT FINDING. Substrate ABSORPTION is fine — 3 /v1/vectors/index/workers/add calls at 200 status, 3 vectors persisted. But none of the 3 fresh workers surfaced in top-8 even with semantic queries matching their resume content (e.g. "Senior tower crane rigger NCCCO Chicago" vs fresh-001's resume "Senior rigger with 12 years tower-crane signaling..." NCCCO + Chicago). Top-1 came from existing workers at distance ~0.25; fresh workers' distances must be > 0.25, pushing them past rank 8. Cause: dense retrieval at 5000+ workers means many existing profiles cluster near any specific query in cosine space; nomic-embed-text-v2 (137M) introduces enough noise that a fresh worker doesn't reliably outrank them just because the text content overlaps. Workarounds (Phase 3 work): (a) hybrid retrieval (keyword + semantic), (b) playbook-layer score boost for fresh adds, (c) larger embedder. Documented in run #004 report. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -85,6 +85,15 @@ type SearchRequest struct {
|
||||
PlaybookMaxDistance float64 `json:"playbook_max_distance,omitempty"`
|
||||
PlaybookMaxInjectDistance float64 `json:"playbook_max_inject_distance,omitempty"`
|
||||
MetadataFilter map[string]any `json:"metadata_filter,omitempty"`
|
||||
// ExcludeIDs filters out specific worker IDs post-retrieval.
|
||||
// Real-world driver: a coordinator places 200 workers at a
|
||||
// contract, then mid-day the client asks for a different set —
|
||||
// the next query should NOT return the already-placed workers.
|
||||
// Filter runs after merge but before metadata filter, so an
|
||||
// excluded ID never wastes a slot in the post-filter top-K.
|
||||
// Also applies to playbook boost + Shape B inject — excluded
|
||||
// answers are skipped at injection time.
|
||||
ExcludeIDs []string `json:"exclude_ids,omitempty"`
|
||||
}
|
||||
|
||||
// SearchResponse wraps the merged results plus per-corpus return
|
||||
@@ -204,6 +213,25 @@ func (r *Retriever) Search(ctx context.Context, req SearchRequest) (*SearchRespo
|
||||
return allHits[i].Distance < allHits[j].Distance
|
||||
})
|
||||
|
||||
// ExcludeIDs filter — applied first so excluded IDs don't waste
|
||||
// a slot in the post-filter top-K. Real-world driver: coordinator
|
||||
// has placed N workers at a contract; mid-day the client asks for
|
||||
// alternatives, so this query passes ExcludeIDs=<placed_ids> and
|
||||
// gets back fresh candidates instead of the same N.
|
||||
if len(req.ExcludeIDs) > 0 {
|
||||
excludeSet := make(map[string]bool, len(req.ExcludeIDs))
|
||||
for _, id := range req.ExcludeIDs {
|
||||
excludeSet[id] = true
|
||||
}
|
||||
kept := make([]Result, 0, len(allHits))
|
||||
for _, h := range allHits {
|
||||
if !excludeSet[h.ID] {
|
||||
kept = append(kept, h)
|
||||
}
|
||||
}
|
||||
allHits = kept
|
||||
}
|
||||
|
||||
// Metadata filter (component B — staffing-side structured gate).
|
||||
// Applied BEFORE top-K truncation so the filter doesn't accidentally
|
||||
// reduce coverage further. Caller can request larger PerCorpusK to
|
||||
@@ -239,6 +267,23 @@ func (r *Retriever) Search(ctx context.Context, req SearchRequest) (*SearchRespo
|
||||
if err != nil {
|
||||
slog.Warn("matrix: playbook lookup failed; skipping boost+inject", "err", err)
|
||||
} else if len(hits) > 0 {
|
||||
// Filter playbook hits to honor ExcludeIDs — without this,
|
||||
// an excluded answer in a playbook recording would re-enter
|
||||
// the result set via Shape B inject, defeating the swap
|
||||
// semantics that the exclude list exists to enforce.
|
||||
if len(req.ExcludeIDs) > 0 {
|
||||
excludeSet := make(map[string]bool, len(req.ExcludeIDs))
|
||||
for _, id := range req.ExcludeIDs {
|
||||
excludeSet[id] = true
|
||||
}
|
||||
keptHits := make([]PlaybookHit, 0, len(hits))
|
||||
for _, h := range hits {
|
||||
if !excludeSet[h.Entry.AnswerID] {
|
||||
keptHits = append(keptHits, h)
|
||||
}
|
||||
}
|
||||
hits = keptHits
|
||||
}
|
||||
resp.PlaybookBoosted = ApplyPlaybookBoost(resp.Results, hits)
|
||||
maxInjectDist := float32(req.PlaybookMaxInjectDistance)
|
||||
if maxInjectDist <= 0 {
|
||||
|
||||
Reference in New Issue
Block a user