Lift k cap, drop ornamental reason field, scenario generator
ITEM 1 — k CAP + REASON FIELD The hybrid_search default k was hard-coded to 10. For multi-fill events (5× expansion, 4× emergency) that's pool=10 → propose 5-of-10, half the candidates become the answer with no room for rejection. Executor prompt now instructs k to scale with target_count: k = max(count*5, 20), cap 80. Default helper bumped 10 → 20. Fill.reason dropped from required to optional. Nothing downstream ever consumed it — resolveWorkerIds, sealSale, retrospective all use candidate_id and name. Models loved to write 100-150 char justifications per fill; on 4+ fills that blew the JSON budget before the structure closed. Test 1 run result after this change: FIRST EVER 5/5 on the Riverfront Steel scenario, 13 total turns across 5 events. The event that failed last run (emergency 4×Loader with truncated reason-field continuation) now clears in 2 turns. Progression: mistral baseline: 0/5 qwen3.5 + continuation + think:false: 4/5 qwen3.5 + k=20 + no-reason: 5/5 ✓ ITEM 2 — SCENARIO GENERATOR (NOT YET TESTED E2E) tests/multi-agent/gen_scenarios.ts emits N deterministic ScenarioSpecs with varied clients (15 companies), cities (20 Midwest cities known to exist in workers_500k), role mixes (14 industrial staffing roles, weighted realistic), and event sequences. Each gets a unique sig_hash so the KB populates with distinct neighbor signatures. scripts/run_kb_batch.sh runs all generated specs sequentially against scenario.ts, logs per-scenario outcomes, and reports KB state at the end. Each run takes ~2-4min; 20-30 scenarios = 1-2hr unattended. Next: test the generator+batch on a small N (3-5) to verify KB populates correctly and pathway recommendations start getting neighbor signal instead of cold-starts. Then item 3 (Rust re-weighting of hybrid_search by playbook_memory success).
This commit is contained in:
@@ -318,7 +318,11 @@ export type Action =
|
||||
export interface Fill {
|
||||
candidate_id: string;
|
||||
name: string;
|
||||
reason: string;
|
||||
reason?: string; // optional — the schema used to require it; nothing
|
||||
// downstream consumed it, and qwen3.5 would emit
|
||||
// 100-150 char justifications per fill that blew
|
||||
// the JSON budget on 4+ fills. Kept optional so
|
||||
// models that still emit it don't break parsing.
|
||||
}
|
||||
|
||||
// --- HTTP helpers (fail-fast) ---
|
||||
@@ -348,7 +352,10 @@ export async function callTool(tool: string, args: Record<string, any>): Promise
|
||||
});
|
||||
}
|
||||
|
||||
export async function hybridSearch(sql_filter: string, question: string, k = 10): Promise<any> {
|
||||
// Default k=20 is a floor, not a ceiling — executor prompt instructs
|
||||
// models to scale k to 5× target_count (cap 80) so multi-fill events
|
||||
// get a meaningfully deep pool to rank within.
|
||||
export async function hybridSearch(sql_filter: string, question: string, k = 20): Promise<any> {
|
||||
return http("POST", `${GATEWAY}/vectors/hybrid`, { sql_filter, question, k });
|
||||
}
|
||||
|
||||
@@ -448,7 +455,11 @@ Available tools (each takes a JSON "args" object):
|
||||
{"index_name":"workers_500k_v1",
|
||||
"sql_filter":"role = 'Forklift Operator' AND city = 'Toledo' AND state = 'OH' AND CAST(availability AS DOUBLE) > 0.5",
|
||||
"question":"reliable forklift operator Toledo",
|
||||
"k":10}
|
||||
"k":40}
|
||||
→ k should scale with target_count: roughly 5× the number of fills
|
||||
needed, floor 20, cap 80. For 1-2 fills use k=20. For 5 fills use
|
||||
k=40. A deep pool lets the ranker discriminate across a larger
|
||||
candidate set; k=10 was too tight for multi-fill events.
|
||||
|
||||
- sql(query: string)
|
||||
→ Raw read-only SELECT. Use for verification (confirm a worker exists,
|
||||
@@ -580,8 +591,8 @@ Your next action MUST be a JSON object matching one of these shapes:
|
||||
— use on turn 1 to outline your approach. Steps must be concrete.
|
||||
{"kind":"tool_call","tool":"...","args":{...},"rationale":"why"}
|
||||
— call a tool and see its result next turn.
|
||||
{"kind":"propose_done","fills":[{"candidate_id":"...","name":"First Last","reason":"why them"}],"rationale":"..."}
|
||||
— propose you've met the target. fills MUST have EXACTLY ${task.target_count} entries — count twice before emitting.
|
||||
{"kind":"propose_done","fills":[{"candidate_id":"...","name":"First Last"}],"rationale":"..."}
|
||||
— propose you've met the target. fills MUST have EXACTLY ${task.target_count} entries — count twice before emitting. Each fill is ONLY {candidate_id, name} — no reason field, no scores, no commentary.
|
||||
|
||||
Strategy tip: once "CANDIDATES SURFACED SO FAR" has ≥ ${task.target_count} entries in ${task.target_city}, ${task.target_state} matching ${task.target_role}, verify ONE via the sql tool (to satisfy the reviewer's SQL-verification criterion) and then propose_done with the top ${task.target_count}. Don't keep re-searching.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user