Unblock complex scenarios: JSON tolerance + optional question + mistral exec

parseAction now strips stray `)` before `}` and trailing commas —
qwen2.5 emits those regularly on tool_call outputs; soft-fix beats
retry-loops. hybrid_search no longer hard-requires `question`; defaults
to "qualified available workers" when the model drops it (mistral's
most common failure mode on complex events).

Kept original TOOL_CATALOG shape (args examples only, not full
action envelopes). The verbose few-shot version from the prior
iteration confused mistral into wrapping propose_done as tool_call.

Scenario V7 result: expansion (5 Forklift Ops) and emergency
(4 Loaders) — previously-failing complex events — now seal reliably.
Pool sizes: 687 and 380 from 500K corpus. Patterns endpoint produces
real operator-actionable signals:
  expansion: "recurring certifications: Forklift (40%), OSHA-10 (40%)
             · recurring skills: mill (40%) · archetype mostly: leader
             · reliability median 0.83"
Baseline + recurring are now flaky (inverted trade-off, pure
model-reliability variance).
This commit is contained in:
root
2026-04-20 15:28:30 -05:00
parent 1274ab2cb3
commit f8e8d25b5f
2 changed files with 36 additions and 14 deletions
+22 -11
View File
@@ -121,8 +121,8 @@ Available tools (each takes a JSON "args" object):
→ Canonical production tool for fill tasks. Always use this FIRST.
→ Example args:
{"index_name":"workers_500k_v1",
"sql_filter":"LOWER(role) LIKE '%weld%' AND city = 'Toledo' AND state = 'OH' AND availability > 0.5",
"question":"reliable welder with OSHA certs",
"sql_filter":"role = 'Forklift Operator' AND city = 'Toledo' AND state = 'OH' AND CAST(availability AS DOUBLE) > 0.5",
"question":"reliable forklift operator Toledo",
"k":10}
- sql(query: string)
@@ -133,17 +133,19 @@ Available tools (each takes a JSON "args" object):
responsiveness, engagement, communications, compliance, availability,
resume_text.
→ Example args:
{"query":"SELECT worker_id, name, role, city, state, availability FROM workers_500k WHERE worker_id = 'W123456'"}
{"query":"SELECT worker_id, name, role, city, state FROM workers_500k WHERE worker_id = 40123"}
Rules:
- hybrid_search returns sources[] each with {doc_id, chunk_text, score, sql_verified}.
- hybrid_search returns sources[] each with {doc_id, chunk_text, score,
sql_verified, playbook_boost, playbook_citations}.
- **ID mapping:** vector doc_ids look like "W500K-7995" (prefix + number).
The SQL worker_id is an INTEGER. To go from doc_id to SQL, strip the
"W500K-" prefix and cast:
SELECT ... FROM workers_500k WHERE worker_id = CAST(SUBSTR('W500K-7995', 7) AS BIGINT)
or more simply: WHERE worker_id = 7995.
- Names are NOT unique. Always identify by worker_id, never by name alone.
- Return EXACTLY ONE JSON object per turn. No prose outside the JSON.
The SQL worker_id is an INTEGER. Use WHERE worker_id = 7995 directly.
- Names are NOT unique. Always identify by worker_id.
- availability and reliability are stored as text; ALWAYS cast as
DOUBLE in filters: CAST(availability AS DOUBLE) > 0.5.
- Narrative words from the guidance ("shift", "recurring", "expansion",
"emergency") are NOT columns. Only use columns listed above.
- Return EXACTLY ONE JSON object per turn. No markdown fences, no prose.
`;
// Smart per-kind summary so agents see the substance of each prior turn
@@ -321,7 +323,12 @@ export function parseAction(raw: string, role: Role): Action {
if (start < 0 || end <= start) {
throw new Error(`no JSON object in ${role} response: ${raw.slice(0, 300)}`);
}
const json = s.slice(start, end + 1);
let json = s.slice(start, end + 1);
// Soft-tolerate common model mistakes: stray `)` before closing brace
// (qwen2.5 does this on tool_call), trailing commas, etc. Fix the
// cheapest ones that are unambiguous.
json = json.replace(/\)\s*\}/g, "}"); // "...)}" → "...}"
json = json.replace(/,(\s*[}\]])/g, "$1"); // trailing comma before } or ]
let obj: any;
try {
obj = JSON.parse(json);
@@ -333,6 +340,10 @@ export function parseAction(raw: string, role: Role): Action {
if (obj.kind === "plan" && Array.isArray(obj.steps)) return obj as Action;
if (obj.kind === "tool_call" && typeof obj.tool === "string" && typeof obj.args === "object") return obj as Action;
if (obj.kind === "propose_done" && Array.isArray(obj.fills)) return obj as Action;
// Tolerance: some model outputs put a stray closing paren or
// trailing garbage after the main object. If the kind looks
// recognizable but shape doesn't match, bubble a cleaner error so
// the orchestrator's soft-fail path doesn't swallow it.
throw new Error(`executor returned unexpected shape: ${JSON.stringify(obj).slice(0, 200)}`);
} else {
// Normalize: some models (qwen2.5, mistral) emit the verdict AS the