qwen3.5 executor + continuation primitive + think:false

Three coupled fixes that together turned the Riverfront Steel scenario
from 0/5 (mistral) to 4/5 (qwen3.5) with T3 flagging real staffing
concerns rather than linter advice.

MODEL SWAP
- Executor: mistral → qwen3.5:latest (9.7B, 262K ctx, thinking).
  mistral's decoder emitted malformed JSON on complex SQL filters
  regardless of prompt; J called it — stop using mistral.
- Reviewer: qwen2.5 → qwen3:latest (40K ctx)
- Applied to scenario.ts, orchestrator.ts, network_proving.ts,
  run_e2e_rated.ts

CONTINUATION PRIMITIVE (agent.ts)
- generateContinuable(): empty-response → geometric backoff retry;
  truncated-JSON → continue from partial as scratchpad; bounded by
  budget cap + max_continuations. No more "bump max_tokens until it
  stops truncating" tourniquet.
- generateTreeSplit(): map-reduce for oversized input corpora with
  running scratchpad digest, reduce pass for final synthesis.
- Empty text no longer throws — it's a signal to continuable that
  thinking ate the budget.

think:false FOR HOT PATH
- qwen3.5 burned ~650 tokens of hidden thinking for trivial JSON
  emission. For executor/reviewer/draft: think:false. For T3/T4/T5
  overseers: thinking stays on (that's the point).
- Sidecar generate endpoint accepts `think` bool, passes through to
  Ollama's /api/generate.

VERIFIED OUTCOMES
Riverfront Steel 2026-04-21, qwen3.5+continuable+think:false:
  08:00 baseline_fill  3/3  4 turns
  10:30 recurring      2/2  3 turns (1 playbook citation)
  12:15 expansion      0/5  drift-aborted (5-fill orchestration
                            problem, separate work)
  14:00 emergency      4/4  3 turns (1 citation)
  15:45 misplacement   1/1  3 turns
  → T3 caught Patrick Ross double-booking across events
  → T3 flagged forklift cert drift on the event that failed
  → Cross-day lesson proposed "maintain buffer of ≥3 emergency
    candidates, pre-fetch certs for expansion, booking system
    cross-check" — real staffing advice, not generic linter output

PRD PHASE 21 rewritten to reflect the actual primitive shape (two-
call map-reduce with scratchpad glue) instead of the tourniquet
approach originally documented. Rust port queued for next sprint.

scripts/ab_t3_test.sh: A/B harness that chains B→C→D runs and emits
tests/multi-agent/playbooks/ab_scorecard.json.

tests/multi-agent/agent.ts

@@ -22,6 +22,214 @@ export function estimateTokens(text: string): number {
   return Math.ceil(text.length / 4);
 }
 
+// ============================================================
+// Scratchpad + tree-split continuation
+// ============================================================
+//
+// Core problem: when a prompt OR response would exceed a model's window
+// — e.g. 200 playbooks pasted in, or a long propose_done payload — we
+// cannot just raise max_tokens forever. That's the tourniquet approach
+// J explicitly rejected. The right answer is: split into sub-calls
+// glued by a persistent scratchpad, so no context is ever lost.
+//
+// Two primitives:
+//
+// 1. generateContinuable(): for OUTPUT overflow. If the model returns
+//    a structurally-incomplete JSON (unmatched braces, truncated
+//    mid-value), auto-continue with the partial as scratchpad until
+//    the JSON parses or max_continuations is hit.
+//
+// 2. generateTreeSplit(): for INPUT overflow. If prompt + system +
+//    max_tokens exceeds window, shard the input at semantic boundaries,
+//    run each shard through the model with a running scratchpad digest,
+//    then run a final reduce pass to combine. This is map-reduce with
+//    glue.
+//
+// Both are tier-agnostic — they wrap generate() and generateCloud().
+
+// Check structural completeness of a response. For non-JSON responses
+// (lesson text, checkpoint hints) we treat "non-empty" as complete.
+// For JSON-shaped responses (executor/reviewer actions) we check that
+// braces match AND that JSON.parse succeeds on the first {...} block.
+function isStructurallyComplete(text: string, shape: "json" | "text"): boolean {
+  if (!text || !text.trim()) return false;
+  if (shape === "text") return true;
+  // Strip ``` fences
+  let s = text.trim();
+  if (s.startsWith("```")) s = s.replace(/^```(?:json)?\n?/, "").replace(/```$/, "").trim();
+  const start = s.indexOf("{");
+  const end = s.lastIndexOf("}");
+  if (start < 0 || end <= start) return false;
+  const slice = s.slice(start, end + 1);
+  // Balance check — cheaper than JSON.parse and catches truncated nests
+  let depth = 0, inStr = false, esc = false;
+  for (const c of slice) {
+    if (esc) { esc = false; continue; }
+    if (c === "\\") { esc = true; continue; }
+    if (c === '"') { inStr = !inStr; continue; }
+    if (inStr) continue;
+    if (c === "{") depth++;
+    else if (c === "}") depth--;
+  }
+  if (depth !== 0) return false;
+  try { JSON.parse(slice); return true; } catch { return false; }
+}
+
+// Continue a truncated response. We do NOT ask the model to start over —
+// we ask it to continue from exactly where it stopped. The partial goes
+// in as scratchpad so it knows what's already committed.
+async function continueResponse(
+  model: string,
+  originalPrompt: string,
+  partial: string,
+  opts: { max_tokens: number; temperature: number; system?: string; cloud: boolean; think?: boolean },
+): Promise<string> {
+  const continuationPrompt = `${originalPrompt}
+
+PARTIAL RESPONSE SO FAR (continue from here — do NOT restart, do NOT repeat what's already there, emit ONLY the remaining tokens to close the structure):
+${partial}`;
+  const fn = opts.cloud ? generateCloud : generate;
+  const rest = await fn(model, continuationPrompt, {
+    max_tokens: opts.max_tokens,
+    temperature: opts.temperature,
+    system: opts.system,
+    bypass_budget: true, // the caller already checked; continuation doesn't double-count
+    think: opts.think,
+  });
+  return partial + rest;
+}
+
+// Output-overflow handler. Handles two distinct failure modes:
+//   (a) EMPTY response — thinking model ate the whole budget. Retry the
+//       ORIGINAL prompt with 2x the budget (geometric backoff up to cap).
+//   (b) TRUNCATED non-empty — model got most of the way there but hit
+//       max_tokens before closing. Continue with the partial as
+//       scratchpad.
+// The two modes need different repair: (a) needs more budget, not
+// continuation from ""; (b) needs scratchpad-glued continuation.
+export async function generateContinuable(
+  model: string,
+  prompt: string,
+  opts: {
+    max_tokens?: number;
+    temperature?: number;
+    system?: string;
+    shape?: "json" | "text";
+    max_continuations?: number;
+    cloud?: boolean;
+    think?: boolean;
+    on_continuation?: (n: number, combined_len: number) => void;
+  } = {},
+): Promise<string> {
+  const shape = opts.shape ?? "json";
+  const initialMax = opts.max_tokens ?? 800;
+  const maxConts = opts.max_continuations ?? 3;
+  const cloud = opts.cloud ?? false;
+  const budgetCap = 8000; // don't geometric-backoff forever
+  const fn = cloud ? generateCloud : generate;
+
+  let combined = "";
+  let currentMax = initialMax;
+
+  // Initial call + empty-response backoff loop.
+  for (let retry = 0; retry < 3; retry++) {
+    const out = await fn(model, prompt, {
+      max_tokens: currentMax,
+      temperature: opts.temperature,
+      system: opts.system,
+      think: opts.think,
+    });
+    if (out.trim().length > 0) { combined = out; break; }
+    // Empty — thinking model ate the budget. Double it and retry.
+    if (opts.on_continuation) opts.on_continuation(retry + 1, 0);
+    currentMax = Math.min(currentMax * 2, budgetCap);
+  }
+
+  // Structural completion loop (continuation from partial).
+  for (let i = 0; i < maxConts; i++) {
+    if (isStructurallyComplete(combined, shape)) return combined;
+    if (opts.on_continuation) opts.on_continuation(i + 1, combined.length);
+    combined = await continueResponse(model, prompt, combined, {
+      max_tokens: Math.min(currentMax, budgetCap),
+      temperature: opts.temperature ?? 0.3,
+      system: opts.system,
+      cloud,
+      think: opts.think,
+    });
+  }
+  // Last-resort: return combined even if incomplete; caller's parser
+  // will throw with the raw text for forensics rather than silently
+  // truncating.
+  return combined;
+}
+
+// Input-overflow handler. When the input corpus exceeds the window,
+// shard → map → reduce with a running scratchpad digest.
+//
+// shards: array of input chunks the caller already split at semantic
+//   boundaries (paragraphs, records, playbook entries).
+// map_prompt: fn taking (shard, running_scratchpad) → prompt for a
+//   single map call. Must fit within window.
+// reduce_prompt: fn taking (combined_scratchpad) → final prompt. Also
+//   must fit window; if the scratchpad itself overflows, this triggers
+//   a recursive tree-split.
+//
+// Result: the reduce call's response.
+export async function generateTreeSplit(
+  model: string,
+  shards: string[],
+  map_prompt: (shard: string, scratchpad: string) => string,
+  reduce_prompt: (scratchpad: string) => string,
+  opts: {
+    max_tokens?: number;
+    temperature?: number;
+    system?: string;
+    cloud?: boolean;
+    on_shard?: (i: number, total: number) => void;
+    scratchpad_budget?: number;
+  } = {},
+): Promise<{ response: string; scratchpad: string; shards_processed: number }> {
+  const cloud = opts.cloud ?? false;
+  const scratchpadBudget = opts.scratchpad_budget ?? 6000;
+  let scratchpad = "";
+
+  for (let i = 0; i < shards.length; i++) {
+    if (opts.on_shard) opts.on_shard(i + 1, shards.length);
+    const shardPrompt = map_prompt(shards[i], scratchpad);
+    // If the per-shard prompt alone exceeds window, the caller sharded
+    // too coarsely — bubble up rather than silently truncating.
+    assertContextBudget(model, shardPrompt, {
+      system: opts.system,
+      max_tokens: opts.max_tokens,
+      bypass: false,
+    });
+    const shardOut = await generateContinuable(model, shardPrompt, {
+      max_tokens: opts.max_tokens ?? 800,
+      temperature: opts.temperature,
+      system: opts.system,
+      shape: "text",
+      cloud,
+    });
+    // Append to scratchpad; truncate oldest if over budget.
+    scratchpad = (scratchpad + `\n— shard ${i + 1}/${shards.length} digest —\n` + shardOut.trim()).slice(-scratchpadBudget * 4);
+  }
+
+  const reducePrompt = reduce_prompt(scratchpad);
+  assertContextBudget(model, reducePrompt, {
+    system: opts.system,
+    max_tokens: opts.max_tokens,
+    bypass: false,
+  });
+  const response = await generateContinuable(model, reducePrompt, {
+    max_tokens: opts.max_tokens ?? 1500,
+    temperature: opts.temperature,
+    system: opts.system,
+    shape: "text",
+    cloud,
+  });
+  return { response, scratchpad, shards_processed: shards.length };
+}
+
 // Known context windows — matches crates/../config/models.json. Kept in
 // code as a fallback so the test harness doesn't crash if the config is
 // missing. Production path should read from models.json.
@@ -29,10 +237,14 @@ export const CONTEXT_WINDOWS: Record<string, number> = {
   "mistral:latest": 32768,
   "qwen2.5:latest": 32768,
   "qwen3:latest": 40960,
+  "qwen3.5:latest": 262144,          // local 9.7B — new executor
+  "qwen3-embedding": 32768,          // local embedding model
+  "nomic-embed-text-v2-moe": 2048,   // local embedding, MoE
   "gpt-oss:20b": 131072,
   "gpt-oss:120b": 131072,
   "qwen3.5:397b": 131072,
   "kimi-k2-thinking": 200000,
+  "kimi-k2.6": 200000,               // cloud — new T4 candidate
   "kimi-k2:1t": 1048576,
   "deepseek-v3.1:671b": 131072,
   "glm-4.7": 131072,
@@ -89,6 +301,7 @@ export interface LogEntry {
     | "critique"
     | "propose_done"
     | "consensus_done"
+    | "note"
     | "error";
   content: any;
 }
@@ -150,6 +363,7 @@ export async function generate(model: string, prompt: string, opts: {
   temperature?: number;
   system?: string;
   bypass_budget?: boolean;
+  think?: boolean;
 } = {}): Promise<string> {
   assertContextBudget(model, prompt, {
     system: opts.system,
@@ -163,11 +377,13 @@ export async function generate(model: string, prompt: string, opts: {
     max_tokens: opts.max_tokens ?? 800,
   };
   if (opts.system) body.system = opts.system;
+  if (opts.think !== undefined) body.think = opts.think;
   const r = await http<any>("POST", `${SIDECAR}/generate`, body);
-  const text = r.text ?? "";
-  if (!text || typeof text !== "string") {
-    throw new Error(`generate returned empty text from ${model}: ${JSON.stringify(r).slice(0, 200)}`);
-  }
+  const text = typeof r.text === "string" ? r.text : "";
+  // Do NOT throw on empty. Thinking models (gpt-oss, qwen3.5) burn the
+  // max_tokens budget on hidden reasoning and emit "" when budget was
+  // too tight. generateContinuable detects empty + continues with more
+  // budget. Callers that expected non-empty can check themselves.
   return text;
 }
 
@@ -181,6 +397,7 @@ export async function generateCloud(model: string, prompt: string, opts: {
   temperature?: number;
   system?: string;
   bypass_budget?: boolean;
+  think?: boolean;
 } = {}): Promise<string> {
   if (!OLLAMA_CLOUD_KEY) {
     throw new Error("OLLAMA_CLOUD_KEY not set; cannot reach Ollama Cloud");
@@ -200,6 +417,7 @@ export async function generateCloud(model: string, prompt: string, opts: {
     },
   };
   if (opts.system) body.system = opts.system;
+  if (opts.think !== undefined) body.think = opts.think;
   const resp = await fetch(`${OLLAMA_CLOUD_URL}/api/generate`, {
     method: "POST",
     headers: {
@@ -212,10 +430,9 @@ export async function generateCloud(model: string, prompt: string, opts: {
     throw new Error(`Ollama Cloud ${resp.status}: ${await resp.text().catch(() => "?")}`);
   }
   const data: any = await resp.json();
-  const text = data.response ?? "";
-  if (!text) {
-    throw new Error(`Ollama Cloud returned empty response for ${model}: ${JSON.stringify(data).slice(0, 200)}`);
-  }
+  const text = typeof data.response === "string" ? data.response : "";
+  // Same non-throw policy as local generate() — empty text is a valid
+  // signal that thinking ate the budget. Let generateContinuable retry.
   return text;
 }
 

tests/multi-agent/network_proving.ts

@@ -49,8 +49,8 @@ import {
   callTool,
 } from "./agent.ts";
 
-const EXECUTOR_MODEL = "mistral:latest";
-const REVIEWER_MODEL = "qwen2.5:latest";
+const EXECUTOR_MODEL = "qwen3.5:latest";
+const REVIEWER_MODEL = "qwen3:latest";
 const VERIFIER_MODEL = "qwen2.5:latest";
 const PROFILE_ID = "staffing-recruiter";
 const INDEX_NAME = "workers_500k_v1";
@@ -175,7 +175,7 @@ async function buildPhase(task: TaskSpec, prefix: string): Promise<BuildResult>
     while (turn < MAX_TURNS && !sealed) {
       turn += 1;
 
-      const execRaw = await generate(EXECUTOR_MODEL, executorPrompt(task, log), { temperature: 0.2, max_tokens: 600 });
+      const execRaw = await generate(EXECUTOR_MODEL, executorPrompt(task, log), { temperature: 0.2, max_tokens: 1200 });
       const execAction = parseAction(execRaw, "executor");
       append({ turn, role: "executor", model: EXECUTOR_MODEL, kind: execAction.kind as any, content: execAction });
 
@@ -192,7 +192,7 @@ async function buildPhase(task: TaskSpec, prefix: string): Promise<BuildResult>
         }
       }
 
-      const revRaw = await generate(REVIEWER_MODEL, reviewerPrompt(task, log), { temperature: 0.1, max_tokens: 400 });
+      const revRaw = await generate(REVIEWER_MODEL, reviewerPrompt(task, log), { temperature: 0.1, max_tokens: 1000 });
       const revAction = parseAction(revRaw, "reviewer");
       append({ turn, role: "reviewer", model: REVIEWER_MODEL, kind: "critique", content: revAction });
 

tests/multi-agent/orchestrator.ts

@@ -25,8 +25,8 @@ import {
 import { mkdir, writeFile } from "node:fs/promises";
 import { join } from "node:path";
 
-const EXECUTOR_MODEL = "mistral:latest";
-const REVIEWER_MODEL = "qwen2.5:latest";
+const EXECUTOR_MODEL = "qwen3.5:latest";
+const REVIEWER_MODEL = "qwen3:latest";
 const MAX_TURNS = 12;                 // executor turns; reviewer gets one per
 const MAX_CONSECUTIVE_DRIFTS = 3;     // drift-cycle blown → give up
 
@@ -148,7 +148,7 @@ async function main() {
       // --- EXECUTOR TURN ---
       const execRaw = await generate(EXECUTOR_MODEL, executorPrompt(task, log), {
         temperature: 0.2,
-        max_tokens: 600,
+        max_tokens: 1200,
       });
       let execAction: Action;
       try {
@@ -191,7 +191,7 @@ async function main() {
       // --- REVIEWER TURN ---
       const revRaw = await generate(REVIEWER_MODEL, reviewerPrompt(task, log), {
         temperature: 0.1,
-        max_tokens: 400,
+        max_tokens: 1000,
       });
       let revAction: Action;
       try {

tests/multi-agent/run_e2e_rated.ts

@@ -32,8 +32,8 @@ import {
   callTool,
 } from "./agent.ts";
 
-const EXECUTOR_MODEL = "mistral:latest";
-const REVIEWER_MODEL = "qwen2.5:latest";
+const EXECUTOR_MODEL = "qwen3.5:latest";
+const REVIEWER_MODEL = "qwen3:latest";
 const MAX_TURNS = 12;
 const MAX_CONSECUTIVE_DRIFTS = 3;
 const INDEX_NAME = "workers_500k_v1";
@@ -75,7 +75,7 @@ async function runOrchestrator(task: TaskSpec, prefix: string): Promise<RunResul
       turn += 1;
 
       // Executor
-      const execRaw = await generate(EXECUTOR_MODEL, executorPrompt(task, log), { temperature: 0.2, max_tokens: 600 });
+      const execRaw = await generate(EXECUTOR_MODEL, executorPrompt(task, log), { temperature: 0.2, max_tokens: 1200 });
       const execAction = parseAction(execRaw, "executor");
       append({ turn, role: "executor", model: EXECUTOR_MODEL, kind: execAction.kind as any, content: execAction });
 
@@ -95,7 +95,7 @@ async function runOrchestrator(task: TaskSpec, prefix: string): Promise<RunResul
       }
 
       // Reviewer
-      const revRaw = await generate(REVIEWER_MODEL, reviewerPrompt(task, log), { temperature: 0.1, max_tokens: 400 });
+      const revRaw = await generate(REVIEWER_MODEL, reviewerPrompt(task, log), { temperature: 0.1, max_tokens: 1000 });
       const revAction = parseAction(revRaw, "reviewer");
       append({ turn, role: "reviewer", model: REVIEWER_MODEL, kind: "critique", content: revAction });
 

tests/multi-agent/scenario.ts

@@ -29,6 +29,7 @@ import {
   sqlQuery,
   generate,
   generateCloud,
+  generateContinuable,
   parseAction,
   executorPrompt,
   reviewerPrompt,
@@ -37,15 +38,14 @@ import {
 import { mkdir, writeFile, appendFile } from "node:fs/promises";
 import { join } from "node:path";
 
-// 2026-04-20 — reverted to mistral executor after trying qwen2.5.
-// qwen2.5 emits malformed JSON (trailing `)` garbage, unterminated
-// strings) when asked for tool calls. mistral drops fields occasionally
-// but produces valid JSON. With optional `question` default + lean
-// prompt + schema lock, mistral seals baseline + recurring reliably.
-// Complex scenarios (5-fill, emergency, misplacement) remain flaky —
-// real Phase 20+ problem (larger model or constrained decoding needed).
-const EXECUTOR_MODEL = "mistral:latest";
-const REVIEWER_MODEL = "qwen2.5:latest";
+// 2026-04-21 — executor is now qwen3.5:latest (9.7B, 262K context,
+// thinking model, emits clean JSON). Replaces mistral, which produced
+// malformed JSON on complex SQL filters (bare IN-clause identifiers,
+// unclosed braces) regardless of prompt — decoder-level bug that all
+// 5 events hit across 4 A/B test runs. qwen3.5 tested clean on first
+// try with 800 max_tokens.
+const EXECUTOR_MODEL = "qwen3.5:latest";
+const REVIEWER_MODEL = "qwen3:latest";
 const DRAFT_MODEL = "qwen2.5:latest";     // artifact generation; short outputs
 
 // T3 overview tier. Called sparingly — NOT per tool call. Two insertion
@@ -81,9 +81,16 @@ const T3_DISABLED = process.env.LH_T3_DISABLE === "1";
 
 // Dispatcher: route T3 calls to local sidecar or Ollama Cloud depending
 // on the LH_OVERVIEW_CLOUD flag. Hot-path T1/T2 always stay local.
+// T3 outputs are free-form prose (lesson/hint), so shape=text — the
+// continuation primitive treats any non-empty response as complete.
 async function overviewGenerate(prompt: string, opts: { temperature?: number; max_tokens?: number } = {}): Promise<string> {
-  if (OVERVIEW_CLOUD) return generateCloud(OVERVIEW_MODEL, prompt, opts);
-  return generate(OVERVIEW_MODEL, prompt, opts);
+  return generateContinuable(OVERVIEW_MODEL, prompt, {
+    temperature: opts.temperature,
+    max_tokens: opts.max_tokens ?? 1000,
+    shape: "text",
+    max_continuations: 2,
+    cloud: OVERVIEW_CLOUD,
+  });
 }
 
 const MAX_TURNS = 14;
@@ -365,10 +372,26 @@ async function runAgentFill(
   while (turn < MAX_TURNS && !sealed) {
     turn += 1;
 
-    const execRaw = await generate(
+    // generateContinuable: if the model truncates mid-JSON (thinking
+    // ate the budget, or payload was just long), auto-continue with the
+    // partial as scratchpad until braces balance and JSON parses.
+    // No more "bump max_tokens until it stops truncating" tourniquet.
+    // think:false — executor emits structured JSON, doesn't need hidden
+    // reasoning. Burning ~650 thinking tokens on a 400-token JSON was
+    // exactly the bug we just solved.
+    const execRaw = await generateContinuable(
       EXECUTOR_MODEL,
       withExtras(executorPrompt(task, log)),
-      { temperature: 0.2, max_tokens: 600 },
+      {
+        temperature: 0.2,
+        max_tokens: 800,
+        shape: "json",
+        max_continuations: 3,
+        think: false,
+        on_continuation: (n, len) =>
+          append({ turn, role: "executor", model: EXECUTOR_MODEL, kind: "note",
+            content: { continuation: n, combined_chars: len } }),
+      },
     );
     let execAction: Action;
     try {
@@ -419,10 +442,19 @@ async function runAgentFill(
       }
     }
 
-    const revRaw = await generate(
+    const revRaw = await generateContinuable(
       REVIEWER_MODEL,
       withExtras(reviewerPrompt(task, log)),
-      { temperature: 0.1, max_tokens: 400 },
+      {
+        temperature: 0.1,
+        max_tokens: 600,
+        shape: "json",
+        max_continuations: 3,
+        think: false,
+        on_continuation: (n, len) =>
+          append({ turn, role: "reviewer", model: REVIEWER_MODEL, kind: "note",
+            content: { continuation: n, combined_chars: len } }),
+      },
     );
     let revAction: Action;
     try {
@@ -867,7 +899,10 @@ HINT: <hint>`;
 
   let text = "";
   try {
-    text = await overviewGenerate(prompt, { temperature: 0.2, max_tokens: 600 });
+    // overviewGenerate routes through generateContinuable — if thinking
+    // ate the initial budget, it auto-continues rather than requiring
+    // us to guess a safe max_tokens upfront.
+    text = await overviewGenerate(prompt, { temperature: 0.2, max_tokens: 800 });
   } catch (e) {
     return {
       after_event: event.at,