profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

qwen3.5 executor + continuation primitive + think:false

Browse Source 
Three coupled fixes that together turned the Riverfront Steel scenario
from 0/5 (mistral) to 4/5 (qwen3.5) with T3 flagging real staffing
concerns rather than linter advice.

MODEL SWAP
- Executor: mistral → qwen3.5:latest (9.7B, 262K ctx, thinking).
  mistral's decoder emitted malformed JSON on complex SQL filters
  regardless of prompt; J called it — stop using mistral.
- Reviewer: qwen2.5 → qwen3:latest (40K ctx)
- Applied to scenario.ts, orchestrator.ts, network_proving.ts,
  run_e2e_rated.ts

CONTINUATION PRIMITIVE (agent.ts)
- generateContinuable(): empty-response → geometric backoff retry;
  truncated-JSON → continue from partial as scratchpad; bounded by
  budget cap + max_continuations. No more "bump max_tokens until it
  stops truncating" tourniquet.
- generateTreeSplit(): map-reduce for oversized input corpora with
  running scratchpad digest, reduce pass for final synthesis.
- Empty text no longer throws — it's a signal to continuable that
  thinking ate the budget.

think:false FOR HOT PATH
- qwen3.5 burned ~650 tokens of hidden thinking for trivial JSON
  emission. For executor/reviewer/draft: think:false. For T3/T4/T5
  overseers: thinking stays on (that's the point).
- Sidecar generate endpoint accepts `think` bool, passes through to
  Ollama's /api/generate.

VERIFIED OUTCOMES
Riverfront Steel 2026-04-21, qwen3.5+continuable+think:false:
  08:00 baseline_fill  3/3  4 turns
  10:30 recurring      2/2  3 turns (1 playbook citation)
  12:15 expansion      0/5  drift-aborted (5-fill orchestration
                            problem, separate work)
  14:00 emergency      4/4  3 turns (1 citation)
  15:45 misplacement   1/1  3 turns
  → T3 caught Patrick Ross double-booking across events
  → T3 flagged forklift cert drift on the event that failed
  → Cross-day lesson proposed "maintain buffer of ≥3 emergency
    candidates, pre-fetch certs for expansion, booking system
    cross-check" — real staffing advice, not generic linter output

PRD PHASE 21 rewritten to reflect the actual primitive shape (two-
call map-reduce with scratchpad glue) instead of the tourniquet
approach originally documented. Rust port queued for next sprint.

scripts/ab_t3_test.sh: A/B harness that chains B→C→D runs and emits
tests/multi-agent/playbooks/ab_scorecard.json.
This commit is contained in:
root
2026-04-20 20:19:02 -05:00
parent 6e7ca1830e
commit 0c4868c191
9 changed files with 410 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
config/models.json
+16 -7
View File
@@ -22,9 +22,9 @@
"t1_hot": {
"purpose": "Per tool call — SQL generation, hybrid_search, sql(). Runs 50-200 times per scenario. Latency-sensitive.",
"kind": "local_fast",
"primary": { "model": "mistral:latest", "provider": "ollama_local", "context_window": 32768 },
"primary": { "model": "qwen3.5:latest", "provider": "ollama_local", "context_window": 262144 },
"fallback": { "model": "qwen2.5:latest", "provider": "ollama_local", "context_window": 32768 },
"max_tokens": 800,
"max_tokens": 1000,
"temperature": 0.3,
"never_route_cloud": true,
"context_budget": {
@@ -34,14 +34,14 @@
"safety_margin": 2000,
"overflow_policy": "summarize_oldest_tool_results_via_t3"
},
"rationale": "Mistral produces valid JSON reliably. Qwen2.5 is the consensus reviewer. Known flakiness on 5-fill + misplacement events — do NOT mask by upgrading; route to T3 for post-hoc review instead."
"rationale": "qwen3.5:latest is a 9.7B thinking model with 262K context that emits clean JSON (verified 2026-04-21 after mistral A/B wipeout). Thinking budget requires max_tokens ≥ 400; we hold 1000 to keep propose_done payloads intact. qwen2.5 stays as fallback."
},
"t2_review": {
"purpose": "Per step consensus — executor ↔ reviewer loop critique. 5-14 calls per event.",
"kind": "local_balanced",
"primary": { "model": "qwen2.5:latest", "provider": "ollama_local", "context_window": 32768 },
"fallback": { "model": "qwen3:latest", "provider": "ollama_local", "context_window": 40960 },
"max_tokens": 600,
"primary": { "model": "qwen3:latest", "provider": "ollama_local", "context_window": 40960 },
"fallback": { "model": "qwen2.5:latest", "provider": "ollama_local", "context_window": 32768 },
"max_tokens": 800,
"temperature": 0.3,
"never_route_cloud": true,
"context_budget": {
@@ -72,7 +72,8 @@
"t4_strategic": {
"purpose": "Daily playbook board re-ranking, weekly gap audit, pattern discovery across accumulated playbooks. 1-10 calls per day.",
"kind": "thinking_cloud_large",
"primary": { "model": "qwen3.5:397b", "provider": "ollama_cloud", "context_window": 131072 },
"primary": { "model": "kimi-k2.6", "provider": "ollama_cloud", "context_window": 200000 },
"secondary": { "model": "qwen3.5:397b", "provider": "ollama_cloud", "context_window": 131072 },
"fallback": { "model": "glm-4.7", "provider": "ollama_cloud", "context_window": 131072 },
"local_fallback": { "model": "gpt-oss:20b", "provider": "ollama_local", "context_window": 131072 },
"max_tokens": 2000,
@@ -108,6 +109,14 @@
}
},
"embeddings": {
"_description": "Vector embedding models for hybrid search + playbook_memory similarity. Always local — embedding calls are too high-volume for cloud. qwen3-embedding is the new primary (2026-04-21); nomic-embed-text-v2-moe is the MoE alternative for corpus-scale embedding; original nomic-embed-text stays as legacy fallback while vectord indexes are being re-embedded.",
"primary": { "model": "qwen3-embedding", "provider": "ollama_local", "dim": 1024, "context_window": 32768 },
"alternate": { "model": "nomic-embed-text-v2-moe", "provider": "ollama_local", "dim": 768, "context_window": 2048, "notes": "MoE variant — better for long documents, slower per-call, roughly matches nomic-embed-text on short text." },
"legacy": { "model": "nomic-embed-text:latest", "provider": "ollama_local", "dim": 768, "context_window": 8192, "notes": "Used by existing vectord indexes. Do NOT switch live until indexes are rebuilt, or hybrid search recall will crater." },
"migration_plan": "Re-embed workers_500k (and all production indexes) with qwen3-embedding, keep the old index under a _v1 suffix for rollback, flip the profile pointer only after recall benchmarks match or exceed legacy. Touches vectord::agent + autotune."
},
"context_management": {
"_description": "Rule zero: NEVER call a model with more tokens than its context_window minus safety_margin. Every call goes through the budget checker first. If over budget → chunk, summarize, or escalate. This is the stability floor.",
"token_estimator": {