demo: real synthetic headshots — fetch pool + serve route + UI wire

Three layers shipped: 1. SCRIPT — scripts/staffing/fetch_face_pool.py Pulls N synthetic StyleGAN faces from thispersondoesnotexist.com into data/headshots/face_NNNN.jpg, writes manifest.jsonl. Idempotent: re-running skips existing files. Optional gender tagging via deepface (currently unavailable on this box; the script handles ImportError gracefully and tags everything as untagged). Fetched 198 faces with concurrency=3 in ~67s. 2. SERVER — /headshots/:key route in mcp-server/index.ts Loads manifest at first hit, caches in globalThis._faces. Hashes the key with djb2-style mixing → pool index → returns the JPG. Same key always gets the same face (deterministic). Accepts ?g=man|woman&e=caucasian|black|hispanic|south_asian|east_asian|middle_eastern to bias pool selection — the gender/ethnicity buckets fall back to the full pool when no tagged matches exist. Cache-Control: 86400 immutable so faces ride the browser cache after first hit. /headshots/__reload re-reads the manifest without restart. 3. UI — search.html + console.html worker cards Re-added overlay <img> on top of the monogram .av circle. img.src = /headshots/<encoded-key>?g=<hint>&e=<hint>. img.onerror removes the failed image so the monogram stays visible if the face pool isn't fetched / CDN is blocked. .av now has overflow:hidden + position:relative to clip the img to a perfect circle. Forced-confident name resolution (J: "we're CREATING the profile, created as though you truly have the information Xavier is more likely Hispanic and he's a male"): genderFor(name) — looks up MALE_NAMES + FEMALE_NAMES, falls back to a deterministic hash split so unknown names spread ~50/50. Sets now include cross-cultural names: Alejandro/ Andres/Mateo/Santiago/Joaquin/Cesar/Hugo/ Felipe/Gerardo/Salvador/Ramon (Hispanic), Raj/Anil/Vikram/Krishna/Pradeep (South Asian), Wei/Yi/Hiroshi/Akira/Hyun (East Asian), Demetrius/Kareem/DaQuan/Khalil (Black), Omar/Khalid/Hassan/Ahmed/Bilal (Middle Eastern). FEMALE_NAMES extended in parallel. guessEthnicityFromFirstName(name) — confident default of 'caucasian' for any name not in the cultural buckets so every worker resolves to a category the face pool can be biased toward. Order: ME → Black → Hispanic → South Asian → East Asian → Caucasian (matters where names overlap, e.g. Aisha appears in ME + Black, biases toward ME for visual fit). Both helpers also ported into console.html so the triage backfills and try-it-yourself rendering get the same hint stack. Privacy note in the script + route comments: the synthetic data uses the worker's name as the seed; production should hash worker_id (not name) to avoid leaking PII to a third-party CDN. The fetch URL itself is referenced once per pool build, not per-worker. .gitignore — added data/headshots/face_*.jpg (~100MB for 198 faces; the manifest + script are tracked). Re-running the script on a fresh checkout rebuilds the pool from scratch. Verified end-to-end via playwright on devop.live/lakehouse: forklift query → 10 worker cards 10/10 with face images (real synthetic headshots, not monograms) 0/10 broken Alejandro G. Nelson → ?g=man&e=hispanic Patricia K. Garcia → ?g=woman&e=caucasian Each name → unique face, deterministic across loads. Console triage backfills get the same treatment.
demo: console — sober worker cards (mirror dashboard styling)
2026-04-28 00:04:03 -05:00 · 2026-04-27 23:47:12 -05:00 · 2026-04-27 23:43:36 -05:00 · 2026-04-27 23:22:07 -05:00 · 2026-04-27 23:13:46 -05:00 · 2026-04-27 23:04:37 -05:00
40 changed files with 617 additions and 5745 deletions
@@ -6,9 +6,5 @@ __pycache__/
 *.pyc
 # Headshot pool — binary face JPGs are fetched by scripts/staffing/fetch_face_pool.py
-# (synthetic StyleGAN, ~580MB for 1000 faces). Manifest + fetch script are tracked.
+# (synthetic StyleGAN, ~100MB for 200 faces). Manifest + fetch script are tracked.
 data/headshots/face_*.jpg
 data/headshots/_thumbs/
 # ComfyUI on-demand generated portraits (per-worker unique). Cached on first
 # request; fully regeneratable via /headshots/generate/:key.
 data/headshots_gen/
@@ -48,7 +48,6 @@ version = "0.1.0"
 dependencies = [
 "async-trait",
 "axum",
 "lru",
 "reqwest",
 "serde",
 "serde_json",
@@ -1,239 +0,0 @@
 # STATE OF PLAY — Lakehouse
 **Last verified:** 2026-04-27 ~20:35 CDT
 **Verified by:** live probe, not memory.
 > **Read this FIRST.** When the user says "we're working on lakehouse," they mean the working code captured below — NOT what `git log` framed as "the cutover" or what memory snapshots from 2 days ago suggest. If memory contradicts this file, this file wins. Update it when something is verified working — not when a phase finishes.
 ---
 ## VERIFIED WORKING RIGHT NOW
 ### The client demo (Staffing Co-Pilot)
 **Public URL:** `https://devop.live/lakehouse/` — 200, "Staffing Co-Pilot" (159 KB SPA, leaflet maps, dark theme).
 **Local URL:** `http://localhost:3700/` — same page, served by `mcp-server/index.ts` (PID 1271, started 09:48 CDT today).
 **The staffers console** (the one the client was thoroughly impressed with):
 - `https://devop.live/lakehouse/console` — 200, "Lakehouse — What Your Staffing System Would Do" (26 KB)
 - Pulls project index via `/api/catalog/datasets` (36 datasets) + playbook memory via `/api/vectors/playbook_memory/stats` (4,701 entries with embeddings, real ops like *"fill: Maintenance Tech x2 in Milwaukee, WI"*)
 Client-visible flow that works end-to-end on the public URL:
 | Endpoint | Sample output |
 |---|---|
 | `GET /api/catalog/datasets` | 36 datasets indexed: timesheets 1M, call_log 800K, workers_500k 500K, email_log 500K, workers_100k 100K, candidates 100K, placements 50K, job_orders 15K, successful_playbooks_live 2,077 |
 | `GET /api/vectors/playbook_memory/stats` | 4,701 fill operations with embeddings |
 | `GET /system/summary` | 36 datasets, 2.98M rows, 60 indexes, 500K workers loaded, 1K candidates |
 | `POST /intelligence/staffing_forecast` | 744 Production Workers needed in 30d, 11,281 bench (4,687 reliable), coverage 1,444%, risk=ok. Same for Electrician (need 32, bench 2,440) and Maintenance Tech (need 17, bench 5,004). |
 | `POST /intelligence/permit_contracts` | permit `3442956` $500K → 3 Production Workers, 886-candidate pool, 95% fill, $36K gross. 5 more Chicago permits with 8 workers each, same pool, 95% fill, $96K each. |
 | `POST /intelligence/market` | major Chicago permits ranked: $730M O'Hare, $615M 307 N Michigan, $580M casino, $445M Loop transit (real geo coords). |
 | `POST /intelligence/permit_entities` | architects + contractors from permit contacts (e.g. "KACPRZYNSKI, ANDY", "SLS ELECTRICAL SERVICE"). |
 | `POST /intelligence/activity` + `/intelligence/arch_signals` + `/intelligence/chat` | all 200 |
 The demo tells the story: *"upcoming Chicago contracts → workers needed → coverage from the bench → architects/contractors involved → revenue and margin."* That's the "live data + anticipating contracts + complete workflow" pitch — working as of right now.
 ### Backend, verified live this session
 | Surface | State |
 |---|---|
 | Gateway `:3100` | up, 4 providers configured, `/v1/health` 200 with 500K workers loaded |
 | MCP server `:3700` (Co-Pilot demo) | up, all `/intelligence/*` endpoints respond |
 | VCP UI `:3950` | started this session, `/data/*` 200, real numbers |
 | Observer `:3800` | ring full (2,000/2,000) — older events evicted, query Langfuse for 24h-ago state |
 | Sidecar `:3200` | up |
 | Langfuse `:3001` | recording, `gw:/log` + `v1.chat:openrouter` traces visible |
 | LLM Team UI `:5000` | up, only `extract` mode registered |
 | OpenCode fleet | **40 models reachable through one `sk-*` key** (verified live `GET https://opencode.ai/zen/v1/models`) |
 OpenCode catalog (live):
 - Claude: opus-4-7, opus-4-6, opus-4-5, opus-4-1, sonnet-4-6, sonnet-4-5, sonnet-4, haiku-4-5
 - GPT-5: 5.5-pro, 5.5, 5.4-pro, 5.4, 5.4-mini, 5.4-nano, 5.3-codex-spark, 5.3-codex, 5.2, 5.2-codex, 5.1-codex-max, 5.1-codex, 5.1-codex-mini, 5.1, 5-codex, 5-nano, 5
 - Gemini: 3.1-pro, 3-flash
 - GLM: 5.1, 5
 - Minimax: m2.7, m2.5
 - Kimi: k2.6, k2.5
 - Qwen: 3.6-plus, 3.5-plus
 - Other: BIG-PKL (was a typo-prone name in the catalog, model id starts with "big-pkl-something")
 - Free tier: minimax-m2.5-free, hy3-preview-free, ling-2.6-flash-free, trinity-large-preview-free
 ### The substrate (frozen — do not re-architect)
 - Distillation v1.0.0 at tag `e7636f2` — **145/145 bun tests pass, 22/22 acceptance, 16/16 audit-full**
 - Output: `data/_kb/distilled_{facts,procedures,config_hints}.jsonl` + `data/vectors/distilled_{factual,procedural,config_hint}_v20260423102847.parquet`
 - Auditor cross-lineage: Kimi K2.6 ↔ Haiku 4.5 alternation, Opus auto-promote on diffs >100k chars, **per-PR cap=3 with auto-reset on new head SHA**
 - Pathway memory: 88 traces, 11/11 successful replays (probation gate crossed)
 - Mode runner: 5 native modes; `codereview_isolation` is default; composed-corpus auto-downgrade verified Apr 26 (composed lost 5/5 vs isolation, p=0.031)
 ### Matrix indexer
 30+ live corpora including:
 - 5 versions of `workers_500k_v1..v9` (50K embedded chunks each)
 - 11 batched 2K-row shards `w500k_b3..b17`
 - `chicago_permits_v1` (3,420), `resumes_100k_v2` (100K candidates), `ethereal_workers_v1` (10K)
 - `lakehouse_arch_v1` (2,119), `lakehouse_symbols_v1` (2,470), `lakehouse_answers_v1` (1,269), `scrum_findings_v1` (1,260)
 - `kb_team_runs_v1` (12,693) + `kb_team_runs_agent` (4,407) — LLM-team play history embedded
 - `distilled_factual_v20260423102507` (8) — distillation output
 ### Code health
 - `cargo check --workspace` → **0 warnings, 0 errors**
 - `bun test auditor + tests/distillation` → **145/145 pass**
 - `ui/server.ts` + `auditor.ts` bundle clean
 ---
 ## DO NOT RELITIGATE
 - **PR #11 is merged into `origin/main` as `ed57eda`** — do not "still need to merge PR #11."
 - **Distillation tag `distillation-v1.0.0` at `e7636f2` is FROZEN** — do not re-architect schemas, scorer rules, audit fixtures.
 - **Kimi forensic HOLD verdict (2026-04-27) was 2/8 false + 6/8 latent** — do not re-debate, see `reports/kimi/audit-last-week-full.md`.
 - **`candidates_safe` `vertical` column bug** — fixed at catalog metadata layer in commit `c3c9c21`. Do not "discover" it again.
 - **Decisions A/B/C/D from `synthetic-data-gap-report.md`** — all four scripts shipped today (`d56f08e`, `940737d`, `c3c9c21`). Do not "ask J for approval."
 - **`workers_500k.phone` type fixup** — already string. The fixup script is idempotent; running it is a no-op.
 - **`client_workerskjkk` typo dataset** — was breaking every SQL query (catalog had it registered, file didn't exist). Removed via `DELETE /catalog/datasets/by-name/client_workerskjkk` this session. Do not re-add. Adding a startup gate that errors on unrecognized parquet names is the long-term fix per now.md Step 2C.
 ---
 ## FIXES MADE THIS SESSION (2026-04-27 evening)
 1. **`crates/gateway/src/v1/iterate.rs:93`** — `state` → `_state` (cleared the one cargo warning).
 2. **`lakehouse-ui.service` (Dioxus)** — disabled. Was failing 7,242 times against a missing `target/dx/ui/debug/web/public` build dir. `systemctl stop && disable`.
 3. **VCP UI on `:3950`** — started `bun run ui/server.ts` (PID 1162212, log `/tmp/lakehouse_ui.log`). `/data/*` endpoints now 200 with real data.
 4. **`client_workerskjkk` catalog entry** — `DELETE /catalog/datasets/by-name/client_workerskjkk` removed the dead manifest. **This was the actual root cause** of `/system/summary` reporting `workers_500k_rows: 0` and the demo showing zero bench. Every SQL query was failing schema inference on the missing file before reaching its target table. Fixed → `workers_500k_rows: 500000`, `candidates_rows: 1000`, demo coverage flipped from "critical 0%" to actual percentages on devop.live/lakehouse.
 ## FIXES MADE THIS SESSION (2026-04-28 early — face pool)
 5. **Synthetic StyleGAN face pool — 1000 faces, gender+race+age tagged.** `scripts/staffing/fetch_face_pool.py` fetches from thispersondoesnotexist.com; `scripts/staffing/tag_face_pool.py --min-age 22` runs deepface and excludes minors. `data/headshots/manifest.jsonl` now has gender (494 men / 458 women), race (caucasian 662 · east_asian 128 · hispanic 86 · middle_eastern 59 · black 14 · south_asian 3), age, and 48 minor exclusions. Server pool = 952 servable faces.
 6. **`mcp-server/index.ts:1308` `/headshots/:key` route** — gender×race×age intersection bucketing with graceful fallback (gender-only → all). Same key always returns same face; different keys spread evenly.
 7. **`/headshots/_thumbs/` pre-resized 384×384 webp** (60× smaller: 587KB → ~11KB). Without this, 40-card grids overran Chrome's parallel-connection budget and ~75% of tiles never finished decoding. Generated via parallel ffmpeg (`xargs -P 8`); `.gitignore`d.
 8. **`mcp-server/search.html` + `console.html`** — dropped `img.loading='lazy'`. With 11KB thumbs, eager load is cheap (~500KB for 50 cards) and avoids the off-screen race that lazy decode produced.
 9. **ComfyUI on-demand uniqueness — `serve_imagegen.py:32`** added `seed` to `_cache_key()` (was caching by prompt only — 3 different worker seeds collapsed to 1 cached image). Verified: seed=839185194/195/196 → 3 distinct md5s.
 10. **`mcp-server/index.ts:1234` `/headshots/generate/:key`** — ComfyUI hot-path that derives a deterministic-per-worker seed via djb2-style hash; cold ~1.5s, cached ~1ms. Worker prompt format: `professional corporate headshot portrait of a {age}-year-old {race} {gender}, {role}, neutral expression, plain studio background, soft natural lighting, sharp focus, photorealistic, dslr`. Cache at `data/headshots_gen/` (gitignored, regeneratable).
 11. **Confidence-default name resolution** in `search.html` — `genderFor()` and `guessEthnicityFromFirstName()` lookup tables (FEMALE_NAMES, MALE_NAMES, NAMES_HISPANIC, NAMES_BLACK, NAMES_SOUTH_ASIAN, NAMES_EAST_ASIAN, NAMES_MIDDLE_EASTERN). Xavier → man+hispanic, Aisha → woman+black, etc. Every worker resolves to a face-pool bucket.
 End-to-end verified: playwright run on `https://devop.live/lakehouse/?q=forklift+operators+IL` → 21/21 cards loaded, 0 broken, all 384×384 webp thumbs.
 ---
 ## OPEN — but not blocking the demo
 | Item | What | When to act |
 |---|---|---|
 | `modes.toml` `staffing_inference.matrix_corpus` | still says `workers_500k_v8`. v9 in vector index is from Apr 17 (raw-sourced, not safe-view). The new `build_workers_v9.sh` rebuilds from `workers_safe`. | Run when you have 30+ min for the rebuild. |
 | Open PRs #6, #7, #10 | sitting since Apr 22-24, auditor verdicts on disk at `data/_auditor/kimi_verdicts/{6,7,10}-*.json` | Read verdicts, decide reconcile/close. |
 | `test/enrich-prd-pipeline` branch | 35 unmerged commits, includes more-evolved auditor/inference.ts (666 vs main's 580 lines), curation+fact-extractor wiring | Reconcile or formally archive — see `memory/project_unmerged_architecture_work.md`. |
 | `federation-hnsw-trials` stash | Lance + S3/MinIO prototype, `aws-config` crate added, 708 insertions | Phase B from EXECUTION_PLAN.md — revisit when Parquet vector ceiling actually hurts. |
 | `candidates` manifest drift | manifest 100K vs SQL 1K. Cosmetic. | Run a metadata resync if it matters. |
 ---
 ## RUNTIME CHEATSHEET
 ```bash
 # Verify the demo (public + local both work)
 curl -sS https://devop.live/lakehouse/                   # Co-Pilot HTML
 curl -sS https://devop.live/lakehouse/console            # staffers console
 curl -sS -X POST https://devop.live/lakehouse/intelligence/staffing_forecast \
  -d '{}' -H 'content-type: application/json' \
  | jq '.forecast[] | {role, demand_workers, bench_total, coverage_pct, risk}'
 # Restart sequence (after Rust changes)
 sudo systemctl restart lakehouse.service                 # gateway :3100
 sudo systemctl restart lakehouse-auditor                 # auditor daemon
 sudo systemctl restart lakehouse-observer                # observer :3800
 # UI bun on :3950 is NOT systemd-managed (lakehouse-ui.service is disabled).
 # Restart manually:  kill <pid>; nohup bun run ui/server.ts > /tmp/lakehouse_ui.log 2>&1 &
 # Health checks
 curl -sS http://localhost:3100/v1/health | jq            # workers_count, providers
 curl -sS http://localhost:3100/vectors/pathway/stats | jq
 curl -sS http://localhost:3100/v1/usage | jq             # since-restart cost
 curl -sS http://localhost:3700/system/summary | jq       # dataset counts
 ```
 ---
 ## VISION — what we're actually building (not what's done)
 J's framing for the legacy staffing company:
 - Pull live data, anticipate contracts based on Chicago permits → real architect/contractor associations, headcount, time period, money, scope.
 - Hybrid + memory index → search large corpora cheaply.
 - Email comes in → verify against contract; SMS comes in → alert when index changes.
 - Real-time.
 - Invent metrics nobody else has using the hybrid index.
 - Next stage: workers download an app → geolocation clock-in → automatic responsiveness measurement, no user effort, with incentives for using it.
 - Find people getting certificates (passive cert tracking).
 - Pull union data → bring contracts that work for **employees**, not just employers.
 - All metrics visible, nothing hidden, value-aligned with what each side actually needs.
 If a future session is shaving away from this vision toward "fix the cutover" or "land Phase X," the vision wins. Phases are scaffolding for the vision, not the goal.
 ---
 ## CURRENT PLAN — fix the demo for the legacy staffing client
 Built from playwright audit of the live demo (2026-04-27 evening). Each item ends in something the client can SEE, not internal cleanups.
 **Demo state is anchored by git tag `demo-2026-04-27`** (commit `ed57eda`, the merge of PR #11). To restore code state: `git checkout demo-2026-04-27`. To restore runtime state: `DELETE /catalog/datasets/by-name/client_workerskjkk` (catalog hot-fix is not in git).
 ### P1 — Search box that actually filters (highest visible impact)
 **Problem:** typing in `#sq` and pressing Enter fires `POST /intelligence/chat` with body `{"message":"<query>"}`. The state (`#sst`) and role (`#srl`) selects are ignored — never sent in the body. So every search returns a generic chat completion, never a SQL+vector hybrid filter against `workers_500k`. That is the "cached/generic response" the client sees.
 **Fix:** in `mcp-server/search.html`, change the search-submit handler to call the real worker search endpoint with `{query, state, role, top_k}`. The MCP `search_workers` tool surface already exists; route the form there. Render returned worker rows in the existing card grid.
 **Done when:** typing "forklift" + state IL + role "Forklift Operator" returns ≤ top_k IL Forklift Operators, and changing state to WI returns different workers.
 ### P2 — Contractor-name click → `/contractor` profile page
 **Problem:** clicking a contractor name in any rendered card stays on `/lakehouse/`. URL doesn't change.
 **Fix:** wrap contractor names in `<a href="/contractor?name=<encoded>">`. The page `mcp-server/contractor.html` (14.8 KB, "Contractor Profile · Staffing Co-Pilot") already exists at `/contractor` and the data endpoint `/intelligence/contractor_profile` already returns rich data.
 **Then check contractor.html actually shows:** full history of every record the database has on that contractor + heat map of locations underneath + relevant info (per J 2026-04-27). If the page is incomplete, finish it. Otherwise just wire the link.
 **Done when:** clicking "KACPRZYNSKI, ANDY" opens a profile with: every Chicago permit they're contact_1 or contact_2 on, a leaflet map with markers for each address, and any matched workers from prior placements at their sites.
 ### P3 — Substrate signal at the bottom shows the right numbers
 **Problem:** J reports the bottom panel says "playbook memory empty, 80 traces 0 replies." Reality from the live endpoints: `/api/vectors/playbook_memory/stats` = 4,701 entries with embeddings; `/vectors/pathway/stats` = 88 traces, 11/11 replays.
 **Fix:** find the renderer in search.html that builds the substrate signal panel; verify it's hitting the right endpoints and reading the right keys; fix shape mismatches.
 **Done when:** bottom panel shows real numbers (4,701 playbooks, 88 traces, 11/11 replays) and references at least one specific recent operation from the playbook stats sample.
 ### P4 — Top nav reflects today's architecture
 **Problem:** Walkthrough/Architecture/Spec/Onboard/Alerts/Workspaces tabs all return 200 but content is from old architecture. Doesn't mention: gateway scratchpad, memory indexer, ranker, mode runner, OpenCode 40-model fleet, distillation substrate, auditor cross-lineage.
 **Fix:** rewrite `mcp-server/proof.html` (or add a single new page "What's running" that replaces Architecture+Spec) to describe what's actually shipped as of `demo-2026-04-27`. Keep one architecture page, drop redundancy. Either complete or hide Onboard/Alerts/Workspaces — J's call which.
 **Done when:** the architecture page tells a non-technical reader, in 2 minutes, what each piece does in coordinator-relatable terms ("intern that read every email", not "3-stage adversarial inference pipeline").
 ### P5 — Caching for the project-index build_signal (J flagged unfinished)
 **Problem:** "we never finished our caching for project index build signal it's not pulling new information." Need to find what `build_signal` refers to. Likely a scrum/auditor signal that should rebuild the `lakehouse_arch_v1` corpus on commit but isn't wired to.
 **Fix:** identify the build-signal pipeline (likely in `auditor/` or `crates/vectord/`), wire its emit to a corpus rebuild, verify by making a test commit and watching the new chunk appear in `/vectors/indexes` for `lakehouse_arch_v1`.
 **Done when:** committing a new file to `crates/` causes `lakehouse_arch_v1` chunk_count to increase within N minutes.
 ### P0 — Anchor the demo state (DONE)
 Tagged `ed57eda` as `demo-2026-04-27`. Future sessions: `git checkout demo-2026-04-27` to land in this exact code state.
 ---
 ## EXECUTION ORDER
 1. **P1 first** — biggest visible bug, ~30-60 min
 2. **P2 next** — contractor click is the second-biggest "doesn't work" the client sees, ~20 min if profile is mostly done
 3. **P3** — small fix, big "looks alive" win
 4. **P4** — biggest scope; might split across sessions
 5. **P5** — feature work, only after the visible bugs are fixed
 Each item commits independently with the format `demo: P<n> — <one-line>` so the commit log doubles as a progress journal. After each merge to main, re-tag `demo-latest` to point at the new HEAD.
 Stop here and let J pick which item to start with. Do not silently extend scope.
@@ -16,14 +16,12 @@ import type { Gap, Proposal } from "./types.ts";
 // Phase 44 migration (2026-04-27): bot/propose.ts now flows through
 // the gateway's /v1/chat instead of hitting the sidecar's /generate
 // directly. /v1/usage tracks the call, Langfuse traces it, observer
-// sees it. Gateway owns the routing.
+// sees it. Same upstream model (CLOUD_MODEL gpt-oss:120b on
-//
+// Ollama Cloud) — gateway just owns the routing.
 // 2026-04-28: gpt-oss:120b → deepseek-v3.2 via Ollama Pro. Newer
 // DeepSeek revision, faster, still on the same OLLAMA_CLOUD_KEY.
 const GATEWAY_URL = process.env.LH_GATEWAY_URL ?? "http://localhost:3100";
 const REPO_ROOT = "/home/profit/lakehouse";
 const PRD_PATH = `${REPO_ROOT}/docs/PRD.md`;
-const CLOUD_MODEL = process.env.LH_BOT_MODEL ?? "deepseek-v3.2";
+const CLOUD_MODEL = process.env.LH_BOT_MODEL ?? "gpt-oss:120b";
 const MAX_TOKENS = 6000;
 export async function findGaps(): Promise<Gap[]> {
@@ -44,10 +44,7 @@ name = "staffing_inference"
 # pattern generalizes beyond code review.
 preferred_mode = "staffing_inference_lakehouse"
 fallback_modes = ["ladder", "consensus", "pipeline"]
-# 2026-04-28: gpt-oss-120b:free → kimi-k2.6 via Ollama Pro. Coding-
+default_model = "openai/gpt-oss-120b:free"
 # specialized, faster than gpt-oss, on the same OLLAMA_CLOUD_KEY so
 # no extra provider hop.
 default_model = "kimi-k2.6"
 matrix_corpus = "workers_500k_v8"
 [[task_class]]
@@ -61,9 +58,7 @@ matrix_corpus = "kb_team_runs_v1"
 name = "doc_drift_check"
 preferred_mode = "drift"
 fallback_modes = ["validator"]
-# 2026-04-28: gpt-oss:120b → gemini-3-flash-preview via Ollama Pro.
+default_model = "gpt-oss:120b"
 # Speed leader on factual checking, same OLLAMA_CLOUD_KEY.
 default_model = "gemini-3-flash-preview"
 matrix_corpus = "distilled_factual_v20260423095819"
 [[task_class]]
@@ -15,27 +15,22 @@
 [[provider]]
 name = "ollama"
-base_url = "http://localhost:11434"
+base_url = "http://localhost:3200"
 auth = "none"
 default_model = "qwen3.5:latest"
-# Hot-path local inference. No bearer needed — direct to Ollama as of
+# Hot-path local inference. No bearer needed — Python sidecar on
-# 2026-05-02 (Python sidecar's pass-through wrapper retired). Model
+# localhost handles the Ollama API. Model names are bare
-# names are bare (e.g. "qwen3.5:latest", not "ollama/qwen3.5:latest").
+# (e.g. "qwen3.5:latest", not "ollama/qwen3.5:latest").
 [[provider]]
 name = "ollama_cloud"
 base_url = "https://ollama.com"
 auth = "bearer"
 auth_env = "OLLAMA_CLOUD_KEY"
-default_model = "deepseek-v3.2"
+default_model = "gpt-oss:120b"
-# Cloud-tier Ollama (Pro plan as of 2026-04-28). Key resolved from
+# Cloud-tier Ollama. Key resolved from OLLAMA_CLOUD_KEY env at gateway
-# OLLAMA_CLOUD_KEY at gateway boot; Pro tier upgraded the account so
+# boot. Model-prefix routing: "cloud/<model>" auto-routes here
-# rate limits + model access widen without a key change. Model-prefix
+# (see gateway::v1::resolve_provider).
 # routing: "cloud/<model>" auto-routes here. 39-model fleet now
 # includes deepseek-v3.2, deepseek-v4-{flash,pro}, gemini-3-flash-
 # preview, glm-{5,5.1}, kimi-k2.6, qwen3-coder-next.
 # 2026-04-28: default upgraded gpt-oss:120b → deepseek-v3.2 (newest
 # DeepSeek revision; kimi-k2:1t still upstream-broken with HTTP 500).
 [[provider]]
 name = "openrouter"
@@ -43,7 +38,7 @@ base_url = "https://openrouter.ai/api/v1"
 auth = "bearer"
 auth_env = "OPENROUTER_API_KEY"
 auth_fallback_files = ["/home/profit/.env", "/root/llm_team_config.json"]
-default_model = "x-ai/grok-4.1-fast"
+default_model = "openai/gpt-oss-120b:free"
 # Multi-provider gateway. Covers Anthropic, Google, OpenAI, MiniMax,
 # Qwen, Gemma, etc. Key resolved via crates/gateway/src/v1/openrouter.rs
 # resolve_openrouter_key() — env first, then fallback files.
@@ -12,4 +12,3 @@ serde_json = { workspace = true }
 tracing = { workspace = true }
 reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
 async-trait = "0.1"
 lru = "0.12"
@@ -1,74 +1,28 @@
 use lru::LruCache;
 use reqwest::Client;
 use serde::{Deserialize, Serialize};
 use std::num::NonZeroUsize;
 use std::sync::Mutex;
 use std::sync::atomic::{AtomicU64, Ordering};
 use std::sync::Arc;
 use std::time::Duration;
-/// HTTP client for Ollama (post-2026-05-02 — sidecar dropped).
+/// HTTP client for the Python AI sidecar.
 ///
 /// `base_url` was historically the Python sidecar at `:3200`, which
 /// pass-through-proxied to Ollama at `:11434`. The sidecar added zero
 /// logic on the hot path (embed.py + generate.py + rerank.py +
 /// admin.py = ~120 LOC of pure Ollama wrappers), so this client now
 /// talks to Ollama directly and the sidecar process can be retired.
 ///
 /// What stayed Python: `lab_ui.py` + `pipeline_lab.py` (~888 LOC of
 /// dev-mode Streamlit-shape UIs) — those aren't on the runtime hot
 /// path and continue running for prompt experimentation.
 ///
 /// `generate()` has two transport modes:
-/// - When `gateway_url` is None (default), posts directly to Ollama's
+/// - When `gateway_url` is None (default), it posts to
-///   `${base_url}/api/generate`.
+///   `${base_url}/generate` (sidecar direct).
-/// - When `gateway_url` is `Some(url)`, posts to `${url}/v1/chat`
+/// - When `gateway_url` is `Some(url)`, it posts to
-///   with `provider="ollama"` so the call appears in `/v1/usage` and
+///   `${url}/v1/chat` with `provider="ollama"` so the call appears
-///   Langfuse traces.
+///   in `/v1/usage` and Langfuse traces.
 ///
-/// `embed()`, `rerank()`, and admin methods always go direct to
+/// `embed()`, `rerank()`, and admin methods always go direct to the
-/// Ollama — no `/v1` equivalent for those surfaces yet.
+/// sidecar — no `/v1` equivalent yet, no point round-tripping.
 ///
 /// Phase 44 part 2 (2026-04-27): the gateway URL is wired in by
 /// callers that want observability (vectord modules); it's left
 /// unset by callers that ARE the gateway internals (avoids self-loops
 /// + redundant hops).
 /// Per-text embed cache key. We key on (model, text) so different
 /// model selections produce distinct cache lines — a query embedded
 /// under nomic-embed-text-v2-moe must NOT collide with the same
 /// query under nomic-embed-text v1.
 #[derive(Eq, PartialEq, Hash, Clone)]
 struct EmbedCacheKey {
    model: String,
    text: String,
 }
 /// Default LRU cache size — 4096 entries × ~6KB per 768-d f64
 /// vector ≈ 24MB. Sized for typical staffing-domain repetition
 /// (coordinator workflows have query repetition rates around 70-90%
 /// per session). Tunable via [aibridge].embed_cache_size in the
 /// config; 0 disables the cache entirely.
 const DEFAULT_EMBED_CACHE_SIZE: usize = 4096;
 #[derive(Clone)]
 pub struct AiClient {
    client: Client,
    base_url: String,
    gateway_url: Option<String>,
    /// Closes the 63× perf gap with Go side. Mirrors the shape of
    /// Go's internal/embed/cached.go::CachedProvider — same
    /// (model, text) → vector caching, same nil-disable semantics.
    /// None = caching disabled (cache_size=0); Some = bounded LRU.
    embed_cache: Option<Arc<Mutex<LruCache<EmbedCacheKey, Vec<f64>>>>>,
    /// Hit / miss counters for /admin observability + load-test
    /// validation. Atomic so Clone'd AiClients share the same counts.
    embed_cache_hits: Arc<AtomicU64>,
    embed_cache_misses: Arc<AtomicU64>,
    /// Pinned at construction time so the EmbedResponse can carry
    /// dimension consistently even when every text was a cache hit
    /// (no fresh upstream call to learn the dim from). Set on first
    /// successful Ollama embed; checked on every cache hit.
    cached_dim: Arc<AtomicU64>,
 }
 // -- Request/Response types --
@@ -141,49 +95,17 @@ pub struct RerankResponse {
 impl AiClient {
    pub fn new(base_url: &str) -> Self {
        Self::with_embed_cache(base_url, DEFAULT_EMBED_CACHE_SIZE)
    }
    /// Constructs an AiClient with an explicit embed-cache size.
    /// Pass 0 to disable the cache entirely (matches Go-side
    /// CachedProvider's nil-cache semantics).
    pub fn with_embed_cache(base_url: &str, cache_size: usize) -> Self {
        let client = Client::builder()
            .timeout(Duration::from_secs(120))
            .build()
            .expect("failed to build HTTP client");
        let embed_cache = if cache_size > 0 {
            // SAFETY: cache_size > 0 just verified, NonZeroUsize::new
            // returns Some.
            let cap = NonZeroUsize::new(cache_size).expect("cache_size > 0");
            Some(Arc::new(Mutex::new(LruCache::new(cap))))
        } else {
            None
        };
        Self {
            client,
            base_url: base_url.trim_end_matches('/').to_string(),
            gateway_url: None,
            embed_cache,
            embed_cache_hits: Arc::new(AtomicU64::new(0)),
            embed_cache_misses: Arc::new(AtomicU64::new(0)),
            cached_dim: Arc::new(AtomicU64::new(0)),
        }
    }
    /// Cache hit/miss/size snapshot. Useful for /admin endpoints +
    /// load-test validation ("did the cache fire as expected?").
    pub fn embed_cache_stats(&self) -> (u64, u64, usize) {
        let hits = self.embed_cache_hits.load(Ordering::Relaxed);
        let misses = self.embed_cache_misses.load(Ordering::Relaxed);
        let len = self
            .embed_cache
            .as_ref()
            .map(|c| c.lock().map(|g| g.len()).unwrap_or(0))
            .unwrap_or(0);
        (hits, misses, len)
    }
    /// Same as `new`, but every `generate()` is routed through
    /// `${gateway_url}/v1/chat` (provider=ollama) for observability.
    /// Use this for callers OUTSIDE the gateway. Inside the gateway
@@ -196,222 +118,50 @@ impl AiClient {
        c
    }
    /// Reachability + version check. Hits Ollama's `/api/version`,
    /// returns a sidecar-shaped envelope so callers reading
    /// `.status` / `.ollama_url` don't break across the
    /// pre-/post-2026-05-02 cutover.
    pub async fn health(&self) -> Result<serde_json::Value, String> {
        let resp = self.client
-            .get(format!("{}/api/version", self.base_url))
+            .get(format!("{}/health", self.base_url))
            .send()
            .await
-            .map_err(|e| format!("ollama unreachable: {e}"))?;
+            .map_err(|e| format!("sidecar unreachable: {e}"))?;
-        let body: serde_json::Value = resp.json().await
+        resp.json().await.map_err(|e| format!("invalid response: {e}"))
            .map_err(|e| format!("invalid response: {e}"))?;
        Ok(serde_json::json!({
            "status": "ok",
            "ollama_url": &self.base_url,
            "ollama_version": body.get("version"),
        }))
    }
    /// Embed with per-text LRU caching. Mirrors Go-side
    /// CachedProvider behavior: cache key is (model, text);
    /// cache-hit texts skip the sidecar; cache-miss texts batch
    /// into a single sidecar call; results are interleaved in the
    /// caller's input order.
    ///
    /// Closes ~95% of the load-test perf gap vs Go side (loadgen
    /// 2026-05-01: Rust 128 RPS → with cache ≥ 7000 RPS expected
    /// for warm-cache workloads). Cold-cache behavior unchanged
    /// (every text is a miss → single sidecar call, identical to
    /// pre-cache).
    pub async fn embed(&self, req: EmbedRequest) -> Result<EmbedResponse, String> {
        let model_key = req.model.clone().unwrap_or_default();
        // Fast path: cache disabled → original behavior.
        let Some(cache) = self.embed_cache.as_ref() else {
            return self.embed_uncached(&req).await;
        };
        if req.texts.is_empty() {
            return self.embed_uncached(&req).await;
        }
        // First pass: check cache for each text. Track which positions
        // need a sidecar fetch.
        let mut embeddings: Vec<Option<Vec<f64>>> = vec![None; req.texts.len()];
        let mut miss_indices: Vec<usize> = Vec::new();
        let mut miss_texts: Vec<String> = Vec::new();
        {
            let mut guard = cache.lock().map_err(|e| format!("cache lock poisoned: {e}"))?;
            for (i, text) in req.texts.iter().enumerate() {
                let key = EmbedCacheKey { model: model_key.clone(), text: text.clone() };
                if let Some(vec) = guard.get(&key) {
                    embeddings[i] = Some(vec.clone());
                    self.embed_cache_hits.fetch_add(1, Ordering::Relaxed);
                } else {
                    miss_indices.push(i);
                    miss_texts.push(text.clone());
                    self.embed_cache_misses.fetch_add(1, Ordering::Relaxed);
                }
            }
        }
        // All hit? Return immediately. Use cached_dim to populate
        // the response dimension (no sidecar to ask).
        if miss_indices.is_empty() {
            let dim = self.cached_dim.load(Ordering::Relaxed) as usize;
            let dim = if dim == 0 { embeddings[0].as_ref().map(|v| v.len()).unwrap_or(0) } else { dim };
            return Ok(EmbedResponse {
                embeddings: embeddings.into_iter().map(|opt| opt.expect("filled")).collect(),
                model: req.model.unwrap_or_else(|| "nomic-embed-text".to_string()),
                dimensions: dim,
            });
        }
        // Second pass: fetch the misses in one sidecar call.
        let miss_req = EmbedRequest { texts: miss_texts.clone(), model: req.model.clone() };
        let resp = self.embed_uncached(&miss_req).await?;
        if resp.embeddings.len() != miss_texts.len() {
            return Err(format!(
                "embed cache: sidecar returned {} embeddings for {} texts",
                resp.embeddings.len(),
                miss_texts.len()
            ));
        }
        // Pin cached_dim on first successful response.
        if resp.dimensions > 0 {
            self.cached_dim.store(resp.dimensions as u64, Ordering::Relaxed);
        }
        // Insert misses into cache + fill response slots.
        {
            let mut guard = cache.lock().map_err(|e| format!("cache lock poisoned: {e}"))?;
            for (j, idx) in miss_indices.iter().enumerate() {
                let key = EmbedCacheKey {
                    model: model_key.clone(),
                    text: miss_texts[j].clone(),
                };
                let vec = resp.embeddings[j].clone();
                guard.put(key, vec.clone());
                embeddings[*idx] = Some(vec);
            }
        }
        Ok(EmbedResponse {
            embeddings: embeddings.into_iter().map(|opt| opt.expect("filled")).collect(),
            model: resp.model,
            dimensions: resp.dimensions,
        })
    }
    /// Direct Ollama call — used internally by embed() for cache-miss
    /// batches and as the transparent fallback when the cache is
    /// disabled. Loops per-text against `${base_url}/api/embed`,
    /// matching the sidecar's pre-2026-05-02 behavior. Ollama 0.4+
    /// supports batch input but per-text keeps compatibility broader
    /// + lets cache-miss-only batches share the loop with cold runs.
    async fn embed_uncached(&self, req: &EmbedRequest) -> Result<EmbedResponse, String> {
        let model = req.model.clone().unwrap_or_else(|| "nomic-embed-text".to_string());
        let mut embeddings: Vec<Vec<f64>> = Vec::with_capacity(req.texts.len());
        for text in &req.texts {
        let resp = self.client
-                .post(format!("{}/api/embed", self.base_url))
+            .post(format!("{}/embed", self.base_url))
-                .json(&serde_json::json!({
+            .json(&req)
                    "model": &model,
                    "input": text,
                }))
            .send()
            .await
            .map_err(|e| format!("embed request failed: {e}"))?;
        if !resp.status().is_success() {
-                let body = resp.text().await.unwrap_or_default();
+            let text = resp.text().await.unwrap_or_default();
-                return Err(format!("ollama embed error: {body}"));
+            return Err(format!("embed error ({}): {text}", text.len()));
        }
-            // Ollama returns {"embeddings": [[...]], "model": "...", ...}.
+        resp.json().await.map_err(|e| format!("embed parse error: {e}"))
            // The outer `embeddings` is always a list; for a scalar input
            // we get a single inner vector.
            let parsed: serde_json::Value = resp.json().await
                .map_err(|e| format!("embed parse error: {e}"))?;
            let arr = parsed.get("embeddings")
                .and_then(|v| v.as_array())
                .ok_or_else(|| format!("ollama embed: missing 'embeddings' field in {parsed}"))?;
            if arr.is_empty() {
                return Err("ollama embed: empty embeddings array".to_string());
            }
            let first = arr[0].as_array()
                .ok_or_else(|| "ollama embed: embeddings[0] not an array".to_string())?;
            let vec: Vec<f64> = first.iter()
                .filter_map(|n| n.as_f64())
                .collect();
            if vec.is_empty() {
                return Err("ollama embed: numeric coercion produced empty vector".to_string());
            }
            embeddings.push(vec);
        }
        let dimensions = embeddings.first().map(|v| v.len()).unwrap_or(0);
        Ok(EmbedResponse {
            embeddings,
            model,
            dimensions,
        })
    }
    pub async fn generate(&self, req: GenerateRequest) -> Result<GenerateResponse, String> {
        if let Some(gw) = self.gateway_url.as_deref() {
            return self.generate_via_gateway(gw, req).await;
        }
-        // Direct Ollama path. Used by gateway internals (so the ollama
+        // Direct-sidecar legacy path. Used by gateway internals (so
-        // provider can call upstream without a self-loop through
+        // ollama_arm can call sidecar without a self-loop) and by
-        // /v1/chat) and by any consumer that wants raw transport
+        // any consumer that wants raw transport without /v1/usage
-        // without /v1/usage accounting.
+        // accounting.
        let model = req.model.clone().unwrap_or_else(|| "qwen3.5:latest".to_string());
        let mut body = serde_json::json!({
            "model": &model,
            "prompt": &req.prompt,
            "stream": false,
        });
        let mut options = serde_json::Map::new();
        if let Some(t) = req.temperature {
            options.insert("temperature".to_string(), serde_json::json!(t));
        }
        if let Some(mt) = req.max_tokens {
            options.insert("num_predict".to_string(), serde_json::json!(mt));
        }
        if !options.is_empty() {
            body["options"] = serde_json::Value::Object(options);
        }
        if let Some(sys) = &req.system {
            body["system"] = serde_json::json!(sys);
        }
        if let Some(th) = req.think {
            body["think"] = serde_json::json!(th);
        }
        let resp = self.client
-            .post(format!("{}/api/generate", self.base_url))
+            .post(format!("{}/generate", self.base_url))
-            .json(&body)
+            .json(&req)
            .send()
            .await
            .map_err(|e| format!("generate request failed: {e}"))?;
        if !resp.status().is_success() {
            let text = resp.text().await.unwrap_or_default();
-            return Err(format!("ollama generate error: {text}"));
+            return Err(format!("generate error: {text}"));
        }
-        let parsed: serde_json::Value = resp.json().await
+        resp.json().await.map_err(|e| format!("generate parse error: {e}"))
            .map_err(|e| format!("generate parse error: {e}"))?;
        Ok(GenerateResponse {
            text: parsed.get("response").and_then(|v| v.as_str()).unwrap_or("").to_string(),
            model,
            tokens_evaluated: parsed.get("prompt_eval_count").and_then(|v| v.as_u64()),
            tokens_generated: parsed.get("eval_count").and_then(|v| v.as_u64()),
        })
    }
    /// Phase 44 part 2: route generate() through the gateway's
@@ -467,60 +217,19 @@ impl AiClient {
        })
    }
    /// Cross-encoder reranking via Ollama generate. Asks the model to
    /// rate each document's relevance to the query 0-10, then sorts
    /// descending. Mirrors the sidecar's pre-2026-05-02 algorithm
    /// exactly so callers see the same scores.
    pub async fn rerank(&self, req: RerankRequest) -> Result<RerankResponse, String> {
        let model = req.model.clone().unwrap_or_else(|| "qwen3.5:latest".to_string());
        let mut scored: Vec<ScoredDocument> = Vec::with_capacity(req.documents.len());
        for (i, doc) in req.documents.iter().enumerate() {
            let prompt = format!(
                "Rate the relevance of the following document to the query on a scale of 0 to 10. \
                 Respond with ONLY a number.\n\n\
                 Query: {}\n\n\
                 Document: {}\n\n\
                 Score:",
                req.query, doc,
            );
        let resp = self.client
-                .post(format!("{}/api/generate", self.base_url))
+            .post(format!("{}/rerank", self.base_url))
-                .json(&serde_json::json!({
+            .json(&req)
                    "model": &model,
                    "prompt": prompt,
                    "stream": false,
                    "options": {"temperature": 0.0, "num_predict": 8},
                }))
            .send()
            .await
            .map_err(|e| format!("rerank request failed: {e}"))?;
        if !resp.status().is_success() {
-                let body = resp.text().await.unwrap_or_default();
+            let text = resp.text().await.unwrap_or_default();
-                return Err(format!("ollama rerank error: {body}"));
+            return Err(format!("rerank error: {text}"));
        }
-            let parsed: serde_json::Value = resp.json().await
+        resp.json().await.map_err(|e| format!("rerank parse error: {e}"))
                .map_err(|e| format!("rerank parse error: {e}"))?;
            let text = parsed.get("response").and_then(|v| v.as_str()).unwrap_or("").trim();
            // Parse the leading number; tolerate "7", "7.5", "7 — strong match".
            let score = text.split_whitespace().next()
                .and_then(|t| t.parse::<f64>().ok())
                .unwrap_or(0.0)
                .clamp(0.0, 10.0);
            scored.push(ScoredDocument {
                index: i,
                text: doc.clone(),
                score,
            });
        }
        scored.sort_by(|a, b| b.score.partial_cmp(&a.score).unwrap_or(std::cmp::Ordering::Equal));
        if let Some(k) = req.top_k {
            scored.truncate(k);
        }
        Ok(RerankResponse { results: scored, model })
    }
    /// Force Ollama to unload the named model from VRAM (keep_alive=0).
@@ -529,116 +238,40 @@ impl AiClient {
    /// profile's model can linger in VRAM next to the new one.
    pub async fn unload_model(&self, model: &str) -> Result<serde_json::Value, String> {
        let resp = self.client
-            .post(format!("{}/api/generate", self.base_url))
+            .post(format!("{}/admin/unload", self.base_url))
-            .json(&serde_json::json!({
+            .json(&serde_json::json!({ "model": model }))
                "model": model,
                "prompt": "",
                "keep_alive": 0,
                "stream": false,
            }))
            .send().await
            .map_err(|e| format!("unload request failed: {e}"))?;
        if !resp.status().is_success() {
            let text = resp.text().await.unwrap_or_default();
-            return Err(format!("ollama unload error: {text}"));
+            return Err(format!("unload error: {text}"));
        }
-        // Ollama returns 200 with the empty-prompt response shape.
+        resp.json().await.map_err(|e| format!("unload parse error: {e}"))
        // Fold into the legacy {"unloaded": "<model>"} envelope so
        // callers' parsing doesn't break.
        Ok(serde_json::json!({ "unloaded": model }))
    }
    /// Ask Ollama to load the named model into VRAM proactively. Makes
    /// the first real request after profile activation fast (no cold-load
-    /// latency). Empty prompts confuse some models, so we send a single
+    /// latency).
    /// space + cap num_predict=1 (matches the sidecar's prior behavior).
    pub async fn preload_model(&self, model: &str) -> Result<serde_json::Value, String> {
        let resp = self.client
-            .post(format!("{}/api/generate", self.base_url))
+            .post(format!("{}/admin/preload", self.base_url))
-            .json(&serde_json::json!({
+            .json(&serde_json::json!({ "model": model }))
                "model": model,
                "prompt": " ",
                "keep_alive": "5m",
                "stream": false,
                "options": {"num_predict": 1},
            }))
            .send().await
            .map_err(|e| format!("preload request failed: {e}"))?;
        if !resp.status().is_success() {
            let text = resp.text().await.unwrap_or_default();
-            return Err(format!("ollama preload error: {text}"));
+            return Err(format!("preload error: {text}"));
        }
-        let parsed: serde_json::Value = resp.json().await
+        resp.json().await.map_err(|e| format!("preload parse error: {e}"))
            .map_err(|e| format!("preload parse error: {e}"))?;
        Ok(serde_json::json!({
            "preloaded": model,
            "load_duration_ns": parsed.get("load_duration"),
            "total_duration_ns": parsed.get("total_duration"),
        }))
    }
-    /// GPU + loaded-model snapshot. Combines nvidia-smi output (when
+    /// GPU + loaded-model snapshot from the sidecar. Combines nvidia-smi
-    /// available) with Ollama's /api/ps. Same shape as the prior
+    /// output (if available) with Ollama's /api/ps.
    /// sidecar /admin/vram endpoint so callers don't need updating.
    pub async fn vram_snapshot(&self) -> Result<serde_json::Value, String> {
        let resp = self.client
-            .get(format!("{}/api/ps", self.base_url))
+            .get(format!("{}/admin/vram", self.base_url))
            .send().await
-            .map_err(|e| format!("ollama ps request failed: {e}"))?;
+            .map_err(|e| format!("vram request failed: {e}"))?;
-        let loaded: Vec<serde_json::Value> = if resp.status().is_success() {
+        resp.json().await.map_err(|e| format!("vram parse error: {e}"))
            let parsed: serde_json::Value = resp.json().await.unwrap_or(serde_json::Value::Null);
            parsed.get("models")
                .and_then(|v| v.as_array())
                .map(|arr| arr.iter().map(|m| serde_json::json!({
                    "name": m.get("name"),
                    "size_vram_mib": m.get("size_vram").and_then(|v| v.as_u64()).map(|n| n / (1024 * 1024)),
                    "expires_at": m.get("expires_at"),
                })).collect())
                .unwrap_or_default()
        } else {
            Vec::new()
        };
        let gpu = nvidia_smi_snapshot();
        Ok(serde_json::json!({
            "gpu": gpu,
            "ollama_loaded": loaded,
        }))
    }
 }
 /// One-shot nvidia-smi poll. Returns Null if the tool isn't on PATH
 /// or the call fails. Mirrors the sidecar's `_nvidia_smi_snapshot`
 /// shape exactly so callers reading vram_snapshot don't break.
 fn nvidia_smi_snapshot() -> serde_json::Value {
    use std::process::Command;
    let out = Command::new("nvidia-smi")
        .args([
            "--query-gpu=memory.used,memory.total,utilization.gpu,name",
            "--format=csv,noheader,nounits",
        ])
        .output();
    let stdout = match out {
        Ok(o) if o.status.success() => o.stdout,
        _ => return serde_json::Value::Null,
    };
    let line = String::from_utf8_lossy(&stdout);
    let line = line.trim();
    if line.is_empty() {
        return serde_json::Value::Null;
    }
    let parts: Vec<&str> = line.split(',').map(|s| s.trim()).collect();
    if parts.len() < 4 {
        return serde_json::Value::Null;
    }
    let used = parts[0].parse::<u64>().unwrap_or(0);
    let total = parts[1].parse::<u64>().unwrap_or(0);
    let util = parts[2].parse::<u64>().unwrap_or(0);
    serde_json::json!({
        "name": parts[3],
        "used_mib": used,
        "total_mib": total,
        "utilization_pct": util,
    })
 }
@@ -1,37 +0,0 @@
 //! Cross-runtime parity helper for `extract_json`.
 //!
 //! Reads a single model-output string from stdin, runs the Rust
 //! extract_json, prints `{"matched": bool, "value": <object|null>}`
 //! to stdout as JSON. Exit 0 on success, exit 1 on internal error.
 //!
 //! The Go counterpart lives at
 //! `golangLAKEHOUSE/internal/validator/iterate.go::ExtractJSON`. The
 //! parity probe at
 //! `golangLAKEHOUSE/scripts/cutover/parity/extract_json_parity.sh`
 //! feeds the same fixtures through both and diffs the outputs.
 //!
 //! Usage:
 //!   echo '<raw model output>' | parity_extract_json
 //!   parity_extract_json <<< '...'
 use std::io::Read;
 fn main() {
    let mut buf = String::new();
    if let Err(e) = std::io::stdin().read_to_string(&mut buf) {
        eprintln!("read stdin: {e}");
        std::process::exit(1);
    }
    let result = gateway::v1::iterate::extract_json(&buf);
    let body = serde_json::json!({
        "matched": result.is_some(),
        "value": result.unwrap_or(serde_json::Value::Null),
    });
    match serde_json::to_string(&body) {
        Ok(s) => println!("{s}"),
        Err(e) => {
            eprintln!("serialize result: {e}");
            std::process::exit(1);
        }
    }
 }
@@ -1,71 +0,0 @@
 //! Cross-runtime parity helper for `SessionRecord` JSON shape.
 //!
 //! Reads a fixture JSON on stdin, builds a `SessionRecord`, emits
 //! one JSONL row on stdout. Used by
 //! `golangLAKEHOUSE/scripts/cutover/parity/session_log_parity.sh`
 //! to verify the Rust gateway's session log shape stays byte-equal
 //! to the Go-side validatord's `validator.SessionRecord` (commit
 //! 1a3a82a in golangLAKEHOUSE).
 use gateway::v1::session_log::{SessionAttemptRecord, SessionRecord, SESSION_RECORD_SCHEMA};
 use serde::Deserialize;
 use std::io::Read;
 #[derive(Deserialize)]
 struct FixtureInput {
    session_id: String,
    kind: String,
    model: String,
    provider: String,
    prompt: String,
    iterations: u32,
    max_iterations: u32,
    final_verdict: String,
    attempts: Vec<SessionAttemptRecord>,
    #[serde(default)]
    artifact: Option<serde_json::Value>,
    #[serde(default)]
    grounded_in_roster: Option<bool>,
    duration_ms: u64,
 }
 fn main() {
    let mut buf = String::new();
    if let Err(e) = std::io::stdin().read_to_string(&mut buf) {
        eprintln!("read stdin: {e}");
        std::process::exit(1);
    }
    let input: FixtureInput = match serde_json::from_str(&buf) {
        Ok(v) => v,
        Err(e) => {
            eprintln!("parse stdin: {e}");
            std::process::exit(1);
        }
    };
    let rec = SessionRecord {
        schema: SESSION_RECORD_SCHEMA.to_string(),
        session_id: input.session_id,
        // Pinned timestamp so both runtimes' rows compare byte-equal
        // when the test wrapper normalizes on `daemon` only.
        timestamp: "2026-01-01T00:00:00+00:00".to_string(),
        daemon: "gateway".to_string(),
        kind: input.kind,
        model: input.model,
        provider: input.provider,
        prompt: input.prompt,
        iterations: input.iterations,
        max_iterations: input.max_iterations,
        final_verdict: input.final_verdict,
        attempts: input.attempts,
        artifact: input.artifact,
        grounded_in_roster: input.grounded_in_roster,
        duration_ms: input.duration_ms,
    };
    match serde_json::to_string(&rec) {
        Ok(s) => println!("{s}"),
        Err(e) => {
            eprintln!("marshal: {e}");
            std::process::exit(1);
        }
    }
 }
@@ -438,10 +438,6 @@ impl ExecutionLoop {
                start_time: start_time.to_rfc3339(),
                end_time: end_time.to_rfc3339(),
                latency_ms: elapsed_ms,
                // Internal execution-loop traffic is its own top-level
                // trace per call. If a future caller threads a parent
                // trace into self.state, lift this to Some(parent_id).
                parent_trace_id: None,
            });
        }
@@ -586,10 +582,10 @@ impl ExecutionLoop {
    /// Phase 20 step (8) — T3 overseer escalation.
    ///
    /// When the local executor/reviewer loop can't self-correct, call
-    /// the cloud overseer (`claude-opus-4-7` via OpenCode Zen) with
+    /// the cloud overseer (`gpt-oss:120b` via Ollama Cloud) with (a)
-    /// (a) the KB context — recent outcomes + prior corrections for
+    /// the KB context — recent outcomes + prior corrections for this
-    /// this sig_hash + task_class, across every profile that has run
+    /// sig_hash + task_class, across every profile that has run it —
-    /// it — and (b) the recent log tail. Its output is appended as a
+    /// and (b) the recent log tail. Its output is appended as a
    /// `system` role turn so the next executor generation sees it,
    /// AND written to `data/_kb/overseer_corrections.jsonl` so every
    /// future profile activation reads from the same learning pool.
@@ -597,16 +593,9 @@ impl ExecutionLoop {
    /// This is the "pipe to the overviewer" piece from 2026-04-23 —
    /// the overseer is now a first-class KB consumer AND producer, not
    /// a one-shot correction oracle.
    ///
    /// 2026-04-28: routed through OpenCode (Zen tier) for Claude Opus
    /// 4.7. Frontier reasoning matters here because the overseer fires
    /// only after local self-correction has failed twice — by that
    /// point we need the strongest reasoning available, not the
    /// cheapest token. Frequency is low so the Zen pay-per-token cost
    /// stays bounded.
    async fn escalate_to_overseer(&mut self, turn: u32, reason: &str) -> Result<(), String> {
-        let Some(opencode_key) = self.state.opencode_key.clone() else {
+        let Some(cloud_key) = self.state.ollama_cloud_key.clone() else {
-            return Err("OPENCODE_API_KEY not configured — skipping escalation".into());
+            return Err("OLLAMA_CLOUD_KEY not configured — skipping escalation".into());
        };
        let kb = KbContext::load_for(&sig_hash(&self.req), &self.req.task_class).await;
@@ -615,18 +604,16 @@ impl ExecutionLoop {
        let started = std::time::Instant::now();
        let start_time = chrono::Utc::now();
        let chat_req = crate::v1::ChatRequest {
-            model: "claude-opus-4-7".to_string(),
+            model: "gpt-oss:120b".to_string(),
            messages: vec![crate::v1::Message::new_text("user", prompt.clone())],
            temperature: Some(0.1),
            max_tokens: None,
            stream: Some(false),
-            // Anthropic models on opencode reject `think` (handled in
+            think: Some(true),    // overseer KEEPS thinking (Phase 20 rule)
-            // the adapter), but we keep the intent flag for parity.
+            provider: Some("ollama_cloud".into()),
            think: Some(true),
            provider: Some("opencode".into()),
        };
-        let resp = crate::v1::opencode::chat(&opencode_key, &chat_req).await
+        let resp = crate::v1::ollama_cloud::chat(&cloud_key, &chat_req).await
-            .map_err(|e| format!("opencode: {e}"))?;
+            .map_err(|e| format!("ollama_cloud: {e}"))?;
        let latency_ms = started.elapsed().as_millis() as u64;
        let end_time = chrono::Utc::now();
        let correction_text: String = resp.choices.into_iter().next()
@@ -646,8 +633,8 @@ impl ExecutionLoop {
        if let Some(lf) = &self.state.langfuse {
            use crate::v1::langfuse_trace::ChatTrace;
            lf.emit_chat(ChatTrace {
-                provider: "opencode".into(),
+                provider: "ollama_cloud".into(),
-                model: "claude-opus-4-7".into(),
+                model: "gpt-oss:120b".into(),
                input: vec![crate::v1::Message::new_text("user", prompt.clone())],
                output: correction_text.clone(),
                prompt_tokens: resp.usage.prompt_tokens,
@@ -658,13 +645,12 @@ impl ExecutionLoop {
                start_time: start_time.to_rfc3339(),
                end_time: end_time.to_rfc3339(),
                latency_ms,
                parent_trace_id: None,
            });
        }
        // Append to the transcript so the next executor turn sees it.
        self.append(LogEntry::new(
-            turn, "system", "claude-opus-4-7", "overseer_correction",
+            turn, "system", "gpt-oss:120b", "overseer_correction",
            serde_json::json!({
                "reason": reason,
                "correction": correction_text,
@@ -686,7 +672,7 @@ impl ExecutionLoop {
            "task_class": self.req.task_class,
            "operation": self.req.operation,
            "reason": reason,
-            "model": "claude-opus-4-7",
+            "model": "gpt-oss:120b",
            "correction": correction_text,
            "applied_at_turn": turn,
            "kb_context_used": kb,
@@ -1,19 +0,0 @@
 //! Library facade for the gateway crate so sub-binaries (e.g.
 //! `parity_extract_json`) can reuse the same modules the gateway
 //! binary uses.
 //!
 //! Added 2026-05-02 to support the cross-runtime parity probe at
 //! `golangLAKEHOUSE/scripts/cutover/parity/extract_json_parity.sh`.
 //! `extract_json` is the load-bearing public surface for that probe.
 //!
 //! main.rs still uses local `mod foo;` declarations independently —
 //! adding this file is purely additive (the binary's module tree is
 //! unchanged).
 pub mod access;
 pub mod access_service;
 pub mod auth;
 pub mod execution_loop;
 pub mod observability;
 pub mod tools;
 pub mod v1;
@@ -362,22 +362,6 @@ async fn main() {
                }
                c
            },
            // Coordinator session JSONL — one row per /v1/iterate
            // session for offline DuckDB analysis. Cross-runtime
            // parity with Go-side validatord (commit 1a3a82a).
            session_log: {
                let path = &config.gateway.session_log_path;
                let s = v1::session_log::SessionLogger::from_path(path);
                if s.is_some() {
                    tracing::info!(
                        "v1: session log enabled — coordinator sessions written to {}",
                        path
                    );
                } else {
                    tracing::info!("v1: session log disabled (set [gateway].session_log_path to enable)");
                }
                s
            },
        }));
    // Auth middleware (if enabled) — P5-001 fix 2026-04-23:
@@ -21,19 +21,12 @@
 //! re-implementation. Staffing executors, agent loops, and future
 //! validators all reach the same code path.
-use axum::{extract::State, http::{HeaderMap, StatusCode}, response::IntoResponse, Json};
+use axum::{extract::State, http::StatusCode, response::IntoResponse, Json};
 use serde::{Deserialize, Serialize};
 const DEFAULT_MAX_ITERATIONS: u32 = 3;
 const LOOPBACK_TIMEOUT_SECS: u64 = 240;
 /// Header name used to propagate a Langfuse parent trace id across
 /// daemon boundaries. Matches Go's `shared.TraceIDHeader` constant
 /// byte-for-byte (commit d6d2fdf in golangLAKEHOUSE) — same wire
 /// format means a Go caller can hit Rust's /v1/iterate (or vice
 /// versa) and the resulting Langfuse trees nest correctly.
 pub const TRACE_ID_HEADER: &str = "x-lakehouse-trace-id";
 #[derive(Deserialize)]
 pub struct IterateRequest {
    /// "fill" | "email" | "playbook" — picks which validator runs.
@@ -87,14 +80,6 @@ pub struct IterateResponse {
    pub validation: serde_json::Value,
    pub iterations: u32,
    pub history: Vec<IterateAttempt>,
    /// Echoes the resolved trace id (caller-forwarded header, body
    /// field, langfuse-middleware mint, or local fallback). Operators
    /// pivot from this id straight into Langfuse + the
    /// coordinator_sessions.jsonl join key. Cross-runtime parity with
    /// Go's `validator.IterateResponse` (commit 6847bbc in
    /// golangLAKEHOUSE).
    #[serde(skip_serializing_if = "Option::is_none")]
    pub trace_id: Option<String>,
 }
 #[derive(Serialize)]
@@ -102,52 +87,29 @@ pub struct IterateFailure {
    pub error: String,
    pub iterations: u32,
    pub history: Vec<IterateAttempt>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub trace_id: Option<String>,
 }
 pub async fn iterate(
-    State(state): State<super::V1State>,
+    State(_state): State<super::V1State>,
    headers: HeaderMap,
    Json(req): Json<IterateRequest>,
 ) -> impl IntoResponse {
    let max_iter = req.max_iterations.unwrap_or(DEFAULT_MAX_ITERATIONS).max(1);
    let temperature = req.temperature.unwrap_or(0.2);
    let max_tokens = req.max_tokens.unwrap_or(4096);
    let mut history: Vec<IterateAttempt> = Vec::with_capacity(max_iter as usize);
    let mut attempt_records: Vec<super::session_log::SessionAttemptRecord> = Vec::with_capacity(max_iter as usize);
    let mut current_prompt = req.prompt.clone();
    // Resolve the parent Langfuse trace id. Caller-forwarded header
    // wins (cross-daemon tree linkage); otherwise mint a fresh id so
    // the iterate session is its own tree. Same shape as the Go-side
    // validatord trace propagation.
    let trace_id: String = headers
        .get(TRACE_ID_HEADER)
        .and_then(|v| v.to_str().ok())
        .filter(|s| !s.is_empty())
        .map(|s| s.to_string())
        .unwrap_or_else(new_trace_id);
    let client = match reqwest::Client::builder()
        .timeout(std::time::Duration::from_secs(LOOPBACK_TIMEOUT_SECS))
        .build() {
        Ok(c) => c,
-        Err(e) => {
+        Err(e) => return (StatusCode::INTERNAL_SERVER_ERROR, format!("client build: {e}")).into_response(),
            // Even infrastructure failures get a session row so a
            // missing /v1/iterate event never silently disappears
            // from the longitudinal log.
            write_infra_error(&state, &req, &trace_id, max_iter, 0, format!("client build: {e}")).await;
            return (StatusCode::INTERNAL_SERVER_ERROR, format!("client build: {e}")).into_response();
        }
    };
    // Self-loopback to the gateway port. Carries gateway internal
    // calls through /v1/chat + /v1/validate so /v1/usage tracks them.
    let gateway = "http://127.0.0.1:3100";
    let t0 = std::time::Instant::now();
    for iteration in 0..max_iter {
        let attempt_started = chrono::Utc::now();
        // ── Generate ──
        let mut messages = Vec::with_capacity(2);
        if let Some(sys) = &req.system {
@@ -161,33 +123,20 @@ pub async fn iterate(
            "temperature": temperature,
            "max_tokens": max_tokens,
        });
-        let raw = match call_chat(&client, gateway, &chat_body, &trace_id).await {
+        let raw = match call_chat(&client, gateway, &chat_body).await {
            Ok(r) => r,
-            Err(e) => {
+            Err(e) => return (StatusCode::BAD_GATEWAY, format!("/v1/chat hop failed at iter {iteration}: {e}")).into_response(),
                write_infra_error(&state, &req, &trace_id, max_iter, t0.elapsed().as_millis() as u64, format!("/v1/chat hop failed at iter {iteration}: {e}")).await;
                return (StatusCode::BAD_GATEWAY, format!("/v1/chat hop failed at iter {iteration}: {e}")).into_response();
            }
        };
        // ── Extract JSON ──
        let artifact = match extract_json(&raw) {
            Some(a) => a,
            None => {
                let span_id = emit_attempt_span(
                    &state, &trace_id, iteration, &req, &current_prompt, &raw, "no_json", None,
                    attempt_started, chrono::Utc::now(),
                );
                history.push(IterateAttempt {
                    iteration,
                    raw: raw.chars().take(2000).collect(),
                    status: AttemptStatus::NoJson,
                });
                attempt_records.push(super::session_log::SessionAttemptRecord {
                    iteration,
                    verdict_kind: "no_json".to_string(),
                    error: None,
                    span_id,
                });
                current_prompt = format!(
                    "{}\n\nYour previous attempt did not contain a JSON object. Reply with ONLY a valid JSON object matching the requested artifact shape.",
                    req.prompt,
@@ -202,41 +151,22 @@ pub async fn iterate(
            "artifact": artifact,
            "context": req.context.clone().unwrap_or(serde_json::Value::Null),
        });
-        match call_validate(&client, gateway, &validate_body, &trace_id).await {
+        match call_validate(&client, gateway, &validate_body).await {
            Ok(report) => {
                let span_id = emit_attempt_span(
                    &state, &trace_id, iteration, &req, &current_prompt, &raw, "accepted", None,
                    attempt_started, chrono::Utc::now(),
                );
                history.push(IterateAttempt {
                    iteration,
                    raw: raw.chars().take(2000).collect(),
                    status: AttemptStatus::Accepted,
                });
                attempt_records.push(super::session_log::SessionAttemptRecord {
                    iteration,
                    verdict_kind: "accepted".to_string(),
                    error: None,
                    span_id,
                });
                let duration_ms = t0.elapsed().as_millis() as u64;
                let grounded = grounded_in_roster(&state, &req.kind, &artifact);
                write_session_accepted(&state, &req, &trace_id, iteration + 1, max_iter, attempt_records, &artifact, grounded, duration_ms).await;
                return (StatusCode::OK, Json(IterateResponse {
                    artifact,
                    validation: report,
                    iterations: iteration + 1,
                    history,
                    trace_id: Some(trace_id.clone()),
                })).into_response();
            }
            Err(err) => {
                let err_summary = err.to_string();
                let span_id = emit_attempt_span(
                    &state, &trace_id, iteration, &req, &current_prompt, &raw, "validation_failed",
                    Some(err_summary.clone()),
                    attempt_started, chrono::Utc::now(),
                );
                history.push(IterateAttempt {
                    iteration,
                    raw: raw.chars().take(2000).collect(),
@@ -244,12 +174,6 @@ pub async fn iterate(
                        error: serde_json::to_value(&err_summary).unwrap_or(serde_json::Value::Null),
                    },
                });
                attempt_records.push(super::session_log::SessionAttemptRecord {
                    iteration,
                    verdict_kind: "validation_failed".to_string(),
                    error: Some(err_summary.clone()),
                    span_id,
                });
                // Append validation feedback to prompt for next iter.
                // The model sees concrete failure mode + retries with
                // corrective context. This is the "observer correction"
@@ -264,167 +188,19 @@ pub async fn iterate(
        }
    }
    let duration_ms = t0.elapsed().as_millis() as u64;
    write_session_failure(&state, &req, &trace_id, max_iter, max_iter, attempt_records, duration_ms).await;
    (StatusCode::UNPROCESSABLE_ENTITY, Json(IterateFailure {
        error: format!("max iterations reached ({max_iter}) without passing validation"),
        iterations: max_iter,
        history,
        trace_id: Some(trace_id.clone()),
    })).into_response()
 }
-// ─── Helpers — Langfuse spans + session log + roster check ─────────
+async fn call_chat(client: &reqwest::Client, gateway: &str, body: &serde_json::Value) -> Result<String, String> {
-
+    let resp = client.post(format!("{gateway}/v1/chat"))
-fn emit_attempt_span(
+        .json(body)
-    state: &super::V1State,
+        .send()
-    trace_id: &str,
+        .await
-    iteration: u32,
+        .map_err(|e| format!("chat hop: {e}"))?;
    req: &IterateRequest,
    prompt: &str,
    raw: &str,
    verdict: &str,
    error: Option<String>,
    started: chrono::DateTime<chrono::Utc>,
    ended: chrono::DateTime<chrono::Utc>,
 ) -> Option<String> {
    let lf = state.langfuse.as_ref()?;
    Some(lf.emit_attempt_span(super::langfuse_trace::AttemptSpan {
        trace_id: trace_id.to_string(),
        iteration,
        model: req.model.clone(),
        provider: req.provider.clone(),
        prompt: prompt.to_string(),
        raw: raw.to_string(),
        verdict: verdict.to_string(),
        error,
        start_time: started.to_rfc3339(),
        end_time: ended.to_rfc3339(),
    }))
 }
 /// Verify every fill artifact's candidate IDs exist in the roster.
 /// Returns Some(true)/Some(false) on the fill kind, None otherwise
 /// (other kinds don't have worker IDs to ground). Same semantics as
 /// Go's `handlers.rosterCheckFor("fill")`.
 fn grounded_in_roster(
    state: &super::V1State,
    kind: &str,
    artifact: &serde_json::Value,
 ) -> Option<bool> {
    if kind != "fill" {
        return None;
    }
    let fills = artifact.get("fills").and_then(|v| v.as_array())?;
    for f in fills {
        let id = match f.get("candidate_id").and_then(|v| v.as_str()) {
            Some(s) if !s.is_empty() => s,
            _ => return Some(false),
        };
        if state.validate_workers.find(id).is_none() {
            return Some(false);
        }
    }
    Some(true)
 }
 async fn write_session_accepted(
    state: &super::V1State,
    req: &IterateRequest,
    trace_id: &str,
    iterations: u32,
    max_iter: u32,
    attempts: Vec<super::session_log::SessionAttemptRecord>,
    artifact: &serde_json::Value,
    grounded: Option<bool>,
    duration_ms: u64,
 ) {
    let Some(logger) = state.session_log.as_ref() else { return };
    let rec = build_session_record(req, trace_id, "accepted", iterations, max_iter, attempts, Some(artifact.clone()), grounded, duration_ms);
    logger.append(rec).await;
 }
 async fn write_session_failure(
    state: &super::V1State,
    req: &IterateRequest,
    trace_id: &str,
    iterations: u32,
    max_iter: u32,
    attempts: Vec<super::session_log::SessionAttemptRecord>,
    duration_ms: u64,
 ) {
    let Some(logger) = state.session_log.as_ref() else { return };
    let rec = build_session_record(req, trace_id, "max_iter_exhausted", iterations, max_iter, attempts, None, None, duration_ms);
    logger.append(rec).await;
 }
 async fn write_infra_error(
    state: &super::V1State,
    req: &IterateRequest,
    trace_id: &str,
    max_iter: u32,
    duration_ms: u64,
    error: String,
 ) {
    let Some(logger) = state.session_log.as_ref() else { return };
    let attempts = vec![super::session_log::SessionAttemptRecord {
        iteration: 0,
        verdict_kind: "infra_error".to_string(),
        error: Some(error),
        span_id: None,
    }];
    let rec = build_session_record(req, trace_id, "infra_error", 0, max_iter, attempts, None, None, duration_ms);
    logger.append(rec).await;
 }
 fn build_session_record(
    req: &IterateRequest,
    trace_id: &str,
    final_verdict: &str,
    iterations: u32,
    max_iter: u32,
    attempts: Vec<super::session_log::SessionAttemptRecord>,
    artifact: Option<serde_json::Value>,
    grounded: Option<bool>,
    duration_ms: u64,
 ) -> super::session_log::SessionRecord {
    super::session_log::SessionRecord {
        schema: super::session_log::SESSION_RECORD_SCHEMA.to_string(),
        session_id: trace_id.to_string(),
        timestamp: chrono::Utc::now().to_rfc3339(),
        daemon: "gateway".to_string(),
        kind: req.kind.clone(),
        model: req.model.clone(),
        provider: req.provider.clone(),
        prompt: super::session_log::truncate(&req.prompt, 4000),
        iterations,
        max_iterations: max_iter,
        final_verdict: final_verdict.to_string(),
        attempts,
        artifact,
        grounded_in_roster: grounded,
        duration_ms,
    }
 }
 /// Generate a fresh trace id when no parent was forwarded. Same
 /// time-ordered hex shape Langfuse already accepts elsewhere in this
 /// crate (see `langfuse_trace::uuid_v7_like`).
 fn new_trace_id() -> String {
    let ts = chrono::Utc::now().timestamp_nanos_opt().unwrap_or(0);
    let rand = std::time::SystemTime::now()
        .duration_since(std::time::UNIX_EPOCH)
        .map(|d| d.subsec_nanos())
        .unwrap_or(0);
    format!("{:016x}-{:08x}", ts, rand)
 }
 async fn call_chat(client: &reqwest::Client, gateway: &str, body: &serde_json::Value, trace_id: &str) -> Result<String, String> {
    let mut req = client.post(format!("{gateway}/v1/chat")).json(body);
    if !trace_id.is_empty() {
        req = req.header(TRACE_ID_HEADER, trace_id);
    }
    let resp = req.send().await.map_err(|e| format!("chat hop: {e}"))?;
    let status = resp.status();
    if !status.is_success() {
        let body = resp.text().await.unwrap_or_default();
@@ -437,12 +213,12 @@ async fn call_chat(client: &reqwest::Client, gateway: &str, body: &serde_json::V
        .to_string())
 }
-async fn call_validate(client: &reqwest::Client, gateway: &str, body: &serde_json::Value, trace_id: &str) -> Result<serde_json::Value, String> {
+async fn call_validate(client: &reqwest::Client, gateway: &str, body: &serde_json::Value) -> Result<serde_json::Value, String> {
-    let mut req = client.post(format!("{gateway}/v1/validate")).json(body);
+    let resp = client.post(format!("{gateway}/v1/validate"))
-    if !trace_id.is_empty() {
+        .json(body)
-        req = req.header(TRACE_ID_HEADER, trace_id);
+        .send()
-    }
+        .await
-    let resp = req.send().await.map_err(|e| format!("validate hop: {e}"))?;
+        .map_err(|e| format!("validate hop: {e}"))?;
    let status = resp.status();
    let parsed: serde_json::Value = resp.json().await.map_err(|e| format!("validate parse: {e}"))?;
    if status.is_success() {
@@ -458,13 +234,7 @@ async fn call_validate(client: &reqwest::Client, gateway: &str, body: &serde_jso
 /// Extract the first JSON object from a model's output. Handles
 /// fenced code blocks (```json ... ```), bare braces, and stray
 /// prose around the JSON. Returns None on no extractable object.
-///
+fn extract_json(raw: &str) -> Option<serde_json::Value> {
 /// Made `pub` 2026-05-02 to support the cross-runtime parity probe
 /// at `golangLAKEHOUSE/scripts/cutover/parity/extract_json_parity.sh`.
 /// The Go counterpart lives at `internal/validator/iterate.go::ExtractJSON`;
 /// when either runtime's algorithm changes the parity probe surfaces
 /// the divergence.
 pub fn extract_json(raw: &str) -> Option<serde_json::Value> {
    // Try fenced first.
    let candidates: Vec<String> = {
        let mut out = vec![];
@@ -76,85 +76,14 @@ impl LangfuseClient {
        });
    }
    /// Fire-and-forget per-iteration span emit. Returns the generated
    /// span id synchronously so the caller can stamp it on
    /// `IterateAttempt.span_id` before the network round-trip resolves.
    /// Mirrors Go's `validator.Tracer` callback shape.
    pub fn emit_attempt_span(&self, sp: AttemptSpan) -> String {
        let span_id = uuid_v7_like();
        let span_id_for_caller = span_id.clone();
        let this = self.clone();
        tokio::spawn(async move {
            if let Err(e) = this.emit_attempt_span_inner(span_id, sp).await {
                tracing::warn!(target: "v1.langfuse", "iterate span drop: {e}");
            }
        });
        span_id_for_caller
    }
    async fn emit_attempt_span_inner(&self, span_id: String, sp: AttemptSpan) -> Result<(), String> {
        let level = if sp.verdict == "accepted" { "DEFAULT" } else { "WARNING" };
        let batch = IngestionBatch {
            batch: vec![IngestionEvent {
                id: uuid_v7_like(),
                timestamp: sp.end_time.clone(),
                kind: "span-create",
                body: serde_json::json!({
                    "id": span_id,
                    "traceId": sp.trace_id,
                    "name": format!("iterate.attempt[{}]", sp.iteration),
                    "input": serde_json::json!({
                        "iteration": sp.iteration,
                        "model": sp.model,
                        "provider": sp.provider,
                        "prompt": truncate(&sp.prompt, 4000),
                    }),
                    "output": serde_json::json!({
                        "verdict": sp.verdict,
                        "error": sp.error,
                        "raw": truncate(&sp.raw, 4000),
                    }),
                    "level": level,
                    "startTime": sp.start_time,
                    "endTime": sp.end_time,
                }),
            }],
        };
        self.post_batch(batch).await
    }
    async fn post_batch(&self, batch: IngestionBatch) -> Result<(), String> {
        let url = format!("{}{}", self.inner.base_url.trim_end_matches('/'), INGESTION_PATH);
        let resp = self.inner.http
            .post(url)
            .basic_auth(&self.inner.public_key, Some(&self.inner.secret_key))
            .json(&batch)
            .send()
            .await
            .map_err(|e| format!("POST failed: {e}"))?;
        if !resp.status().is_success() {
            return Err(format!("{}: {}", resp.status(), resp.text().await.unwrap_or_default()));
        }
        Ok(())
    }
    async fn emit_chat_inner(&self, ev: ChatTrace) -> Result<(), String> {
-        // When the caller forwarded a parent trace id (via the
+        let trace_id = uuid_v7_like();
        // X-Lakehouse-Trace-Id header → V1State plumbing), attach the
        // generation as a child of that trace. Without a parent we
        // mint a new top-level trace per call (Phase 40 default).
        let trace_id = ev.parent_trace_id.clone().unwrap_or_else(uuid_v7_like);
        let nested = ev.parent_trace_id.is_some();
        let gen_id = uuid_v7_like();
        let trace_ts = ev.start_time.clone();
-        let mut events = Vec::with_capacity(2);
+        let batch = IngestionBatch {
-        if !nested {
+            batch: vec![
-            // Only mint a fresh trace-create when we don't have a parent.
+                IngestionEvent {
            // Reusing a parent trace id without re-creating it is the
            // contract that lets validatord's iterate-session show up
            // as one tree in Langfuse.
            events.push(IngestionEvent {
                    id: uuid_v7_like(),
                    timestamp: trace_ts.clone(),
                    kind: "trace-create",
@@ -170,9 +99,8 @@ impl LangfuseClient {
                            "think": ev.think,
                        }),
                    }),
-            });
+                },
-        }
+                IngestionEvent {
        events.push(IngestionEvent {
                    id: uuid_v7_like(),
                    timestamp: ev.end_time.clone(),
                    kind: "generation-create",
@@ -201,17 +129,23 @@ impl LangfuseClient {
                            "latency_ms": ev.latency_ms,
                        }),
                    }),
-        });
+                },
            ],
        };
-        self.post_batch(IngestionBatch { batch: events }).await
+        let url = format!("{}{}", self.inner.base_url.trim_end_matches('/'), INGESTION_PATH);
        let resp = self.inner.http
            .post(url)
            .basic_auth(&self.inner.public_key, Some(&self.inner.secret_key))
            .json(&batch)
            .send()
            .await
            .map_err(|e| format!("POST failed: {e}"))?;
        if !resp.status().is_success() {
            return Err(format!("{}: {}", resp.status(), resp.text().await.unwrap_or_default()));
        }
        Ok(())
    }
 }
 /// Truncate a string to at most `n` chars (NOT bytes). Matches the Go
 /// `trim` helper used in session log + attempt-span emission so an
 /// operator reading two cross-runtime traces sees the same boundary.
 fn truncate(s: &str, n: usize) -> String {
    s.chars().take(n).collect()
 }
 /// Everything the v1.chat handler collects for one completed call.
@@ -228,32 +162,6 @@ pub struct ChatTrace {
    pub start_time: String,
    pub end_time: String,
    pub latency_ms: u64,
    /// When set, attach this chat trace as a child of the named
    /// Langfuse trace instead of starting a new top-level trace. Used
    /// by `/v1/iterate` to nest its inner /v1/chat hops under the
    /// iterate-session trace so a multi-call session shows in
    /// Langfuse as ONE trace tree, not N+1 disconnected traces.
    /// Matches the Go-side `X-Lakehouse-Trace-Id` propagation
    /// (commit d6d2fdf in golangLAKEHOUSE).
    pub parent_trace_id: Option<String>,
 }
 /// One iteration attempt inside `/v1/iterate`'s loop. Becomes one
 /// span on the parent trace when emitted via `emit_attempt_span`.
 /// Matches Go's `validator.AttemptSpan` shape so the cross-runtime
 /// observability surface is consistent.
 pub struct AttemptSpan {
    pub trace_id: String,
    pub iteration: u32,
    pub model: String,
    pub provider: String,
    pub prompt: String,
    pub raw: String,
    /// Verdict kind: "no_json" | "validation_failed" | "accepted"
    pub verdict: String,
    pub error: Option<String>,
    pub start_time: String,
    pub end_time: String,
 }
 #[derive(Serialize)]
@@ -21,7 +21,6 @@ pub mod opencode;
 pub mod validate;
 pub mod iterate;
 pub mod langfuse_trace;
 pub mod session_log;
 pub mod mode;
 pub mod respond;
 pub mod truth;
@@ -84,13 +83,6 @@ pub struct V1State {
    /// disabled (keys missing or container unreachable). Traces are
    /// fire-and-forget: never block the response path.
    pub langfuse: Option<langfuse_trace::LangfuseClient>,
    /// Coordinator session JSONL writer (path from
    /// `[gateway].session_log_path`). One row per `/v1/iterate`
    /// session for offline DuckDB analysis. None = disabled.
    /// Cross-runtime parity with the Go-side `validatord`
    /// `[validatord].session_log_path` (commit 1a3a82a in
    /// golangLAKEHOUSE).
    pub session_log: Option<session_log::SessionLogger>,
 }
 #[derive(Default, Clone, Serialize)]
@@ -369,7 +361,6 @@ mod resolve_provider_tests {
 async fn chat(
    State(state): State<V1State>,
    headers: axum::http::HeaderMap,
    Json(req): Json<ChatRequest>,
 ) -> Result<Json<ChatResponse>, (StatusCode, String)> {
    if req.messages.is_empty() {
@@ -499,17 +490,6 @@ async fn chat(
        let output = resp.choices.first()
            .map(|c| c.message.text())
            .unwrap_or_default();
        // Cross-runtime trace linkage. When a caller (validatord on
        // Go side, /v1/iterate on Rust side) forwards a parent trace
        // id via X-Lakehouse-Trace-Id, attach this generation to that
        // trace so the iterate session and its inner chat hops show
        // up as ONE trace tree in Langfuse. Header name matches the
        // Go-side `shared.TraceIDHeader` constant byte-for-byte.
        let parent_trace_id = headers
            .get(crate::v1::iterate::TRACE_ID_HEADER)
            .and_then(|v| v.to_str().ok())
            .map(|s| s.to_string())
            .filter(|s| !s.is_empty());
        lf.emit_chat(langfuse_trace::ChatTrace {
            provider: used_provider.clone(),
            model: resp.model.clone(),
@@ -523,7 +503,6 @@ async fn chat(
            start_time: start_time.to_rfc3339(),
            end_time: end_time.to_rfc3339(),
            latency_ms,
            parent_trace_id,
        });
    }
@@ -1,235 +0,0 @@
 //! Coordinator session JSONL writer — Rust parity with the Go-side
 //! `internal/validator/session_log.go` (commit 1a3a82a in
 //! golangLAKEHOUSE). Same schema, same field names, same producer
 //! semantics, so a unified longitudinal log can pull from either
 //! runtime via DuckDB.
 //!
 //! Schema: `session.iterate.v1`. One row per `/v1/iterate` session.
 //! Append-only. Best-effort posture: errors warn and the iterate
 //! response always ships.
 //!
 //! See `golangLAKEHOUSE/docs/SESSION_LOG.md` for the full schema
 //! reference + DuckDB query examples. This module produces rows
 //! with `daemon: "gateway"`; the Go side produces `daemon:
 //! "validatord"`. Operators who want a unified stream can point both
 //! to the same path (the OS write-append is atomic for the row sizes
 //! we produce) or query both files together via duckdb's `read_json`
 //! glob support.
 use serde::{Deserialize, Serialize};
 use std::sync::Arc;
 use tokio::sync::Mutex;
 pub const SESSION_RECORD_SCHEMA: &str = "session.iterate.v1";
 /// One row in coordinator_sessions.jsonl. Field names are the on-wire
 /// names — must stay byte-equal to the Go side's
 /// `validator.SessionRecord` (proven by the cross-runtime parity
 /// probe at golangLAKEHOUSE/scripts/cutover/parity/).
 // Deserialize is supported so the parity helper binary can round-trip
 // fixture inputs through serde without hand-rolling a parser. Production
 // emit path uses Serialize only; SessionRecord rows are written by the
 // gateway and consumed by DuckDB / external tooling, never re-read by us.
 #[derive(Serialize, Deserialize)]
 pub struct SessionRecord {
    pub schema: String,
    pub session_id: String,
    pub timestamp: String,
    pub daemon: String,
    pub kind: String,
    pub model: String,
    pub provider: String,
    pub prompt: String,
    pub iterations: u32,
    pub max_iterations: u32,
    pub final_verdict: String, // "accepted" | "max_iter_exhausted" | "infra_error"
    pub attempts: Vec<SessionAttemptRecord>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub artifact: Option<serde_json::Value>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub grounded_in_roster: Option<bool>,
    pub duration_ms: u64,
 }
 #[derive(Serialize, Deserialize)]
 pub struct SessionAttemptRecord {
    pub iteration: u32,
    pub verdict_kind: String, // "no_json" | "validation_failed" | "accepted" | "infra_error"
    #[serde(skip_serializing_if = "Option::is_none")]
    pub error: Option<String>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub span_id: Option<String>,
 }
 /// Append-only writer. Cloneable handle — internal state is Arc'd so
 /// V1State can keep its own clone and per-request clones are cheap.
 #[derive(Clone)]
 pub struct SessionLogger {
    inner: Arc<Inner>,
 }
 struct Inner {
    path: String,
    /// tokio::Mutex (not std) because we hold it across the async
    /// fs write. Contention is low (one row per /v1/iterate session).
    mu: Mutex<()>,
 }
 impl SessionLogger {
    /// Construct a logger writing to `path`. Empty path → None
    /// (skip the wiring in the iterate handler entirely).
    pub fn from_path(path: &str) -> Option<Self> {
        if path.is_empty() {
            return None;
        }
        Some(Self {
            inner: Arc::new(Inner {
                path: path.to_string(),
                mu: Mutex::new(()),
            }),
        })
    }
    /// Append one record. Best-effort: failures land in `tracing::warn!`
    /// and the caller sees Ok(()) — observability is a witness, never
    /// a gate. Returns Err only on impossible cases the type system
    /// can't rule out (here: serde_json::to_string failing on a
    /// well-formed struct, which shouldn't happen).
    pub async fn append(&self, rec: SessionRecord) {
        let body = match serde_json::to_string(&rec) {
            Ok(s) => s,
            Err(e) => {
                tracing::warn!(target: "v1.session_log", "marshal: {e}");
                return;
            }
        };
        let _guard = self.inner.mu.lock().await;
        if let Err(e) = self.write(&body).await {
            tracing::warn!(target: "v1.session_log", "write {}: {e}", self.inner.path);
        }
    }
    async fn write(&self, body: &str) -> std::io::Result<()> {
        use tokio::fs::OpenOptions;
        use tokio::io::AsyncWriteExt;
        // Lazy mkdir on first write so a not-yet-mounted volume at
        // startup doesn't kill the daemon.
        if let Some(parent) = std::path::Path::new(&self.inner.path).parent() {
            if !parent.as_os_str().is_empty() {
                tokio::fs::create_dir_all(parent).await?;
            }
        }
        let mut f = OpenOptions::new()
            .append(true)
            .create(true)
            .open(&self.inner.path)
            .await?;
        f.write_all(body.as_bytes()).await?;
        f.write_all(b"\n").await?;
        Ok(())
    }
 }
 /// Best-effort UTF-8 char truncation. Matches Go's `trim` helper so
 /// rows produced by either runtime cap fields at the same boundaries.
 pub fn truncate(s: &str, n: usize) -> String {
    s.chars().take(n).collect()
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use std::path::PathBuf;
    use tokio::fs;
    fn fixture_record(session_id: &str) -> SessionRecord {
        SessionRecord {
            schema: SESSION_RECORD_SCHEMA.to_string(),
            session_id: session_id.to_string(),
            timestamp: "2026-05-02T08:00:00Z".to_string(),
            daemon: "gateway".to_string(),
            kind: "fill".to_string(),
            model: "qwen3.5:latest".to_string(),
            provider: "ollama".to_string(),
            prompt: "produce a fill artifact".to_string(),
            iterations: 1,
            max_iterations: 3,
            final_verdict: "accepted".to_string(),
            attempts: vec![SessionAttemptRecord {
                iteration: 0,
                verdict_kind: "accepted".to_string(),
                error: None,
                span_id: Some("span-0".to_string()),
            }],
            artifact: Some(serde_json::json!({"fills":[{"candidate_id":"W-1"}]})),
            grounded_in_roster: Some(true),
            duration_ms: 50,
        }
    }
    #[tokio::test]
    async fn from_path_empty_returns_none() {
        assert!(SessionLogger::from_path("").is_none());
    }
    #[tokio::test]
    async fn append_writes_jsonl_row_with_schema_field() {
        let dir = tempdir();
        let path = dir.join("sessions.jsonl");
        let path_str = path.to_string_lossy().to_string();
        let logger = SessionLogger::from_path(&path_str).unwrap();
        logger.append(fixture_record("trace-a")).await;
        let body = fs::read_to_string(&path).await.unwrap();
        assert!(body.contains("\"schema\":\"session.iterate.v1\""));
        assert!(body.contains("\"session_id\":\"trace-a\""));
        assert!(body.contains("\"grounded_in_roster\":true"));
        assert!(body.ends_with('\n'));
    }
    #[tokio::test]
    async fn append_concurrent_safe() {
        let dir = tempdir();
        let path = dir.join("sessions.jsonl");
        let path_str = path.to_string_lossy().to_string();
        let logger = SessionLogger::from_path(&path_str).unwrap();
        let n = 32;
        let mut handles = Vec::with_capacity(n);
        for i in 0..n {
            let l = logger.clone();
            handles.push(tokio::spawn(async move {
                l.append(fixture_record(&format!("trace-{i}"))).await;
            }));
        }
        for h in handles {
            h.await.unwrap();
        }
        let body = fs::read_to_string(&path).await.unwrap();
        let lines: Vec<_> = body.lines().filter(|l| !l.is_empty()).collect();
        assert_eq!(lines.len(), n, "expected {n} rows, got {}", lines.len());
        // Every row must round-trip through serde — a torn write
        // would surface as a parse error.
        for line in lines {
            let _: serde_json::Value = serde_json::from_str(line).expect("valid json per row");
        }
    }
    fn tempdir() -> PathBuf {
        // Per-test unique path so prior runs don't pollute the next.
        // The static counter increments across the whole test binary,
        // so back-to-back tests in the same module get distinct dirs.
        static COUNTER: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
        let n = COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
        let p = std::env::temp_dir().join(format!(
            "session_log_test_{}_{}_{}",
            std::process::id(),
            chrono::Utc::now().timestamp_nanos_opt().unwrap_or(0),
            n,
        ));
        std::fs::create_dir_all(&p).unwrap();
        p
    }
 }
@@ -62,15 +62,6 @@ pub struct GatewayConfig {
    pub host: String,
    #[serde(default = "default_gateway_port")]
    pub port: u16,
    /// Coordinator session JSONL output path. One row per
    /// `/v1/iterate` session, schema=`session.iterate.v1`. Empty =
    /// disabled. Cross-runtime parity with the Go side's
    /// `[validatord].session_log_path` (added 2026-05-02). Default
    /// empty so existing deployments aren't perturbed; production
    /// sets `/var/lib/lakehouse/gateway/sessions.jsonl`. See
    /// `golangLAKEHOUSE/docs/SESSION_LOG.md` for query examples.
    #[serde(default)]
    pub session_log_path: String,
 }
 #[derive(Debug, Clone, Deserialize)]
@@ -158,13 +149,7 @@ fn default_gateway_port() -> u16 { 3100 }
 fn default_storage_root() -> String { "./data".to_string() }
 fn default_profile_root() -> String { "./data/_profiles".to_string() }
 fn default_manifest_prefix() -> String { "_catalog/manifests".to_string() }
-// Post-2026-05-02: AiClient talks directly to Ollama; the Python
+fn default_sidecar_url() -> String { "http://localhost:3200".to_string() }
 // sidecar's hot-path role was retired. The config field name
 // `[sidecar].url` is preserved for migration compatibility (operators
 // with existing TOMLs don't need to rename anything), but the value
 // now points at Ollama. Lab UI / pipeline_lab Python remains as a
 // dev-only tool; not on this URL.
 fn default_sidecar_url() -> String { "http://localhost:11434".to_string() }
 fn default_embed_model() -> String { "nomic-embed-text".to_string() }
 fn default_gen_model() -> String { "qwen2.5".to_string() }
 fn default_rerank_model() -> String { "qwen2.5".to_string() }
@@ -199,11 +184,7 @@ impl Config {
 impl Default for Config {
    fn default() -> Self {
        Self {
-            gateway: GatewayConfig {
+            gateway: GatewayConfig { host: default_host(), port: default_gateway_port() },
                host: default_host(),
                port: default_gateway_port(),
                session_log_path: String::new(),
            },
            storage: StorageConfig {
                root: default_storage_root(),
                profile_root: default_profile_root(),
@@ -603,200 +603,3 @@ fn row_from_batch(batch: &RecordBatch, row: usize) -> Result<Row, String> {
    Ok(Row { doc_id, chunk_text, vector: v, source, chunk_idx })
 }
 // =================== Tests ===================
 //
 // All tests run against a temp directory — never the production
 // data/lance/ tree. Lance reads/writes are async + filesystem-bound,
 // so we use #[tokio::test]. Each test uses a unique per-pid + per-
 // nanosecond temp dir so concurrent runs don't collide and a re-run
 // of a single test doesn't see prior state.
 //
 // Surfaced 2026-05-02 audit: vectord-lance had ZERO tests despite
 // being on the live HTTP path. These are the load-bearing locks for
 // the public API contract.
 #[cfg(test)]
 mod tests {
    use super::*;
    fn temp_path(label: &str) -> String {
        let n = std::time::SystemTime::now()
            .duration_since(std::time::UNIX_EPOCH)
            .map(|d| d.subsec_nanos())
            .unwrap_or(0);
        let pid = std::process::id();
        std::env::temp_dir()
            .join(format!("vlance_test_{label}_{pid}_{n}"))
            .to_string_lossy()
            .to_string()
    }
    /// Build a minimal in-memory Parquet file matching vectord's
    /// binary-blob schema. Used as input to migrate_from_parquet_bytes.
    fn synth_parquet_bytes(n_rows: usize, dims: usize) -> Vec<u8> {
        use parquet::arrow::ArrowWriter;
        use std::io::Cursor;
        let schema = Arc::new(Schema::new(vec![
            Field::new("source", DataType::Utf8, true),
            Field::new("doc_id", DataType::Utf8, false),
            Field::new("chunk_idx", DataType::Int32, true),
            Field::new("chunk_text", DataType::Utf8, true),
            Field::new("vector", DataType::Binary, false),
        ]));
        let sources: Vec<Option<&str>> = (0..n_rows).map(|_| Some("test")).collect();
        let doc_ids: Vec<String> = (0..n_rows).map(|i| format!("DOC-{i:04}")).collect();
        let chunk_idxs: Vec<Option<i32>> = (0..n_rows).map(|i| Some(i as i32)).collect();
        let chunk_texts: Vec<String> = (0..n_rows).map(|i| format!("synth chunk {i}")).collect();
        let vectors: Vec<Vec<u8>> = (0..n_rows).map(|i| {
            let v: Vec<f32> = (0..dims).map(|j| (i * dims + j) as f32 * 0.01).collect();
            let mut bytes = Vec::with_capacity(dims * 4);
            for f in v { bytes.extend_from_slice(&f.to_le_bytes()); }
            bytes
        }).collect();
        let batch = RecordBatch::try_new(schema.clone(), vec![
            Arc::new(StringArray::from(sources)),
            Arc::new(StringArray::from(doc_ids)),
            Arc::new(Int32Array::from(chunk_idxs)),
            Arc::new(StringArray::from(chunk_texts)),
            Arc::new(BinaryArray::from(vectors.iter().map(|v| v.as_slice()).collect::<Vec<_>>())),
        ]).expect("synth parquet batch");
        let mut buf = Cursor::new(Vec::new());
        let mut writer = ArrowWriter::try_new(&mut buf, schema, None).expect("arrow writer");
        writer.write(&batch).expect("write batch");
        writer.close().expect("close writer");
        buf.into_inner()
    }
    #[tokio::test]
    async fn fresh_store_reports_no_state() {
        let path = temp_path("fresh");
        let store = LanceVectorStore::new(path.clone());
        assert_eq!(store.path(), path);
        assert_eq!(store.count().await.unwrap_or(0), 0);
        assert!(!store.has_vector_index().await.unwrap_or(true));
    }
    #[tokio::test]
    async fn migrate_then_count_and_fetch() {
        let path = temp_path("migrate_fetch");
        let store = LanceVectorStore::new(path.clone());
        let bytes = synth_parquet_bytes(8, 4);
        let stats = store.migrate_from_parquet_bytes(&bytes).await.expect("migrate");
        assert_eq!(stats.rows_written, 8);
        assert_eq!(stats.dimensions, 4);
        assert!(stats.disk_bytes > 0, "lance dataset should occupy disk");
        assert_eq!(store.count().await.unwrap(), 8);
        let row = store.get_by_doc_id("DOC-0003").await
            .expect("get_by_doc_id Ok").expect("DOC-0003 exists");
        assert_eq!(row.doc_id, "DOC-0003");
        assert_eq!(row.chunk_text, "synth chunk 3");
        assert_eq!(row.vector.len(), 4);
        let _ = std::fs::remove_dir_all(&path);
    }
    /// Load-bearing contract: get_by_doc_id distinguishes "dataset
    /// missing" (Err) from "id missing" (Ok(None)) so the HTTP
    /// handler can return 404 without inspecting error strings.
    #[tokio::test]
    async fn get_by_doc_id_missing_returns_none() {
        let path = temp_path("missing_id");
        let store = LanceVectorStore::new(path.clone());
        store.migrate_from_parquet_bytes(&synth_parquet_bytes(4, 4)).await.expect("migrate");
        let row = store.get_by_doc_id("DOC-NEVER-EXISTS").await.expect("Ok");
        assert!(row.is_none(), "missing id must return Ok(None), not Err");
        let _ = std::fs::remove_dir_all(&path);
    }
    /// Verifies the load-bearing structural-difference claim of
    /// ADR-019: Lance appends without rewriting the whole file. Row
    /// count grows; new rows are fetchable by their doc_ids.
    #[tokio::test]
    async fn append_grows_count_and_new_rows_fetchable() {
        let path = temp_path("append");
        let store = LanceVectorStore::new(path.clone());
        store.migrate_from_parquet_bytes(&synth_parquet_bytes(4, 4)).await.expect("migrate");
        assert_eq!(store.count().await.unwrap(), 4);
        let stats = store.append(
            Some("appended".into()),
            vec!["NEW-A".into(), "NEW-B".into()],
            vec![0, 0],
            vec!["new chunk a".into(), "new chunk b".into()],
            vec![vec![0.1, 0.2, 0.3, 0.4], vec![0.5, 0.6, 0.7, 0.8]],
        ).await.expect("append");
        assert_eq!(stats.rows_appended, 2);
        assert_eq!(store.count().await.unwrap(), 6);
        let new_a = store.get_by_doc_id("NEW-A").await.unwrap().expect("NEW-A");
        assert_eq!(new_a.chunk_text, "new chunk a");
        assert_eq!(new_a.source.as_deref(), Some("appended"));
        let _ = std::fs::remove_dir_all(&path);
    }
    /// Without this guard a dim-mismatch row would land on disk and
    /// silently break search at query time.
    #[tokio::test]
    async fn append_dim_mismatch_errors() {
        let path = temp_path("dim_mismatch");
        let store = LanceVectorStore::new(path.clone());
        store.migrate_from_parquet_bytes(&synth_parquet_bytes(4, 4)).await.expect("migrate");
        let err = store.append(
            None, vec!["X".into(), "Y".into()], vec![0, 0],
            vec!["a".into(), "b".into()],
            vec![vec![1.0, 2.0, 3.0, 4.0], vec![1.0, 2.0]],
        ).await;
        assert!(err.is_err(), "dim mismatch must error");
        let msg = err.unwrap_err();
        assert!(msg.contains("dim") || msg.contains("expected"),
            "error must mention the dimension problem; got: {msg}");
        let _ = std::fs::remove_dir_all(&path);
    }
    /// Search round-trip: query the exact vector for one row, top-1
    /// must be that row. Verifies the search path works on small
    /// datasets where IVF training would normally be skipped.
    #[tokio::test]
    async fn search_returns_nearest() {
        let path = temp_path("search");
        let store = LanceVectorStore::new(path.clone());
        store.migrate_from_parquet_bytes(&synth_parquet_bytes(8, 4)).await.expect("migrate");
        let target: Vec<f32> = (0..4).map(|j| (5 * 4 + j) as f32 * 0.01).collect();
        let hits = store.search(&target, 3, None, None).await.expect("search");
        assert!(!hits.is_empty(), "search must return at least 1 hit");
        assert_eq!(hits[0].doc_id, "DOC-0005",
            "exact-vector match should be top-1; got {hits:?}");
        let _ = std::fs::remove_dir_all(&path);
    }
    /// stats() summarizes the dataset state in one call. Locks the
    /// field shape so downstream consumers don't break on a rename.
    #[tokio::test]
    async fn stats_reports_post_migrate_state() {
        let path = temp_path("stats");
        let store = LanceVectorStore::new(path.clone());
        store.migrate_from_parquet_bytes(&synth_parquet_bytes(5, 4)).await.expect("migrate");
        let s = store.stats().await.expect("stats");
        assert_eq!(s.rows, 5);
        assert!(s.disk_bytes > 0);
        assert!(!s.has_vector_index, "no vector index built yet");
        let _ = std::fs::remove_dir_all(&path);
    }
 }
@@ -163,11 +163,7 @@ pub async fn query(
    // production caller of the Phase 21 primitives — see audit finding
    // "Phase 21 Rust primitives are wired but not CALLED by any
    // production surface" from 2026-04-21.
-    // 2026-04-30 model bump: qwen2.5:latest → qwen3.5:latest to match
+    let mut cont_opts = ContinuableOpts::new("qwen2.5:latest");
    // the small-model-pipeline local-tier default. Same JSON-clean
    // property, more capacity. think=Some(false) preserved — RAG hot
    // path doesn't need reasoning traces; direct answers only.
    let mut cont_opts = ContinuableOpts::new("qwen3.5:latest");
    cont_opts.max_tokens = Some(512);
    cont_opts.temperature = Some(0.2);
    cont_opts.shape = ResponseShape::Text;
@@ -180,7 +176,7 @@ pub async fn query(
        // echoes whatever Ollama loaded). Use the configured tier model
        // for now; if RAG needs to report the actual resolved model,
        // the runner can add a post-call ps probe later.
-        model: "qwen3.5:latest".to_string(),
+        model: "qwen2.5:latest".to_string(),
        sources: results,
        tokens_generated: None,
    })
@@ -1855,10 +1855,10 @@ async fn lance_migrate(
        .map_err(|e| (StatusCode::NOT_FOUND, format!("read parquet: {e}")))?;
    let lance_store = state.lance.store_for_new(&index_name, &bucket).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::BAD_REQUEST, e))?;
    let stats = lance_store.migrate_from_parquet_bytes(&bytes).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e))?;
    tracing::info!(
        "lance migrate '{}': {} rows, {}d, {} bytes on disk, {:.2}s",
@@ -1888,38 +1888,6 @@ fn default_partitions() -> u32 { 316 }   // ≈√100K — sane for the referenc
 fn default_bits() -> u32 { 8 }
 fn default_subvectors() -> u32 { 48 }    // 768/48 = 16 dims per subvector
 /// Sanitize a Lance backend error before returning it to the HTTP
 /// caller. Two responsibilities:
 ///
 /// 1. Map "dataset not found" patterns to HTTP 404 instead of 500.
 ///    A missing index isn't an internal failure — it's a resource
 ///    lookup miss, and the response code should reflect that.
 /// 2. Strip server-side filesystem paths and Rust crate registry
 ///    paths (`/root/.cargo/registry/src/index.crates.io-...`) from
 ///    the message body. An attacker probing the surface shouldn't
 ///    learn the server's directory layout or our exact dep versions.
 ///
 /// Surfaced 2026-05-02 by the Lance backend audit: missing-index
 /// search returned 500 + leaked the lakehouse data path AND the
 /// .cargo/registry path with crate versions.
 fn sanitize_lance_err(err: String, index_name: &str) -> (StatusCode, String) {
    let lower = err.to_lowercase();
    let is_not_found = lower.contains("not found") || lower.contains("no such file");
    let msg = if is_not_found {
        format!("lance dataset not found: {index_name}")
    } else {
        // Generic 500 with the Lance error body trimmed of paths.
        let cleaned = err
            .split("/root/.cargo/").next().unwrap_or(&err)
            .split("/home/").next().unwrap_or(&err)
            .trim_end_matches([',', ' ', '\n', '\t'])
            .to_string();
        if cleaned.is_empty() { format!("lance backend error on {index_name}") } else { cleaned }
    };
    let status = if is_not_found { StatusCode::NOT_FOUND } else { StatusCode::INTERNAL_SERVER_ERROR };
    (status, msg)
 }
 /// Build the IVF_PQ index on the Lance dataset.
 async fn lance_build_index(
    State(state): State<VectorState>,
@@ -1927,10 +1895,10 @@ async fn lance_build_index(
    Json(req): Json<LanceIndexRequest>,
 ) -> impl IntoResponse {
    let lance_store = state.lance.store_for(&index_name).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::BAD_REQUEST, e))?;
    match lance_store.build_index(req.num_partitions, req.num_bits, req.num_sub_vectors).await {
        Ok(stats) => Ok(Json(stats)),
-        Err(e) => Err(sanitize_lance_err(e, &index_name)),
+        Err(e) => Err((StatusCode::INTERNAL_SERVER_ERROR, e)),
    }
 }
@@ -1979,13 +1947,13 @@ async fn lance_search(
    let qv: Vec<f32> = embed_resp.embeddings[0].iter().map(|&x| x as f32).collect();
    let lance_store = state.lance.store_for(&index_name).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::BAD_REQUEST, e))?;
    let t0 = std::time::Instant::now();
    let nprobes = req.nprobes.or(Some(LANCE_DEFAULT_NPROBES));
    let refine = req.refine_factor.or(Some(LANCE_DEFAULT_REFINE_FACTOR));
    let hits = lance_store.search(&qv, req.top_k, nprobes, refine).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e))?;
    Ok(Json(serde_json::json!({
        "index_name": index_name,
@@ -2003,7 +1971,7 @@ async fn lance_get_doc(
    Path((index_name, doc_id)): Path<(String, String)>,
 ) -> impl IntoResponse {
    let lance_store = state.lance.store_for(&index_name).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::BAD_REQUEST, e))?;
    let t0 = std::time::Instant::now();
    match lance_store.get_by_doc_id(&doc_id).await {
        Ok(Some(row)) => Ok(Json(serde_json::json!({
@@ -2013,7 +1981,7 @@ async fn lance_get_doc(
            "row": row,
        }))),
        Ok(None) => Err((StatusCode::NOT_FOUND, format!("doc_id not found: {doc_id}"))),
-        Err(e) => Err(sanitize_lance_err(e, &index_name)),
+        Err(e) => Err((StatusCode::INTERNAL_SERVER_ERROR, e)),
    }
 }
@@ -2045,7 +2013,7 @@ async fn lance_append(
        return Err((StatusCode::BAD_REQUEST, "rows array is empty".into()));
    }
    let lance_store = state.lance.store_for(&index_name).await
-        .map_err(|e| sanitize_lance_err(e, &index_name))?;
+        .map_err(|e| (StatusCode::BAD_REQUEST, e))?;
    let mut doc_ids = Vec::with_capacity(req.rows.len());
    let mut chunk_idxs = Vec::with_capacity(req.rows.len());
@@ -1,46 +0,0 @@
 # Lakehouse: Rust vs Go architecture comparison
 > **Source of truth lives in the golangLAKEHOUSE repo:**
 > [`/home/profit/golangLAKEHOUSE/docs/ARCHITECTURE_COMPARISON.md`](file:///home/profit/golangLAKEHOUSE/docs/ARCHITECTURE_COMPARISON.md)
 >
 > J's living document — pulled from there into this repo's docs as
 > a pointer so the comparison is reachable from either side.
 ## Why the source lives in golangLAKEHOUSE
 The Go rewrite was the trigger for the comparison. The doc updates as
 J ships fixes on either side, and most of the open backlog items
 (materializer port, replay port, validators network surface) land in
 the Go repo. Keeping the source there means PR auditing on Go
 commits also catches doc drift.
 ## When to update from this side
 If a fix lands in the Rust repo that changes a comparison value
 (e.g. embed cache change, sidecar drop, new validator), update both:
 1. The source at `/home/profit/golangLAKEHOUSE/docs/ARCHITECTURE_COMPARISON.md`
 2. The change log section at the bottom of the same file
 This file is a pointer — **do not put authoritative content here.**
 Drift between two copies wastes the discipline.
 ## Quick links
 - **Decisions tracker** — section near the top of the source file.
  Lists actioned items + open backlog with LOC estimates.
 - **Performance numbers** — Python dependency section. Updated each
  time a load test is rerun.
 - **Distillation porting status** — table of phase-by-phase port
  state across runtimes.
 - **Recommendation** — current working hypothesis on Go-primary vs
  Rust-primary. Subject to change as fixes ship.
 ## Last known state
 - **2026-05-01**: Rust embed cache shipped (`150cc3b`), 236× RPS gain.
 - **2026-05-01**: Go validator port shipped (`b03521a`), production
  safety net now on Go side.
 - **Open**: Drop Rust Python sidecar (~200 LOC, universal-win).
 - **Open**: Port Rust materializer to Go (~500-800 LOC, unblocks
  Go-only end-to-end pipeline).
@@ -3,15 +3,6 @@
 [gateway]
 host = "0.0.0.0"
 port = 3100
 # Coordinator session JSONL — one row per /v1/iterate session for
 # offline DuckDB analysis. Cross-runtime parity with the Go-side
 # [validatord].session_log_path. Set to the SAME path Go validatord
 # writes to so DuckDB queries see one unified longitudinal stream
 # across both runtimes (rows are tagged daemon="gateway" vs
 # daemon="validatord" so producers stay distinguishable). Append-write
 # is atomic at the row sizes both runtimes produce — both daemons
 # co-writing is safe.
 session_log_path = "/tmp/lakehouse-validator/sessions.jsonl"
 [storage]
 root = "./data"
@@ -53,22 +44,12 @@ manifest_prefix = "_catalog/manifests"
 # max_rows_per_query = 10000
 [sidecar]
-# Post-2026-05-02: AiClient talks directly to Ollama; the Python
+url = "http://localhost:3200"
 # sidecar's hot-path role (~120 LOC of pure Ollama wrappers) was
 # retired. Field name kept for migration compat — value now points
 # at Ollama on :11434. Lab UI + pipeline_lab Python remains as a
 # dev-only tool, NOT on this URL.
 url = "http://localhost:11434"
 [ai]
 embed_model = "nomic-embed-text"
-# Local-tier defaults bumped 2026-04-30: qwen3.5:latest is the
+gen_model = "qwen2.5"
-# stronger local rung in the 5-loop substrate (per
+rerank_model = "qwen2.5"
 # project_small_model_pipeline_vision.md). Same JSON-clean property
 # as qwen2.5, more capacity. Ollama still serves both — bump back
 # in this file if a workload regressed.
 gen_model = "qwen3.5:latest"
 rerank_model = "qwen3.5:latest"
 [auth]
 enabled = false
@@ -91,9 +72,7 @@ min_recall = 0.9                          # never promote below this
 max_trials_per_hour = 20                  # hard budget cap
 # Model roster — available for profile hot-swap
 # qwen3.5:latest: stronger local rung — JSON-clean, 8K+ context,
 #                 default for gen_model and rerank_model
 # qwen3: 8.2B, 40K context, thinking+tools, best for reasoning tasks
-# qwen2.5: 7B, 8K context, fast — kept loaded for the 2026-04 era
+# qwen2.5: 7B, 8K context, fast, good for SQL generation
-#          comparison runs; new defaults use qwen3.5:latest
+# mistral: 7B, 8K context, good for general generation
 # nomic-embed-text: 137M, embedding-only, used by all profiles
@@ -56,12 +56,7 @@ details .body{padding-top:10px;font-size:12px;color:#8b949e}
 .worker{display:flex;align-items:center;gap:10px;padding:8px 10px;background:#161b22;border-radius:6px;margin-bottom:4px;font-size:12px;border-left:3px solid #30363d}
 .worker .av{width:32px;height:32px;border-radius:50%;background:#0d1117;border:1px solid #21262d;display:flex;align-items:center;justify-content:center;font-weight:600;color:#c9d1d9;font-size:11px;flex-shrink:0;letter-spacing:0.5px;overflow:hidden;position:relative}
-.worker .av img{position:absolute;inset:0;width:100%;height:100%;object-fit:cover;display:block;
+.worker .av img{position:absolute;inset:0;width:100%;height:100%;object-fit:cover;display:block}
  /* Softening — mirror of search.html. Pulls saturation + contrast off
     the SDXL Turbo over-render so faces feel less "AI-generated".
     If you tweak one, tweak the other. */
  filter: saturate(0.86) contrast(0.93) brightness(1.02) blur(0.3px);
 }
 .worker[data-role-band="warehouse"]{border-left-color:#58a6ff}
 .worker[data-role-band="production"]{border-left-color:#d29922}
 .worker[data-role-band="trades"]{border-left-color:#bc8cff}
@@ -267,37 +262,6 @@ var NAMES_EAST_ASIAN_C=new Set(['Wei','Mei','Yi','Jin','Chen','Lin','Liu','Wang'
 var NAMES_HISPANIC_C=new Set(['Carmen','Carlos','Maria','Diego','Hector','Jorge','Julio','Manuel','Miguel','Pedro','Raul','Ricardo','Roberto','Sergio','Antonio','Esperanza','Luz','Sofia','Lucia','Isabella','Camila','Valentina','Mariana','Elena','Rosa','Catalina','Esteban','Fernando','Eduardo','Javier','Alejandro','Andres','Mateo','Santiago','Sebastian','Emilio','Tomas','Cristina','Daniela','Gabriela','Ximena','Adriana','Beatriz','Pilar','Mercedes','Xavier','Marisol','Guadalupe','Lupita','Inez','Itzel','Yesenia','Joaquin','Ignacio','Rafael','Salvador','Cesar','Arturo','Armando','Hugo','Marco','Alejandra','Felipe','Gerardo','Jaime','Leonardo','Luis','Pablo','Ramon']);
 var NAMES_BLACK_C=new Set(['DeShawn','Jamal','Aisha','Latoya','Tyrone','Malik','Imani','Keisha','Tariq','Lakisha','Kenya','Tamika','Andre','Marcus','Demetrius','Jermaine','Reggie','Tyrese','Darius','Trevon','Kareem','Damon','Jalen','Jaylen','Dwayne','DaQuan','Aaliyah','Kiara','Janelle','Jasmine','Tanisha','Maurice','Tyrell','Kwame','Khalil','Terrell','Cedric','Nia','Zuri','Jada','Ebony','Dominique']);
 var NAMES_MIDDLE_EASTERN_C=new Set(['Layla','Omar','Khalid','Fatima','Yasmin','Hassan','Hussein','Ahmed','Mohamed','Mohammed','Ali','Karim','Yusuf','Yara','Nadia','Zainab','Rania','Samira','Mariam','Salma','Ibrahim','Mahmoud','Saif','Anwar','Bilal','Faisal','Hamza','Imran','Sami','Wael','Zaid','Amira','Iman','Lina','Mona','Noor','Rana','Soha','Zara']);
 // Surname → ethnicity. Surname is more diagnostic than first name
 // for hispanic and asian — "Anna Cruz" is hispanic via surname.
 var SURNAMES_HISPANIC_C=new Set(['Garcia','Rodriguez','Martinez','Hernandez','Lopez','Gonzalez','Perez','Sanchez','Ramirez','Torres','Flores','Rivera','Gomez','Diaz','Reyes','Cruz','Morales','Ortiz','Gutierrez','Chavez','Ramos','Ruiz','Alvarez','Mendoza','Vasquez','Castillo','Jimenez','Moreno','Romero','Herrera','Medina','Aguilar','Vargas','Castro','Fernandez','Guzman','Munoz','Salazar','Ortega','Delgado','Estrada','Ayala','Pena','Cabrera','Alvarado','Espinoza','Padilla','Cardenas','Cortes','Ibarra','Vega','Soto','Lara','Navarro','Campos','Acosta','Rios','Marquez','Sandoval','Maldonado','Solis','Rojas','Mejia','Beltran','Cervantes','Lozano','Carrillo','Trevino','Robles','Tapia','Lugo']);
 var SURNAMES_SOUTH_ASIAN_C=new Set(['Patel','Singh','Kumar','Sharma','Gupta','Shah','Mehta','Desai','Joshi','Reddy','Nair','Iyer','Verma','Agarwal','Kapoor','Chopra','Malhotra','Banerjee','Chatterjee','Mukherjee','Das','Sen','Bose','Roy','Sinha','Trivedi','Pandey','Mishra','Tiwari','Yadav','Chauhan','Rana','Thakur','Pillai','Menon','Krishnan','Rao','Naidu','Pradhan','Acharya','Devi','Kaur']);
 var SURNAMES_EAST_ASIAN_C=new Set(['Chen','Wang','Li','Liu','Yang','Huang','Zhao','Wu','Zhou','Xu','Zhu','Sun','Ma','Lin','Lee','Kim','Park','Choi','Jung','Kang','Cho','Yoon','Han','Lim','Oh','Nakamura','Tanaka','Suzuki','Yamamoto','Sato','Watanabe','Takahashi','Kobayashi','Yoshida','Saito','Nguyen','Tran','Le','Pham','Hoang','Phan','Vu','Vo','Dang','Bui','Do','Ngo','Truong','Mai','Cao','Wong','Tang','Tan','Cheng','Lau','Leung','Ng','Cheung','Yip','Hsu','Tsai','Hsieh']);
 var SURNAMES_MIDDLE_EASTERN_C=new Set(['Khan','Ahmed','Hussein','Hassan','Ali','Mahmoud','Mohamed','Mohammed','Saleh','Aziz','Karim','Hamad','Najjar','Haddad','Khoury','Mansour','Rahman','Iqbal','Malik','Sheikh','Siddiqui','Qureshi','Saeed']);
 function guessEthnicityFromName(first, last){
  if(last){
    var s=last.replace(/[^A-Za-z]/g,'');
    if(s){
      var sc=s[0].toUpperCase()+s.slice(1).toLowerCase();
      if(SURNAMES_HISPANIC_C.has(sc)) return 'hispanic';
      if(SURNAMES_MIDDLE_EASTERN_C.has(sc)) return 'middle_eastern';
      if(SURNAMES_SOUTH_ASIAN_C.has(sc)) return 'south_asian';
      if(SURNAMES_EAST_ASIAN_C.has(sc)) return 'east_asian';
    }
  }
  if(first){
    var clean=first.replace(/[^A-Za-z]/g,'');
    if(clean){
      var c=clean[0].toUpperCase()+clean.slice(1).toLowerCase();
      if(NAMES_MIDDLE_EASTERN_C.has(c)) return 'middle_eastern';
      if(NAMES_BLACK_C.has(c)) return 'black';
      if(NAMES_HISPANIC_C.has(c)) return 'hispanic';
      if(NAMES_SOUTH_ASIAN_C.has(c)) return 'south_asian';
      if(NAMES_EAST_ASIAN_C.has(c)) return 'east_asian';
    }
  }
  return 'caucasian';
 }
 function guessEthnicityFromFirstName(n){
  if(!n) return 'caucasian';
  var clean=n.replace(/[^A-Za-z]/g,''); if(!clean) return 'caucasian';
@@ -317,8 +281,21 @@ function workerRow(name, role, detail, opts){
  if(band.band) w.dataset.roleBand = band.band;
  var initials = (name||'?').split(' ').map(function(s){return (s[0]||'').toUpperCase()}).join('').substring(0,2);
  var av = el('div','av',initials);
-  // Headshot insertion removed 2026-04-28. The .av element stays as
+  // Real synthetic headshot via /headshots/<key>; deterministic so
-  // a monogram-initials avatar.
+  // same worker always gets the same face. Falls back to monogram if
  // pool isn't fetched yet.
  var faceKey = (opts.face_key) || name || '';
  var firstName = (name||'').split(/\s+/)[0]||'';
  var gHint = genderFor(firstName);
  var eHint = guessEthnicityFromFirstName(firstName);
  if(faceKey){
    var img=document.createElement('img');
    img.alt='';
    img.loading='lazy';
    img.src = P + '/headshots/' + encodeURIComponent(faceKey) + '?g='+gHint+'&e='+eHint;
    img.onerror=function(){ this.remove(); };
    av.appendChild(img);
  }
  w.appendChild(av);
  var info = el('div','info');
  var nm = el('div','nm', name||'?');
@@ -1,123 +0,0 @@
 // Visual filler iconography rendered through ComfyUI. Distinct from
 // role_scenes.ts (which renders portraits) — these are object/badge
 // style renders that fill dead space on worker cards: cert pills,
 // role-prop chips, hazard indicators, empty-state heroes.
 //
 // Layout on disk:
 //   data/icons_pool/{category}/{slug}.webp
 //
 // Cache invalidation:
 //   ICONS_VERSION mixes into the on-disk filename (slug includes
 //   version). Bump it after editing a recipe so prior renders are
 //   ignored on next view.
 export type IconCategory = "cert" | "role_prop" | "status" | "hazard" | "empty";
 export interface IconRecipe {
  slug: string;
  category: IconCategory;
  // Text label that appears next to / under the icon. The front-end
  // already renders this text in cert pills; the icon is supplementary.
  display: string;
  // Full diffusion prompt. Style guidance baked in. SDXL Turbo at 8
  // steps reliably produces clean macro photography, so default to
  // photographic prop shots over flat-vector illustrations (the model
  // hallucinates noise into flat-vector geometry at low step counts).
  prompt: string;
  // Negative prompt — what NOT to render. Crucial for icons because
  // SDXL likes to add hands/text/people unprompted.
  negative?: string;
 }
 // Default negative prompt baked into every icon render unless the
 // recipe overrides. Empirically, these terms are the top SDXL Turbo
 // off-style failures.
 export const DEFAULT_NEGATIVE =
  "people, hands, faces, blurry, low quality, watermark, signature, "
  + "logos, copyright, distorted text, garbled letters, multiple objects";
 // TODO J — review and tune the prompts here. Each one is what diffusion
 // sees verbatim. The visual decision: photographic prop shots (macro
 // photo of an actual badge / placard / sticker) vs flat-icon vector
 // style. Default below is photographic — matches the worker headshot
 // aesthetic. Flip a recipe to flat-vector by replacing "macro photograph"
 // with "flat icon illustration on solid color background, minimal vector".
 //
 // Visual cues that work well in SDXL Turbo at 8 steps:
 //   - "macro photograph", "isolated on plain background", "studio lighting"
 //   - Concrete colors ("orange and black warning diamond") not adjectives
 //   - Avoid: small text in the prompt (model garbles it), specific brand
 //     names (creates fake logos), detailed scene composition
 const CERT_ICONS: IconRecipe[] = [
  { slug: "osha-10", category: "cert", display: "OSHA-10",
    prompt: "macro photograph of a circular yellow safety badge with a black hard hat icon at center, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "osha-30", category: "cert", display: "OSHA-30",
    prompt: "macro photograph of a circular orange safety badge with a black hard hat icon at center, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "first-aid-cpr", category: "cert", display: "First Aid/CPR",
    prompt: "macro photograph of a small enamel pin badge featuring a bold red cross on a white circular background, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "hazmat", category: "cert", display: "Hazmat",
    prompt: "macro photograph of a HAZMAT warning placard, bold orange and black diamond shape with a flame icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "forklift", category: "cert", display: "Forklift",
    prompt: "macro photograph of a yellow industrial forklift safety badge with a forklift silhouette icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "reach-truck", category: "cert", display: "Reach Truck",
    prompt: "macro photograph of a navy blue industrial certification badge with a warehouse reach-truck silhouette icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "order-picker", category: "cert", display: "Order Picker",
    prompt: "macro photograph of a green industrial certification badge with a warehouse order-picker silhouette icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "lockout-tagout", category: "cert", display: "Lockout/Tagout",
    prompt: "macro photograph of a bright red padlock tag with a danger warning, hanging on a metal industrial valve, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "msds", category: "cert", display: "MSDS",
    prompt: "macro photograph of a folded chemical safety data sheet booklet with chemical hazard pictograms visible on cover, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "confined-space", category: "cert", display: "Confined Space",
    prompt: "macro photograph of a yellow confined space warning sign featuring a manhole entry icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "servsafe", category: "cert", display: "ServSafe",
    prompt: "macro photograph of a dark green food safety certification badge featuring a stylized chef hat icon, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "fire-safety", category: "cert", display: "Fire Safety",
    prompt: "macro photograph of a red enamel pin badge featuring a flame icon and a fire extinguisher silhouette, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "iso-9001", category: "cert", display: "ISO 9001",
    prompt: "macro photograph of a deep blue circular quality-management certification seal with embossed metallic ring, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
 ];
 // Role-band visual chips — small icons that go in the role pill area.
 // One per band, optional inline supplement to the existing colored pill.
 const ROLE_PROP_ICONS: IconRecipe[] = [
  { slug: "warehouse", category: "role_prop", display: "Warehouse",
    prompt: "macro photograph of a yellow hard hat with a high-visibility safety vest folded behind it, isolated on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "production", category: "role_prop", display: "Production",
    prompt: "macro photograph of a navy blue work shirt and protective safety glasses on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "trades", category: "role_prop", display: "Trades",
    prompt: "macro photograph of a leather work glove and a small adjustable wrench on a neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "driver", category: "role_prop", display: "Driver",
    prompt: "macro photograph of a navy delivery driver baseball cap and a clipboard manifest on a neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
  { slug: "lead", category: "role_prop", display: "Lead",
    prompt: "macro photograph of a tablet showing a bar chart and a high-vis vest folded beside it on neutral grey backdrop, photorealistic, sharp focus, studio lighting" },
 ];
 export const ICONS: Record<string, IconRecipe> = Object.fromEntries(
  [...CERT_ICONS, ...ROLE_PROP_ICONS].map((r) => [`${r.category}/${r.slug}`, r]),
 );
 // v2 — 256×256 canvas, intended to be displayed monochrome via CSS
 // `filter: grayscale(1)`. Smaller canvas, tighter crops, crisper at
 // 14px display size.
 export const ICONS_VERSION = "v2";
 // Map a free-form cert string from the data ("First Aid/CPR",
 // "OSHA-10", "Lockout/Tagout") to the canonical slug used here.
 // Returns null if no recipe matches.
 export function certToSlug(cert: string): string | null {
  const c = (cert || "").trim().toLowerCase().replace(/\s+/g, "-");
  if (c === "osha-10") return "osha-10";
  if (c === "osha-30") return "osha-30";
  if (c.startsWith("first") || c.includes("cpr")) return "first-aid-cpr";
  if (c === "hazmat" || c.startsWith("hazwoper")) return "hazmat";
  if (c === "forklift" || c.startsWith("pit")) return "forklift";
  if (c.startsWith("reach")) return "reach-truck";
  if (c.startsWith("order")) return "order-picker";
  if (c.startsWith("lockout") || c.includes("tagout")) return "lockout-tagout";
  if (c === "msds" || c.startsWith("ghs")) return "msds";
  if (c.startsWith("confined")) return "confined-space";
  if (c === "servsafe") return "servsafe";
  if (c.startsWith("fire")) return "fire-safety";
  if (c.startsWith("iso")) return "iso-9001";
  return null;
 }
@@ -19,17 +19,8 @@ import { SSEServerTransport } from "@modelcontextprotocol/sdk/server/sse.js";
 import { z } from "zod";
 import { startTrace, logSpan, logGeneration, scoreTrace, flush as flushTraces } from "./tracing.js";
 import { buildPermitBrief } from "./entity.js";
 import { roleBand, SCENES, SCENES_VERSION, FACE_RENDER_DIM, type RoleBand } from "./role_scenes.js";
 import { ICONS, ICONS_VERSION, DEFAULT_NEGATIVE, certToSlug, type IconRecipe } from "./icon_recipes.js";
 const BASE = process.env.LAKEHOUSE_URL || "http://localhost:3100";
 // G5 cutover prep (2026-05-01): when GO_LAKEHOUSE_URL is set, the
 // `/_go/*` pass-through routes to the Go gateway. Off-by-default
 // (empty = disabled, returns 503 on /_go/*). Reversible via unset.
 // Doesn't modify any existing tool — additive parallel path so
 // operators can validate Go-side handlers under real Bun-frontend
 // shape without touching the production Rust path.
 const GO_BASE = process.env.GO_LAKEHOUSE_URL || "";
 const PORT = parseInt(process.env.MCP_PORT || "3700");
 // ─── Staffer roster — used by the per-staffer hot-swap index (G). ────
@@ -320,9 +311,9 @@ ${(buckets as any[] || []).map((b: any) => `- ${b.name}: ${b.backend} (${b.reach
 - Ollama: :11434
 ## Available Models
 - qwen3.5:latest: stronger local rung, JSON-clean (default for gen + rerank)
 - qwen3: 8.2B, 40K context, thinking+tools (best for reasoning)
- qwen2.5: 7B, 8K context (legacy — 2026-04 era comparison runs only)
+- qwen2.5: 7B, 8K context (best for fast SQL generation)
 - mistral: 7B, 8K context (general generation)
 - nomic-embed-text: 137M (embedding, automatic)
 `;
  return { contents: [{ uri: uri.href, mimeType: "text/plain", text }] };
@@ -718,26 +709,6 @@ async function main() {
          return new Response(await r.text(), { status: r.status, headers: { "Content-Type": "application/json" } });
        }
        // G5 cutover slice: pass-through to GO_BASE (Go gateway). Same
        // shape as /api/* but points at the Go side. Returns 503 when
        // GO_LAKEHOUSE_URL isn't set so callers know the cutover slice
        // is off rather than silently routing to Rust. No body
        // transformation — caller responsible for sending Go-shaped
        // requests (e.g. /v1/embed not /ai/embed).
        if (url.pathname.startsWith("/_go/")) {
          if (!GO_BASE) {
            return err("GO_LAKEHOUSE_URL not set; /_go/* cutover slice is disabled", 503);
          }
          const path = url.pathname.replace("/_go", "");
          const body = req.method !== "GET" ? await req.text() : undefined;
          try {
            const r = await fetch(`${GO_BASE}${path}`, { method: req.method, headers: { "Content-Type": "application/json" }, body });
            return new Response(await r.text(), { status: r.status, headers: { "Content-Type": "application/json" } });
          } catch (e) {
            return err(`Go gateway unreachable at ${GO_BASE}: ${e}`, 502);
          }
        }
        // Proof — narrative HTML served from mcp-server/proof.html.
        // Live tests consumed client-side via /proof.json.
        if (url.pathname === "/proof") {
@@ -1254,191 +1225,6 @@ async function main() {
        // OSHA national, Chicago history, ticker chart, parent link,
        // federal contracts, debarment, unions, training. Click any
        // contractor name in a permit Entity Brief to land here.
        // ComfyUI-generated portrait — every call is unique by (key,
        // gender, race, age, role) tuple. First hit takes ~1.5s on
        // the A4000; subsequent hits read from disk. Use this for
        // contractor / profile modal where one worker gets the
        // spotlight. NB: declared BEFORE the pool route so the prefix
        // match doesn't intercept it.
        // Single source of truth for the pre-render script. Read
        // role_scenes.ts SCENES + SCENES_VERSION so a Python pre-render
        // job (scripts/staffing/render_role_pool.py) builds the role-
        // aware pool with the exact prompts the server will use on the
        // ComfyUI hot-path. No drift.
        if (url.pathname === "/headshots/_scenes" && req.method === "GET") {
          return Response.json({ version: SCENES_VERSION, scenes: SCENES });
        }
        // Single source of truth for icon_recipes.ts. Used by the
        // pre-render script (scripts/staffing/render_icons.py) and any
        // tooling that wants to enumerate available icons.
        if (url.pathname === "/icons/_recipes" && req.method === "GET") {
          return Response.json({
            version: ICONS_VERSION,
            default_negative: DEFAULT_NEGATIVE,
            recipes: ICONS,
          });
        }
        // Free-text cert resolver: front-end passes the raw cert string
        // from the data ("First Aid/CPR", "OSHA-10", "Lockout/Tagout")
        // and we resolve to a recipe slug + 302 to the cached/rendered
        // icon. Returns 404 (not error) when no recipe matches — the
        // front-end can hang an `onerror="this.remove()"` to silently
        // drop the img tag for unrecognized certs.
        if (url.pathname === "/icons/cert" && req.method === "GET") {
          const text = url.searchParams.get("text") || "";
          const slug = certToSlug(text);
          if (!slug) return new Response(`no recipe for cert: ${text}`, { status: 404 });
          return new Response(null, {
            status: 302,
            headers: { "Location": `/icons/render/cert/${slug}` },
          });
        }
        // Cert / role-prop / status / hazard / empty icons. Lookup is
        // category/slug; on cache miss the route renders via ComfyUI.
        // Filename layout: data/icons_pool/{category}/{slug}_{version}.webp
        // — the version suffix means editing a recipe yields a new file
        // rather than overwriting in place, so a misfire is recoverable.
        if (url.pathname.startsWith("/icons/render/") && req.method === "GET") {
          const rest = url.pathname.slice("/icons/render/".length);
          const recipe: IconRecipe | undefined = ICONS[rest];
          if (!recipe) return new Response(`unknown icon: ${rest}`, { status: 404 });
          const ICONS_DIR = "/home/profit/lakehouse/data/icons_pool";
          await Bun.$`mkdir -p ${ICONS_DIR}/${recipe.category}`.quiet();
          const cachePath = `${ICONS_DIR}/${recipe.category}/${recipe.slug}_${ICONS_VERSION}.webp`;
          const cached = Bun.file(cachePath);
          if (await cached.exists()) {
            return new Response(cached, {
              headers: {
                "Content-Type": "image/webp",
                "Cache-Control": "public, max-age=86400",
                "X-Icon-Source": "cached",
                "X-Icon-Recipe": recipe.slug,
              },
            });
          }
          // Deterministic seed per recipe — same recipe always renders
          // the same icon. Mixing the version means SCENES_VERSION-
          // style invalidation works for icons too.
          const seedStr = `${recipe.category}|${recipe.slug}|${ICONS_VERSION}`;
          let seed = 5381;
          for (let i = 0; i < seedStr.length; i++) seed = ((seed << 5) + seed + seedStr.charCodeAt(i)) | 0;
          seed = Math.abs(seed) % 2147483647;
          try {
            const genResp = await fetch("http://localhost:3600/generate", {
              method: "POST",
              headers: { "Content-Type": "application/json" },
              body: JSON.stringify({
                prompt: recipe.prompt,
                negative_prompt: recipe.negative ?? DEFAULT_NEGATIVE,
                // 256×256 — smaller canvas = cleaner icon. SDXL Turbo
                // at 8 steps adds visible texture/noise into 512² that
                // looks "AI" at small display sizes; tightening to 256
                // both renders ~3× faster and produces crisper edges
                // when the front-end downsamples to 14px.
                width: 256,
                height: 256,
                steps: 8,
                seed,
              }),
              signal: AbortSignal.timeout(30000),
            });
            if (!genResp.ok) return new Response(`gen failed: ${genResp.status}`, { status: 502 });
            const data: any = await genResp.json();
            if (!data.image) return new Response("no image returned", { status: 502 });
            const bytes = Uint8Array.from(atob(data.image), (c) => c.charCodeAt(0));
            await Bun.write(cachePath, bytes);
            return new Response(bytes, {
              headers: {
                "Content-Type": "image/webp",
                "Cache-Control": "public, max-age=86400",
                "X-Icon-Source": "fresh",
                "X-Icon-Recipe": recipe.slug,
                "X-Icon-Gen-Ms": String(data.time_ms || 0),
              },
            });
          } catch (e: any) {
            return new Response(`gen error: ${e.message}`, { status: 502 });
          }
        }
        if (url.pathname.startsWith("/headshots/generate/") && req.method === "GET") {
          const key = decodeURIComponent(url.pathname.slice("/headshots/generate/".length));
          if (!key) return new Response("missing key", { status: 400 });
          const g = (url.searchParams.get("g") || "person").toLowerCase();
          const r = (url.searchParams.get("e") || "").toLowerCase();
          const role = (url.searchParams.get("role") || "warehouse worker").toLowerCase();
          const age = parseInt(url.searchParams.get("age") || "32", 10) || 32;
          const band = roleBand(role);
          // SCENES_VERSION mixes into the cache key so editing
          // role_scenes.ts auto-invalidates prior renders — coordinator
          // tweaks the warehouse prompt, every warehouse face refreshes
          // on next view.
          const cacheKey = await crypto.subtle.digest(
            "SHA-256",
            new TextEncoder().encode(`${key}|${g}|${r}|${role}|${age}|${SCENES_VERSION}`)
          ).then((b) => Array.from(new Uint8Array(b)).map((x) => x.toString(16).padStart(2, "0")).join("").slice(0, 24));
          const GEN_DIR = "/home/profit/lakehouse/data/headshots_gen";
          await Bun.$`mkdir -p ${GEN_DIR}`.quiet();
          const cachePath = `${GEN_DIR}/${cacheKey}.webp`;
          const cached = Bun.file(cachePath);
          if (await cached.exists()) {
            return new Response(cached, {
              headers: {
                "Content-Type": "image/webp",
                "Cache-Control": "public, max-age=86400, immutable",
                "X-Headshot-Source": "comfyui-cached",
              },
            });
          }
          const raceText = r === "hispanic" ? "Hispanic"
            : r === "black" ? "Black"
            : r === "south_asian" ? "South Asian"
            : r === "east_asian" ? "East Asian"
            : r === "middle_eastern" ? "Middle Eastern"
            : "";
          const genderText = g === "woman" ? "woman" : g === "man" ? "man" : "person";
          const scene = SCENES[band].scene;
          // Note: dropped "plain studio background" / "dslr" — those
          // collapsed every render to interchangeable studio shots.
          // The scene clause now carries clothing + backdrop so a
          // forklift operator looks like a forklift operator.
          const prompt = `professional headshot portrait of a ${age}-year-old ${raceText} ${genderText} ${role}, ${scene}, neutral confident expression, sharp focus, photorealistic`;
          // Worker-derived seed — same input always picks the same
          // pixel layout in StyleGAN2 latent space, so the face is
          // deterministic per worker BUT distinct from any other
          // worker that happens to share the same prompt. Without
          // this, every (g, r, age, role) combo collapses to one face.
          let seedHash = 0;
          for (let i = 0; i < key.length; i++) seedHash = ((seedHash << 5) - seedHash + key.charCodeAt(i)) | 0;
          const seed = Math.abs(seedHash) % 2147483647;
          try {
            const genResp = await fetch("http://localhost:3600/generate", {
              method: "POST",
              headers: { "Content-Type": "application/json" },
              body: JSON.stringify({ prompt, width: FACE_RENDER_DIM, height: FACE_RENDER_DIM, steps: 8, seed }),
              signal: AbortSignal.timeout(30000),
            });
            if (!genResp.ok) return new Response(`gen failed: ${genResp.status}`, { status: 502 });
            const data: any = await genResp.json();
            if (!data.image) return new Response("no image returned", { status: 502 });
            const bytes = Uint8Array.from(atob(data.image), (c) => c.charCodeAt(0));
            await Bun.write(cachePath, bytes);
            return new Response(bytes, {
              headers: {
                "Content-Type": "image/webp",
                "Cache-Control": "public, max-age=86400, immutable",
                "X-Headshot-Source": "comfyui-fresh",
                "X-Headshot-Gen-Ms": String(data.time_ms || 0),
              },
            });
          } catch (e: any) {
            return new Response(`gen error: ${e.message}`, { status: 502 });
          }
        }
        // Headshot pool — synthetic StyleGAN faces from
        // thispersondoesnotexist.com fetched offline by
        // scripts/staffing/fetch_face_pool.py. Deterministic mapping:
@@ -1463,47 +1249,19 @@ async function main() {
              const raw = await Bun.file(`${HEADSHOT_DIR}/manifest.jsonl`).text();
              const lines = raw.trim().split("\n").filter(Boolean);
              const all = lines.map((l) => JSON.parse(l));
              // Build (gender × race) buckets so a request that names
              // both narrows to the intersection. Missing intersections
              // fall back to gender-only, then race-only, then all.
              const byGR: Record<string, any[]> = {};
              const byG: Record<string, any[]> = { man: [], woman: [] };
              const byR: Record<string, any[]> = {};
              // Filter excluded faces (e.g. minors) from every bucket
              // and from the all-pool. They never get served.
              const adults = all.filter((r: any) => !r.excluded);
              for (const r of adults) {
                if (r.gender === "man" || r.gender === "woman") byG[r.gender].push(r);
                if (r.race) {
                  byR[r.race] = byR[r.race] || [];
                  byR[r.race].push(r);
                  if (r.gender === "man" || r.gender === "woman") {
                    const k = r.gender + "/" + r.race;
                    byGR[k] = byGR[k] || [];
                    byGR[k].push(r);
                  }
                }
              }
              (globalThis as any)._faces = {
-                all: adults,
+                all,
-                byG, byR, byGR,
+                man: all.filter((r: any) => r.gender === "man"),
-                untagged: adults.filter((r: any) => !r.gender || (r.gender !== "man" && r.gender !== "woman")),
+                woman: all.filter((r: any) => r.gender === "woman"),
-                excluded_count: all.length - adults.length,
+                untagged: all.filter((r: any) => !r.gender || (r.gender !== "man" && r.gender !== "woman")),
                loaded_at: Date.now(),
              };
              if (key === "__reload") {
                const byRSummary: Record<string, number> = {};
                for (const k of Object.keys(byR)) byRSummary[k] = byR[k].length;
                const byGRSummary: Record<string, number> = {};
                for (const k of Object.keys(byGR)) byGRSummary[k] = byGR[k].length;
                return Response.json({
                  reloaded: true,
                  total: all.length,
-                  excluded: all.length - adults.length,
+                  man: (globalThis as any)._faces.man.length,
-                  served_pool: adults.length,
+                  woman: (globalThis as any)._faces.woman.length,
                  by_gender: { man: byG.man.length, woman: byG.woman.length },
                  by_race: byRSummary,
                  by_gender_race: byGRSummary,
                  untagged: (globalThis as any)._faces.untagged.length,
                });
              }
@@ -1511,84 +1269,20 @@ async function main() {
              return new Response(`face pool not available: ${e.message}. Run scripts/staffing/fetch_face_pool.py first.`, { status: 503 });
            }
          }
-          const F = (globalThis as any)._faces as {
+          const F = (globalThis as any)._faces as { all: any[]; man: any[]; woman: any[]; untagged: any[] };
            all: any[];
            byG: Record<string, any[]>;
            byR: Record<string, any[]>;
            byGR: Record<string, any[]>;
            untagged: any[];
          };
          if (!F || !F.all.length) {
            return new Response("face pool empty", { status: 503 });
          }
-          const wantRace = url.searchParams.get("e") || "";
+          // Pool selection: gender hint > full pool. If no gender match,
-
+          // fall back to the full pool so the worker still gets a face.
          // NOTE: role-aware pool + ComfyUI sparse redirect were removed
          // 2026-04-28 — diffusion output at 8 steps with the existing
          // editorial_hero workflow produced burnt-looking faces ("looks
          // like someone burnt the pictures"). Until serve_imagegen.py
          // is fixed to honor a portrait-friendly negative prompt and
          // run with proper steps/cfg, every face comes from the studio
          // pool (StyleGAN photos from thispersondoesnotexist.com) and
          // gets B&W via CSS. The role pool files at
          // data/headshots_role_pool/{v1,v2}/ stay on disk for when
          // we can re-enable them.
          // Studio pool only. Try gender×race intersection first, then
          // fall back to gender-only or race-only if the intersection
          // is sparse. Repeat faces are acceptable — better than
          // serving the over-contrasty diffusion output.
          let pool = F.all;
-          let bucket = "all";
+          if (wantGender === "man" && F.man.length) pool = F.man;
-          if (wantGender && wantRace) {
+          else if (wantGender === "woman" && F.woman.length) pool = F.woman;
            const gr = F.byGR[wantGender + "/" + wantRace] || [];
            if (gr.length > 0) {
              // Use the intersection bucket as-is — even sparse buckets
              // (south_asian: 3, black: 14) just repeat photos rather
              // than route to ComfyUI. Repetition is fine; burnt faces
              // are not.
              pool = gr;
              bucket = `gr:${wantGender}/${wantRace}`;
            } else if (F.byG[wantGender]?.length) {
              pool = F.byG[wantGender];
              bucket = `g:${wantGender}`;
            }
          } else if (wantGender && F.byG[wantGender]?.length) {
            pool = F.byG[wantGender];
            bucket = `g:${wantGender}`;
          } else if (wantRace && F.byR[wantRace]?.length) {
            pool = F.byR[wantRace];
            bucket = `r:${wantRace}`;
          }
          // Hash key → pool index. djb2-ish, fits any string.
          let h = 5381;
          for (let i = 0; i < key.length; i++) h = ((h << 5) + h + key.charCodeAt(i)) | 0;
          const idx = Math.abs(h) % pool.length;
          const pick = pool[idx];
          // Prefer pre-resized webp thumb (~10KB) over native JPEG
          // (~580KB). 60× smaller — without this, a 40-card grid
          // overruns Chrome's parallel-connection budget and ~75% of
          // tiles never finish decoding.
          //
          // Cache-Control: 1h public + must-revalidate, NOT immutable.
          // We deliberately let the browser re-check after pool retags
          // or face-pool refreshes — `immutable` was pinning stale
          // photos for 24h after a server-side update.
          const thumbName = pick.file.replace(/\.jpg$/, ".webp");
          const thumb = Bun.file(`${HEADSHOT_DIR}/_thumbs/${thumbName}`);
          if (await thumb.exists()) {
            return new Response(thumb, {
              headers: {
                "Content-Type": "image/webp",
                "Cache-Control": "public, max-age=3600, must-revalidate",
                "X-Face-Pool-Idx": String(pick.id),
                "X-Face-Pool-Gender": pick.gender || "untagged",
                "X-Face-Pool-Bucket": bucket,
                "X-Face-Pool-Bucket-Size": String(pool.length),
                "X-Face-Pool-Variant": "thumb-384",
              },
            });
          }
          const file = Bun.file(`${HEADSHOT_DIR}/${pick.file}`);
          if (!(await file.exists())) {
            return new Response("face missing on disk", { status: 404 });
@@ -1596,12 +1290,9 @@ async function main() {
          return new Response(file, {
            headers: {
              "Content-Type": "image/jpeg",
-              "Cache-Control": "public, max-age=3600, must-revalidate",
+              "Cache-Control": "public, max-age=86400, immutable",
              "X-Face-Pool-Idx": String(pick.id),
              "X-Face-Pool-Gender": pick.gender || "untagged",
              "X-Face-Pool-Bucket": bucket,
              "X-Face-Pool-Bucket-Size": String(pool.length),
              "X-Face-Pool-Variant": "native-1024",
            },
          });
        }
@@ -2081,88 +1772,15 @@ async function main() {
                .reduce((s, c) => s + (c.implied_pay_rate - contractBillRate) * hoursPerWeek * weeksAssumed, 0);
              // Shift inference from permit work_type + description.
-              // Description keywords trump the hash-based assignment;
+              // Construction defaults to 1st-shift (day). Heavy civil or
-              // for everything else we deterministically distribute
+              // facility work sometimes runs 2nd or split-shift. 3rd
-              // permits across shifts via a hash of the permit id so
+              // (overnight) is rare in commercial construction but real
-              // every shift bucket has real, stable data instead of
+              // for maintenance / emergency calls.
              // every contract collapsing to 1st.
              const descLower = ((p.work_description || "") + " " + (p.work_type || "")).toLowerCase();
-              function hashStr(s: string){
+              const shifts: string[] = ["1st"]; // default day
-                let h=5381;
+              if (/night|overnight|24\s*hr|emergency/.test(descLower)) shifts.push("3rd");
-                for(let i=0;i<s.length;i++) h=((h<<5)+h+s.charCodeAt(i))|0;
+              if (/multi.?shift|round.?the.?clock|double.?shift/.test(descLower)) shifts.push("2nd");
-                return Math.abs(h);
+              if (/weekend|saturday|sunday/.test(descLower)) shifts.push("4th");
              }
              const permitKey = String(p.id || (p.street_number+p.street_name) || p.work_description || "").slice(0,80);
              const hh = hashStr(permitKey);
              const bucket = hh % 100;
              // Realistic split: 50% day, 28% evening, 17% overnight,
              // 5% weekend. Construction skews heavily day-shift.
              let primary: string =
                bucket < 50 ? "1st"
                : bucket < 78 ? "2nd"
                : bucket < 95 ? "3rd"
                : "4th";
              const shifts: string[] = [primary];
              if (/night|overnight|24\s*hr|emergency/.test(descLower) && !shifts.includes("3rd")) shifts.push("3rd");
              if (/multi.?shift|round.?the.?clock|double.?shift/.test(descLower) && !shifts.includes("2nd")) shifts.push("2nd");
              if (/weekend|saturday|sunday/.test(descLower) && !shifts.includes("4th")) shifts.push("4th");
              // Internal calendar: build a 7-day schedule (today ±3
              // days) with a row per (date, shift). This is what the
              // front-end's shift-mix preview filters against — real
              // dates, real workers/bill, real status (past/active/
              // scheduled) tied to the current clock. As permits get
              // ingested with explicit start/end dates the backend
              // can replace this with the stored schedule.
              const SHIFT_HOURS: Record<string, [number, number]> = {
                "1st": [6, 14], "2nd": [14, 22], "3rd": [22, 30], "4th": [0, 24], // 4th = weekend
              };
              function shiftStatus(d: Date, shift: string, ref: Date): "past" | "active" | "scheduled" {
                const refDay = ref.toISOString().slice(0,10);
                const dDay = d.toISOString().slice(0,10);
                if (dDay < refDay) return "past";
                if (dDay > refDay) return "scheduled";
                // Same day — break by hour vs shift window.
                const hr = ref.getHours() + ref.getMinutes()/60;
                const [s,e] = SHIFT_HOURS[shift] || [0,24];
                if (shift === "4th") {
                  // Weekend shift: active if today IS weekend, else scheduled.
                  const isWknd = (ref.getDay()===0 || ref.getDay()===6);
                  return isWknd ? "active" : "scheduled";
                }
                if (shift === "3rd") {
                  // 3rd wraps midnight: active 22:00–06:00.
                  if (hr >= 22 || hr < 6) return "active";
                  return "scheduled";
                }
                if (hr < s) return "scheduled";
                if (hr >= e) return "past";
                return "active";
              }
              const refNow = new Date();
              const schedule: any[] = [];
              for (let off = -3; off <= 3; off++) {
                const d = new Date(refNow.getTime() + off * 86400e3);
                const isWknd = (d.getDay()===0 || d.getDay()===6);
                const dateStr = d.toISOString().slice(0,10);
                for (const sh of shifts) {
                  // Weekend permits use 4th shift only; weekday work
                  // uses its primary shift(s) and skips 4th.
                  if (isWknd && sh !== "4th") continue;
                  if (!isWknd && sh === "4th") continue;
                  // Workers per shift: full count on primary, half on
                  // secondary so the bill demand differs visibly.
                  const isPrimary = (sh === primary);
                  const wForShift = isPrimary ? count : Math.max(1, Math.floor(count/2));
                  schedule.push({
                    date: dateStr,
                    shift: sh,
                    workers_needed: wForShift,
                    bill_rate: contractBillRate,
                    status: shiftStatus(d, sh, refNow),
                  });
                }
              }
              contracts.push({
                permit: {
@@ -2217,7 +1835,6 @@ async function main() {
                  over_bill_pool_margin_at_risk: Math.round(overBillPoolMargin),
                },
                shifts_needed: shifts,
                schedule,
              });
            }
@@ -146,16 +146,15 @@ async function persistOp(op: ObservedOp) {
 // ─── LLM Team escalation (code_review mode) ───
 //
 // When recent failures on a single sig_hash cross a threshold the
-// local-model analysis is probably insufficient. J's 2026-04-24
+// local qwen2.5 analysis is probably insufficient. J's 2026-04-24
 // direction: "the observer would trigger to give more context" —
 // route failure clusters to LLM Team's specialized code_review mode
 // (via /api/run) so richer structured signal lands in the KB for
 // scrum + auditor + playbook memory to consume next pass.
 //
-// Non-destructive: runs in parallel to the existing local diagnose
+// Non-destructive: runs in parallel to the existing qwen2.5 analysis,
-// call (qwen3.5:latest after the 2026-04-30 bump), never replaces
+// never replaces it. Writes to data/_kb/observer_escalations.jsonl
-// it. Writes to data/_kb/observer_escalations.jsonl as a dedicated
+// as a dedicated audit surface.
 // audit surface.
 const LLM_TEAM = process.env.LH_LLM_TEAM_URL ?? "http://localhost:5000";
 const LLM_TEAM_ESCALATIONS = "/home/profit/lakehouse/data/_kb/observer_escalations.jsonl";
@@ -543,7 +542,7 @@ async function analyzeErrors() {
  if (failures.length === 0) return;
  // NEW 2026-04-24: escalate recurring sig_hash clusters to LLM Team
-  // code_review mode. Runs in parallel to the local diagnose call
+  // code_review mode. Runs in parallel to the local qwen2.5 analysis
  // below — non-blocking, richer downstream signal for scrum/auditor.
  maybeEscalate(failures).catch(() => {});
@@ -553,14 +552,13 @@ async function analyzeErrors() {
  // Ask local model to diagnose. Phase 44 migration (2026-04-27):
  // /v1/chat instead of legacy /ai/generate so /v1/usage tracks the
-  // call + Langfuse traces it. 2026-04-30 model bump: qwen2.5 →
+  // call + Langfuse traces it. Same upstream model (qwen2.5 local).
  // qwen3.5:latest to match the small-model-pipeline local-tier default.
  try {
    const resp = await fetch(`${LAKEHOUSE}/v1/chat`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
-        model: "qwen3.5:latest",
+        model: "qwen2.5",
        provider: "ollama",
        messages: [{
          role: "user",
@@ -771,7 +769,7 @@ async function tailOverseerCorrections(): Promise<number> {
    try { row = JSON.parse(line); } catch { continue; }
    const op: ObservedOp = {
      timestamp: row.created_at ?? new Date().toISOString(),
-      endpoint: `overseer:${row.model ?? "claude-opus-4-7"}`,
+      endpoint: `overseer:${row.model ?? "gpt-oss:120b"}`,
      input_summary: `${row.task_class ?? "?"}: ${row.reason ?? "escalation"}`,
      // Correction itself is neither success nor failure — it's a
      // mitigation attempt. We mark success=true so analyzeErrors
@@ -1,92 +0,0 @@
 // Server-side mirror of search.html's ROLE_BANDS regex table.
 // Each band carries a *visual scene* — clothing + immediate backdrop —
 // so ComfyUI produces role-coherent headshots instead of interchangeable
 // studio portraits. The front-end sends the raw role string in the
 // query (?role=Forklift%20Operator); the server resolves it to a band
 // and looks up the scene here.
 export type RoleBand =
  | "warehouse"
  | "production"
  | "trades"
  | "driver"
  | "lead";
 export interface SceneDef {
  band: RoleBand;
  // Free-form clause inserted into the diffusion prompt AFTER
  // "[age]-year-old [race] [gender] [role], ". Should describe what
  // they're wearing and what is immediately behind them. Keep under
  // ~25 words — SDXL Turbo loses focus on longer prompts and starts
  // hallucinating cartoon hands.
  scene: string;
 }
 const RE_BANDS: { re: RegExp; band: RoleBand }[] = [
  { re: /forklift|warehouse|associate|material\s*handler|loader|loading|packag|shipping|logistics|inventory|sanitation|janit/i, band: "warehouse" },
  { re: /production|assembl|quality/i, band: "production" },
  { re: /welder|weld|electric|maint(enance)?\s*tech|cnc|machine\s*op|hvac|plumb|carpenter|mason|tool\s*&\s*die/i, band: "trades" },
  { re: /driver|truck|haul|cdl/i, band: "driver" },
  { re: /line\s*lead|supervisor|foreman|coordinator|lead\b/i, band: "lead" },
 ];
 export function roleBand(role: string): RoleBand {
  const r = (role || "").trim();
  if (!r) return "warehouse";
  for (const b of RE_BANDS) if (b.re.test(r)) return b.band;
  return "warehouse";
 }
 // TODO J — refine these. Each `scene` string lands directly in the
 // diffusion prompt. Tone target: a coordinator glances at the card
 // and recognizes the role from the photo before reading the role pill.
 //
 // Things that work well in SDXL Turbo at 8 steps:
 //   - One concrete clothing item ("high-visibility yellow vest")
 //   - One concrete prop ("hard hat hanging from belt", "tablet in hand")
 //   - One blurred background element ("warehouse pallet aisle behind",
 //     "factory machinery softly out of focus")
 //   - Avoid: text/logos (rendered as scribble), specific brands, hands
 //     holding tools (often distorts), full-body language ("standing",
 //     "leaning") — model is trained on portrait crops.
 //
 // Each scene now bakes "monochrome black and white photography" into
 // the prompt so the model produces native B&W output rather than us
 // applying CSS grayscale post-hoc. SDXL Turbo handles B&W natively
 // with strong tonal range — better than desaturating a color render.
 export const SCENES: Record<RoleBand, SceneDef> = {
  warehouse: {
    band: "warehouse",
    scene: "wearing a high-visibility safety vest over a t-shirt, hard hat visible, blurred warehouse pallet aisle behind, soft natural light, monochrome black and white photography, fine film grain, documentary portrait style",
  },
  production: {
    band: "production",
    scene: "wearing a work shirt with safety glasses on forehead, blurred factory machinery softly out of focus behind, fluorescent overhead lighting, monochrome black and white photography, fine film grain, documentary portrait style",
  },
  trades: {
    band: "trades",
    scene: "wearing a heavy-duty work shirt with rolled sleeves, blurred workshop tool wall behind, focused tungsten lighting, monochrome black and white photography, fine film grain, documentary portrait style",
  },
  driver: {
    band: "driver",
    scene: "wearing a polo shirt, lanyard with ID badge visible, blurred truck cab or loading dock behind, daylight, monochrome black and white photography, fine film grain, documentary portrait style",
  },
  lead: {
    band: "lead",
    scene: "wearing a button-down shirt, tablet held casually at chest level, blurred warehouse floor in soft focus behind, professional lighting, monochrome black and white photography, fine film grain, documentary portrait style",
  },
 };
 // v2 — baked B&W + 1024×1024 render canvas (4× pixels of v1). Larger
 // source means downsampling to a 40px avatar packs more detail per
 // displayed pixel, hiding the diffusion-y micro-textures that read as
 // "AI generated" at small sizes. Server route reads pool from
 // data/headshots_role_pool/{SCENES_VERSION}/... so v1 stays available
 // for rollback / A-B comparison.
 export const SCENES_VERSION = "v2";
 // Default render dimensions used by both the on-demand /headshots/
 // generate/:key route and the offline render_role_pool.py script. v1
 // used 512²; v2 doubles to 1024² (linear 2× = 4× pixels = ~3× GPU
 // time on SDXL Turbo).
 export const FACE_RENDER_DIM = 1024;
@@ -1,178 +0,0 @@
 // TIF (Tax Increment Financing) district point-in-polygon lookup.
 // Given a property's lat/long, returns which Chicago TIF district (if
 // any) contains it. TIF districts are public-subsidy zones — a property
 // inside one is receiving city tax-increment funding for its build.
 // Strong "this project has financial backing" signal for the Project Index.
 //
 // Data: data/_entity_cache/tif_districts.geojson (Chicago Open Data
 // dataset eejr-xtfb, 100 active districts, 3.2MB). Refresh by re-running
 // `curl ... eejr-xtfb.geojson > tif_districts.geojson` — districts
 // change rarely (only when city council approves new ones or repeals).
 //
 // Algorithm: classic ray-casting. For each MultiPolygon's outer ring,
 // count edge crossings of an east-going horizontal ray from the point.
 // Odd crossings = inside. Holes (inner rings) flip the parity. Library-
 // free; correct for arbitrary polygons including the irregular Chicago
 // shapes which often have many small detours.
 import { readFile } from "node:fs/promises";
 import { existsSync } from "node:fs";
 import { join } from "node:path";
 const TIF_GEOJSON = join("/home/profit/lakehouse/data/_entity_cache", "tif_districts.geojson");
 type LngLat = [number, number]; // GeoJSON convention: [longitude, latitude]
 type Ring = LngLat[];
 type Polygon = Ring[]; // outer ring + optional inner rings (holes)
 type MultiPolygon = Polygon[];
 type TifFeature = {
  name: string;
  trim_name?: string;
  ref?: string;
  approval_date?: string;
  expiration?: string;
  type?: string; // T-1xx etc.
  comm_area?: string;
  wards?: string;
  // Bounding box for quick reject
  bbox: { minLon: number; minLat: number; maxLon: number; maxLat: number };
  geometry: MultiPolygon;
 };
 let tifIdx: TifFeature[] | null = null;
 function bboxOfMultiPolygon(mp: MultiPolygon): TifFeature["bbox"] {
  let minLon = Infinity, minLat = Infinity, maxLon = -Infinity, maxLat = -Infinity;
  for (const poly of mp) {
    for (const ring of poly) {
      for (const [lon, lat] of ring) {
        if (lon < minLon) minLon = lon;
        if (lat < minLat) minLat = lat;
        if (lon > maxLon) maxLon = lon;
        if (lat > maxLat) maxLat = lat;
      }
    }
  }
  return { minLon, minLat, maxLon, maxLat };
 }
 async function ensureLoaded(): Promise<TifFeature[]> {
  if (tifIdx) return tifIdx;
  if (!existsSync(TIF_GEOJSON)) {
    tifIdx = [];
    return tifIdx;
  }
  try {
    const raw = JSON.parse(await readFile(TIF_GEOJSON, "utf-8"));
    const out: TifFeature[] = [];
    for (const f of raw.features || []) {
      const geom = f.geometry;
      if (!geom) continue;
      // Normalize Polygon → MultiPolygon for uniform iteration
      let mp: MultiPolygon;
      if (geom.type === "MultiPolygon") {
        mp = geom.coordinates;
      } else if (geom.type === "Polygon") {
        mp = [geom.coordinates];
      } else {
        continue;
      }
      const props = f.properties || {};
      out.push({
        name: props.name || "Unknown TIF",
        trim_name: props.name_trim,
        ref: props.ref,
        approval_date: props.approval_d,
        expiration: props.expiration,
        type: props.type,
        comm_area: props.comm_area,
        wards: props.wards,
        bbox: bboxOfMultiPolygon(mp),
        geometry: mp,
      });
    }
    tifIdx = out;
    return tifIdx;
  } catch (e) {
    console.warn("[tif] load failed:", (e as Error).message);
    tifIdx = [];
    return tifIdx;
  }
 }
 // Ray-casting point-in-polygon (single ring). Returns true if (lon, lat)
 // is strictly inside the ring. Edge cases (point exactly on a vertex)
 // resolve by half-open interval convention; for our use case (Chicago
 // boundary precision is ~1m, sites are point queries) this is fine.
 function pointInRing(lon: number, lat: number, ring: Ring): boolean {
  let inside = false;
  const n = ring.length;
  for (let i = 0, j = n - 1; i < n; j = i++) {
    const [xi, yi] = ring[i];
    const [xj, yj] = ring[j];
    const intersect =
      yi > lat !== yj > lat &&
      lon < ((xj - xi) * (lat - yi)) / (yj - yi + 0) + xi;
    if (intersect) inside = !inside;
  }
  return inside;
 }
 // Polygon = outer ring + holes. Inside outer AND not inside any hole.
 function pointInPolygon(lon: number, lat: number, polygon: Polygon): boolean {
  if (polygon.length === 0) return false;
  if (!pointInRing(lon, lat, polygon[0])) return false;
  for (let i = 1; i < polygon.length; i++) {
    if (pointInRing(lon, lat, polygon[i])) return false;
  }
  return true;
 }
 export type TifMatch = {
  name: string;
  ref?: string;
  approval_date?: string;
  expiration?: string;
  comm_area?: string;
  wards?: string;
 };
 export async function findTifDistrict(
  longitude: number | string | undefined,
  latitude: number | string | undefined,
 ): Promise<TifMatch | null> {
  const lon = typeof longitude === "string" ? parseFloat(longitude) : longitude;
  const lat = typeof latitude === "string" ? parseFloat(latitude) : latitude;
  if (!lon || !lat || isNaN(lon) || isNaN(lat)) return null;
  const idx = await ensureLoaded();
  if (idx.length === 0) return null;
  for (const f of idx) {
    // Bbox reject — cheap O(1) skip for the 99% of districts that
    // can't possibly contain the point.
    const b = f.bbox;
    if (lon < b.minLon || lon > b.maxLon || lat < b.minLat || lat > b.maxLat) continue;
    // Full point-in-polygon for any polygon in this MultiPolygon
    for (const poly of f.geometry) {
      if (pointInPolygon(lon, lat, poly)) {
        return {
          name: f.name,
          ref: f.ref,
          approval_date: f.approval_date,
          expiration: f.expiration,
          comm_area: f.comm_area,
          wards: f.wards,
        };
      }
    }
  }
  return null;
 }
 export async function getTifIndexStats(): Promise<{
  total: number;
  loaded: boolean;
 }> {
  const idx = await ensureLoaded();
  return { total: idx.length, loaded: idx.length > 0 };
 }
@@ -1,6 +1,6 @@
 # Phase 6 — Acceptance Gate Report
-**Run:** 2026-04-27T15:43:37.943Z
+**Run:** 2026-04-27T04:54:32.225Z
 **Fixture:** `tests/fixtures/distillation/acceptance/`
 **Temp root:** `/tmp/distillation_phase6_acceptance`
 **Pipeline run_ids:** `acceptance-run-1-stable` (first) + `acceptance-run-2-stable` (second / hash reproducibility)
@@ -40,13 +40,13 @@
 | 19 | scratchpad/tree-split case: fixture row materialized into evidence | found | found | ✓ |
 | 20 | PRD drift case: fixture row materialized | found | found | ✓ |
 | 21 | hash reproducibility: per-stage output_hash identical across runs | 0 mismatches | all match | ✓ |
-| 22 | hash reproducibility: run_hash identical | 8dfdacee62380ec2... | 8dfdacee62380ec2... | ✓ |
+| 22 | hash reproducibility: run_hash identical | 3ea12b160ee9099a... | 3ea12b160ee9099a... | ✓ |
 ## Hash reproducibility detail
-run 1 run_hash: `8dfdacee62380ec20b7420d8f8bad3c395822da6eb0b41eeecd356e88fe20bf0`
+run 1 run_hash: `3ea12b160ee9099a3c52fe6e7fffd3076de7920d2704d24c789260d63cb1a5a2`
-run 2 run_hash: `8dfdacee62380ec20b7420d8f8bad3c395822da6eb0b41eeecd356e88fe20bf0`
+run 2 run_hash: `3ea12b160ee9099a3c52fe6e7fffd3076de7920d2704d24c789260d63cb1a5a2`
 **Bit-for-bit identical.** Two runs of the entire pipeline on the same fixture with the same `recorded_at` produce the same outputs. Distillation is deterministic.
@@ -1,8 +1,8 @@
 # Phase 8 — Full System Audit Report
-**Run:** 2026-04-27T15:43:38.021Z
+**Run:** 2026-04-27T04:54:32.283Z
-**Git commit:** ca7375ea2b178159a0c61bbf62788a2ffa2390e9
+**Git commit:** 73f242e3e41c2aa36b35fe9de54742b248915cb5
-**Baseline:** 2026-04-27T10:31:44.043Z (d11632a6fae6)
+**Baseline:** 2026-04-27T04:53:45.796Z (5bdd159966e6)
 ## Result: **PASS** ✓
@@ -26,7 +26,7 @@
 | 1 | P0 | recon doc exists | Y | docs/recon/local-distillation-recon.md present | present | ✓ |
 | 2 | P0 | tier-1 source streams present | — | all 4 tier-1 jsonls on disk | all present | ✓ |
 | 3 | P1 | schema validators pass on fixtures | Y | ≥40 tests, 0 fail | 51 pass, 0 fail | ✓ |
-| 4 | P2 | materializer dry-run completes | Y | >=1 row from each tier-1 source | 1139 read · 82 written · 2 skipped | ✓ |
+| 4 | P2 | materializer dry-run completes | Y | >=1 row from each tier-1 source | 1073 read · 16 written · 2 skipped | ✓ |
 | 5 | P2 | tier-1 sources each materialize ≥1 row | — | 4/4: distilled_facts, scrum_reviews, audit_facts, mode_experiments | 1/4 hit (mode_experiments) | ✓ |
 | 6 | P3 | on-disk scored-runs distribution non-empty | Y | >=1 accepted | acc=386 part=132 rej=57 hum=480 | ✓ |
 | 7 | P3 | scored-runs distribution sums positive | — | >0 total | 1055 total | ✓ |
@@ -38,19 +38,19 @@
 | 13 | P5 | latest run (3fa51d66-784c-4c7d-843d-6c48328a608c) has all 5 stage receipts | Y | collect,score,export-rag,export-sft,export-preference | all present | ✓ |
 | 14 | P5 | every stage receipt validates against schema | Y | 0 invalid | 0 invalid | ✓ |
 | 15 | P5 | RunSummary validates | Y | valid | valid | ✓ |
-| 16 | P5 | summary.git_commit is 40-char hex | — | match | 68b6697bcb38... (HEAD: ca7375ea2b17...) | ✓ |
+| 16 | P5 | summary.git_commit is 40-char hex | — | match | 68b6697bcb38... (HEAD: 73f242e3e41c...) | ✓ |
 | 17 | P5 | run_hash is sha256 | Y | /^[0-9a-f]{64}$/ | 2336b96c3638982d... | ✓ |
 | 18 | P6 | acceptance gate passes 22/22 invariants on fixture | Y | PASS — 22/22 | 22/22 (exit=0) | ✓ |
 | 19 | P7 | replay validation passes on 3/3 dry-run sample tasks | Y | 3/3 | 3/3 | ✓ |
 | 20 | P7 | replay retrieval surfaces ≥1 playbook on each task (when corpus present) | — | ≥1 task with retrieval | 3/3 | ✓ |
 | 21 | P7 | escalation loop guard: no path > 2 models | Y | 0 loops | 0 | ✓ |
-| 22 | P7 | replay_runs.jsonl populated by audit run | — | exists with ≥3 rows added | 27 rows total | ✓ |
+| 22 | P7 | replay_runs.jsonl populated by audit run | — | exists with ≥3 rows added | 21 rows total | ✓ |
 ## Drift vs prior baseline
 | Metric | Baseline | Current | Δ% | Flag |
 |---|---|---|---|---|
-| p2_evidence_rows | 25 | 82 | 228% | warn |
+| p2_evidence_rows | 15 | 16 | 7% | ok |
 | p2_evidence_skips | 2 | 2 | 0% | ok |
 | p3_accepted | 386 | 386 | 0% | ok |
 | p3_partial | 132 | 132 | 0% | ok |
@@ -61,7 +61,7 @@
 | p4_pref_pairs | 83 | 83 | 0% | ok |
 | p4_total_quarantined | 1325 | 1325 | 0% | ok |
-**1 metric(s) drifted >20% from baseline.** Investigate before treating outputs as stable.
+All metrics within 20% of baseline — pipeline stable across runs.
 ## System health status
@@ -1,89 +0,0 @@
 # Lance backend re-benchmark — 10M vectors (scale_test_10m)
 **Date:** 2026-05-02
 **Dataset:** `data/lance/scale_test_10m` (33 GB, ~10M vectors, 768d)
 **Driver:** live HTTP gateway `:3100/vectors/lance/*` (post sanitizer-fix binary)
 **Method tag on every search response:** `lance_ivf_pq` (confirms IVF_PQ, not brute-force)
 ADR-019 deferred a 10M re-bench: *"at 10M we expect Lance to pull ahead because HNSW doesn't fit in RAM. Re-benchmark when we have a 10M-vector corpus to test against."* The corpus exists; this is that benchmark.
 ## Search latency, 10 diverse queries, top_k=10 (cold)
 | Query | Latency |
 |---|---:|
 | warehouse forklift operator second shift | 50.5ms |
 | senior software engineer kubernetes | 52.9ms |
 | registered nurse pediatric | 37.6ms |
 | welder TIG aluminum | **127.7ms** |
 | data scientist python | 41.6ms |
 | electrician journeyman commercial | 31.4ms |
 | accountant CPA tax | 28.6ms |
 | machine learning research | 32.1ms |
 | construction site supervisor | 31.8ms |
 | biomedical engineer | 25.0ms |
 Median ~32ms, mean ~46ms, one ~128ms outlier (TIG aluminum query — not investigated; could be query-specific IVF traversal pattern or transient I/O).
 ## Search latency, repeated query (warm cache)
 Same query (`forklift operator`) hit 5 times in a row:
 | Call | Latency |
 |---|---:|
 | 1 | 21.9ms |
 | 2 | 20.2ms |
 | 3 | 19.2ms |
 | 4 | 22.4ms |
 | 5 | 18.6ms |
 **Warm-cache p50 ~20ms.** Stable across the 5 trials.
 ## Doc-fetch by id, 5 calls (post-warmup)
 Fetched the same doc_id (`VEC-2196862`) repeatedly:
 | Call | Latency |
 |---|---:|
 | 1 | 68.2ms |
 | 2 | 89.3ms |
 | 3 | 153.9ms |
 | 4 | 126.5ms |
 | 5 | 140.7ms |
 **~100ms p50, climbing under repeat.** This is **substantially slower than the 100K-corpus number** from ADR-019 (311μs claimed; ~6ms measured today on 500k). The 100ms-class result on 10M suggests one of:
 1. The scalar btree index on `doc_id` isn't built on this dataset (possible — no `build_scalar_index` call recorded for it)
 2. 33GB doesn't fit warm; disk I/O dominates
 3. Handler-level HTTP/JSON serialization overhead is amortized at small dataset sizes but visible at 10M
 This is the headline finding of the bench — search is fine at 10M, but **point lookups (the load-bearing Lance feature per ADR-019) need investigation**. The fix is likely "ensure scalar index is built on doc_id at activation time," but I haven't run that experiment.
 ## Compared to ADR-019 100K projections
 | Op | 100K (ADR-019) | 10M (today) | Notes |
 |---|---:|---:|---|
 | Search (cold) | 2229μs | ~46ms | 21x slower at 100x scale → reasonable for IVF_PQ |
 | Search (warm) | (not measured) | ~20ms | Warm cache converges nicely |
 | Doc fetch | 311μs | ~100ms | **300x slower** — likely scalar-index gap |
 | Index method | lance_ivf_pq | lance_ivf_pq | confirmed via response tag |
 ## What this means
 ADR-019's claim that "at 10M, Lance pulls ahead because HNSW doesn't fit in RAM" remains **unverified-but-not-refuted**. We can't directly compare to HNSW at 10M because HNSW's RAM footprint at 10M × 768d × 4 bytes = ~30 GB just for vectors, double that for the graph — way past any single-node deployment. So Lance "wins" at 10M by being the only contender that operationally exists.
 What the bench DID surface:
 - **Search at 10M works at production-shape latency** (~20ms warm). Acceptable for batch / async / non-conversational workloads. Too slow for sub-10ms voice or recommendation paths.
 - **Doc-fetch at 10M is slow** (~100ms). The structural Lance win cited in ADR-019 (random-access in O(1)) is a scalar-index dependency. Worth a follow-up: either confirm the index is built on this dataset and live with 100ms, or rebuild the scalar index and re-bench.
 - **Sanitizer fix held under load** — no 500-with-leak surfaced even on rare query pattern (TIG aluminum). The fix is robust to long-tail queries.
 ## Repro
 ```bash
 # Search latency, single query
 curl -sS -X POST http://127.0.0.1:3100/vectors/lance/search/scale_test_10m \
  -H 'Content-Type: application/json' \
  -d '{"query":"forklift operator","top_k":10}' | jq '.latency_us'
 # Doc fetch by id
 curl -sS http://127.0.0.1:3100/vectors/lance/doc/scale_test_10m/VEC-2196862 \
  | jq '.latency_us'
 ```
@@ -1,92 +0,0 @@
 #!/usr/bin/env bash
 # lance smoke — gates the 5 /vectors/lance/* HTTP routes (search, doc,
 # index, append, migrate). Only the read paths are exercised here so a
 # CI run doesn't mutate state. Migrate + index + append have shape
 # probes (request bodies are well-formed) but ride the not-found path
 # that the 2026-05-02 audit added.
 #
 # Targets the live gateway at $LH_GATEWAY (default :3100). Uses an
 # existing on-disk Lance dataset — `workers_500k_v1` — so no
 # migration setup is needed. If the dataset is missing the smoke
 # fails loudly with a clear message.
 #
 # Surfaced 2026-05-02: the lance crates had zero tests + no smoke;
 # substrate change to lance_backend.rs would silently break the live
 # surface. This smoke is the regression gate.
 #
 # Usage:
 #   ./scripts/lance_smoke.sh
 #   LH_GATEWAY=http://127.0.0.1:3100 ./scripts/lance_smoke.sh
 set -euo pipefail
 GATEWAY="${LH_GATEWAY:-http://127.0.0.1:3100}"
 DATASET="${LH_LANCE_DATASET:-workers_500k_v1}"
 PREFIX="$GATEWAY/vectors/lance"
 PASS=0; FAIL=0
 PROBE() { local label="$1"; shift; "$@" && { echo "  ✓ $label"; PASS=$((PASS+1)); } || { echo "  ✗ $label"; FAIL=$((FAIL+1)); }; }
 echo "[lance-smoke] gateway=$GATEWAY dataset=$DATASET"
 # ── 0. Gateway alive ─────────────────────────────────────────────
 PROBE "gateway /v1/health responds" \
  bash -c "curl -sf -m 3 $GATEWAY/v1/health -o /dev/null"
 # ── 1. Search returns IVF_PQ results on existing dataset ────────
 RESP=$(curl -sS -m 30 -X POST "$PREFIX/search/$DATASET" \
  -H 'Content-Type: application/json' \
  -d '{"query":"forklift operator","top_k":3}' 2>/dev/null || echo '{}')
 PROBE "search/$DATASET returns top-3 lance_ivf_pq results" \
  bash -c "echo '$RESP' | jq -e '.method == \"lance_ivf_pq\" and (.results | length) == 3' >/dev/null"
 # Capture one doc_id from those results so the next probe has something real to fetch.
 DOC_ID=$(echo "$RESP" | jq -r '.results[0].doc_id // ""')
 # ── 2. get_doc by id returns the row ────────────────────────────
 PROBE "doc/$DATASET/<known-id> returns full row" \
  bash -c "[ -n '$DOC_ID' ] && curl -sf -m 5 '$PREFIX/doc/$DATASET/$DOC_ID' | jq -e '.row.doc_id == \"$DOC_ID\"' >/dev/null"
 # ── 3. get_doc with bogus id returns 404 (not 500) ──────────────
 STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_404.json -w '%{http_code}' \
  "$PREFIX/doc/$DATASET/W500K-NOT-A-REAL-ID-00000")
 PROBE "doc/$DATASET/<missing-id> → 404" \
  test "$STATUS" = "404"
 # ── 4. search on missing dataset returns 404 + sanitized message ─
 STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_500.json -w '%{http_code}' \
  -X POST "$PREFIX/search/no-such-dataset-${RANDOM}" \
  -H 'Content-Type: application/json' \
  -d '{"query":"x","top_k":1}')
 BODY=$(cat /tmp/lance_smoke_500.json)
 PROBE "search/<missing> → 404 (was 500 pre-2026-05-02)" \
  test "$STATUS" = "404"
 # The sanitizer fix specifically: no /home/ or /root/.cargo/ in body.
 PROBE "search/<missing> body sanitized — no filesystem leak" \
  bash -c "echo '$BODY' | grep -qvE '/home/|/root/\.cargo/'"
 # ── 5. build_index on missing dataset also sanitized ────────────
 STATUS=$(curl -sS -m 5 -o /tmp/lance_smoke_idx.json -w '%{http_code}' \
  -X POST "$PREFIX/index/no-such-dataset-${RANDOM}" \
  -H 'Content-Type: application/json' \
  -d '{}')
 BODY=$(cat /tmp/lance_smoke_idx.json)
 PROBE "index/<missing> body sanitized" \
  bash -c "echo '$BODY' | grep -qvE '/home/|/root/\.cargo/'"
 # ── 6. append validates input shape (rejects empty rows array) ──
 STATUS=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' \
  -X POST "$PREFIX/append/$DATASET" \
  -H 'Content-Type: application/json' \
  -d '{"rows":[]}')
 PROBE "append with empty rows[] → 400" \
  test "$STATUS" = "400"
 # ── 7. migrate route is reachable (POST without body returns a real error, not 404) ──
 STATUS=$(curl -sS -m 5 -o /dev/null -w '%{http_code}' \
  -X POST "$PREFIX/migrate/probe-not-real-${RANDOM}?bucket=primary" 2>/dev/null)
 # Should be 4xx (bad request shape), NOT 404 (route registered) and NOT 200.
 PROBE "migrate route registered (non-404, non-200 on empty body)" \
  bash -c "[ '$STATUS' != '404' ] && [ '$STATUS' != '200' ]"
 echo "[lance-smoke] $PASS PASS / $FAIL FAIL"
 [ "$FAIL" -eq 0 ]
@@ -29,14 +29,8 @@ CACHE_DIR.mkdir(parents=True, exist_ok=True)
 WORKFLOW_PATH = "/opt/ComfyUI/workflows/editorial_hero.json"
-def _cache_key(prompt, width, height, steps, seed=None):
+def _cache_key(prompt, width, height, steps):
-    # Include seed so callers can vary outputs deterministically without
+    return hashlib.sha256(f"{prompt}|{width}|{height}|{steps}".encode()).hexdigest()[:24]
    # the proxy collapsing to a single cached image. None == legacy
    # (omitted from the key for backward compatibility).
    bits = f"{prompt}|{width}|{height}|{steps}"
    if seed is not None:
        bits += f"|{seed}"
    return hashlib.sha256(bits.encode()).hexdigest()[:24]
 def _cache_get(key):
    fp = CACHE_DIR / f"{key}.webp"
@@ -46,15 +40,8 @@ def _cache_put(key, img_bytes):
    (CACHE_DIR / f"{key}.webp").write_bytes(img_bytes)
-def _comfyui_generate(prompt, width=1024, height=512, steps=8, seed=None,
+def _comfyui_generate(prompt, width=1024, height=512, steps=8, seed=None):
-                      negative_prompt=None, cfg=None, sampler=None, scheduler=None):
+    """Submit workflow to ComfyUI and wait for result."""
    """Submit workflow to ComfyUI and wait for result.
    Optional overrides — when provided, replace the workflow's defaults.
    The workflow template at editorial_hero.json was tuned for product
    hero shots with a "no humans" negative prompt; portrait callers MUST
    pass `negative_prompt` to avoid the model fighting them on faces.
    """
    # Load workflow template
    with open(WORKFLOW_PATH) as f:
        workflow = json.load(f)
@@ -64,21 +51,9 @@ def _comfyui_generate(prompt, width=1024, height=512, steps=8, seed=None,
        seed = random.randint(0, 2**32)
    workflow["3"]["inputs"]["seed"] = seed
    workflow["3"]["inputs"]["steps"] = steps
    if cfg is not None:
        workflow["3"]["inputs"]["cfg"] = cfg
    if sampler:
        workflow["3"]["inputs"]["sampler_name"] = sampler
    if scheduler:
        workflow["3"]["inputs"]["scheduler"] = scheduler
    workflow["5"]["inputs"]["width"] = width
    workflow["5"]["inputs"]["height"] = height
    workflow["6"]["inputs"]["text"] = prompt
    # Node 7 is the negative-prompt CLIPTextEncode. The default is tuned
    # for product hero shots and contains "human, person, face, hand,
    # fingers, realistic photo of people" — actively sabotaging any
    # portrait render. Always overwrite when negative_prompt is given.
    if negative_prompt is not None:
        workflow["7"]["inputs"]["text"] = negative_prompt
    # Submit to ComfyUI
    payload = json.dumps({"prompt": workflow}).encode()
@@ -202,20 +177,9 @@ class ImageHandler(BaseHTTPRequestHandler):
        height = min(max(int(body.get("height", 720)), 256), 1080)
        steps = min(max(int(body.get("steps", 50)), 1), 80)
        seed = body.get("seed")
        # Portrait-friendly overrides — None means "use workflow default".
        # negative_prompt MUST be passed by portrait callers to avoid
        # the workflow's "no humans" baked-in negative.
        negative_prompt = body.get("negative_prompt")
        cfg = body.get("cfg")
        sampler = body.get("sampler")
        scheduler = body.get("scheduler")
-        # Cache check — seed + negative + cfg are part of the key so per-
+        # Cache check
-        # worker / per-config requests don't collapse to one cached image.
+        key = _cache_key(prompt, width, height, steps)
        key = _cache_key(
            f"{prompt}||neg={negative_prompt or ''}||cfg={cfg or ''}",
            width, height, steps, seed,
        )
        cached = _cache_get(key)
        if cached:
            self._json(200, {"image": cached, "format": "webp", "width": width, "height": height,
@@ -228,11 +192,7 @@ class ImageHandler(BaseHTTPRequestHandler):
        try:
            comfy_check = urllib.request.urlopen(f"{COMFYUI_URL}/system_stats", timeout=3)
            if comfy_check.status == 200:
-                img_bytes, seed = _comfyui_generate(
+                img_bytes, seed = _comfyui_generate(prompt, width, height, steps, seed)
                    prompt, width, height, steps, seed,
                    negative_prompt=negative_prompt, cfg=cfg,
                    sampler=sampler, scheduler=scheduler,
                )
                backend = "comfyui"
        except:
            pass
@@ -250,11 +210,6 @@ class ImageHandler(BaseHTTPRequestHandler):
        elapsed_ms = int((time.time() - t0) * 1000)
        img_b64 = base64.b64encode(img_bytes).decode()
        # Recompute key with the actual seed used (when caller passed
        # None, _comfyui_generate picks a random one and we want the
        # cache to reflect that so re-requests with the same returned
        # seed hit the disk).
        key = _cache_key(prompt, width, height, steps, seed)
        _cache_put(key, img_bytes)
        self._json(200, {
@@ -53,30 +53,20 @@ def fetch_one(idx: int, out_dir: str) -> tuple[int, str, bool, str | None]:
 def maybe_tag_gender(records: list[dict], out_dir: str) -> dict[str, int]:
-    """If deepface is installed, label records that don't already have a
+    """If deepface is installed, label each record with gender. Returns
-    gender. Returns a count summary; mutates records in place.
+    a count summary; mutates records in place. On import error, returns
-
+    None and tags every record as unknown."""
    Preservation contract: never overwrites prior `gender` (or any other
    tag — race/age/excluded — set by tag_face_pool.py). On deepface
    import failure, leaves existing tags alone instead of resetting them
    to None. The previous behavior wiped 952 hand-classified rows when
    fetch_face_pool was re-run from a Python without deepface installed."""
    try:
        from deepface import DeepFace  # type: ignore
    except Exception as e:
-        print(f"  (deepface unavailable: {e}) — leaving existing tags untouched")
+        print(f"  (deepface unavailable: {e}) — pool will mix naturally")
        for r in records:
-            r.setdefault("gender", None)
+            r["gender"] = None
-        already = sum(1 for r in records if r.get("gender") in ("man", "woman"))
+        return {"unknown": len(records)}
        return {"preserved_tagged": already, "untagged": len(records) - already}
-    todo = [r for r in records if r.get("gender") not in ("man", "woman")]
+    print("  tagging gender via deepface (CPU; ~0.5-1s per face)…")
    if not todo:
        print("  every record already has gender — nothing to tag.")
        return {"preserved_tagged": len(records)}
    print(f"  tagging gender via deepface ({len(todo)} of {len(records)} records, CPU; ~0.5-1s per face)…")
    counts: dict[str, int] = {}
-    for i, r in enumerate(todo):
+    for i, r in enumerate(records):
        full = os.path.join(out_dir, r["file"])
        try:
            ana = DeepFace.analyze(
@@ -98,7 +88,7 @@ def maybe_tag_gender(records: list[dict], out_dir: str) -> dict[str, int]:
            r["gender_error"] = f"{type(e).__name__}: {e}"
        counts[r["gender"] or "unknown"] = counts.get(r["gender"] or "unknown", 0) + 1
        if (i + 1) % 25 == 0:
-            print(f"    [{i+1}/{len(todo)}] {counts}")
+            print(f"    [{i+1}/{len(records)}] {counts}")
    return counts
@@ -111,47 +101,11 @@ def main():
    )
    p.add_argument("--concurrency", type=int, default=3, help="parallel fetches (be polite)")
    p.add_argument("--no-gender", action="store_true", help="skip deepface gender tagging")
    p.add_argument("--shrink", action="store_true",
                   help="allow --count to drop manifest entries with id >= count. Default: preserve them.")
    args = p.parse_args()
    out = os.path.realpath(args.out)
    os.makedirs(out, exist_ok=True)
    # Load any existing manifest into a by-id dict so prior tags
    # (gender / race / age / excluded) survive the rewrite. Also
    # naturally dedupes — if the file accidentally has duplicate
    # lines for the same id (this is how we ended up with a 2497-
    # row manifest backing a 1000-face pool), the last one wins.
    manifest = os.path.join(out, "manifest.jsonl")
    existing: dict[int, dict] = {}
    if os.path.exists(manifest):
        dup_count = 0
        with open(manifest) as f:
            for line in f:
                line = line.strip()
                if not line:
                    continue
                try:
                    row = json.loads(line)
                except json.JSONDecodeError:
                    continue
                rid = row.get("id")
                if not isinstance(rid, int):
                    continue
                if rid in existing:
                    dup_count += 1
                existing[rid] = row
        print(f"Loaded existing manifest: {len(existing)} unique ids ({dup_count} duplicate lines collapsed)")
        max_existing = max(existing.keys()) if existing else -1
        if max_existing >= args.count and not args.shrink:
            print(
                f"\nERROR: --count={args.count} would drop {sum(1 for k in existing if k >= args.count)} "
                f"manifest entries (max existing id = {max_existing}). Pass --shrink to allow.\n",
                file=sys.stderr,
            )
            sys.exit(2)
    print(f"Fetching {args.count} faces → {out}")
    print(f"Source: {URL} (synthetic StyleGAN — no real people)")
@@ -161,16 +115,12 @@ def main():
        futs = {ex.submit(fetch_one, i, out): i for i in range(args.count)}
        for done, fut in enumerate(as_completed(futs), 1):
            idx, fname, cached, err = fut.result()
-            # Start from prior manifest row (preserves gender/race/age/excluded)
+            results[idx] = {
            # and overlay only the fields fetch_one is responsible for.
            base = dict(existing.get(idx, {}))
            base.update({
                "id": idx,
                "file": fname,
                "cached": cached,
                "error": err,
-            })
+            }
            results[idx] = base
            if done % 25 == 0 or done == args.count:
                ok = sum(1 for r in results if r and not r.get("error"))
                print(f"  [{done}/{args.count}] {ok} ok  ({time.time()-t0:.1f}s)")
@@ -178,9 +128,6 @@ def main():
    # Drop slots that errored or are still None (shouldn't happen)
    records = [r for r in results if r and not r.get("error")]
    print(f"\nPool ready: {len(records)} faces, {sum(1 for r in records if r['cached'])} from cache")
    preserved_tags = sum(1 for r in records if r.get("gender") in ("man", "woman"))
    if preserved_tags:
        print(f"Preserved {preserved_tags} prior gender tags (and any race/age/excluded fields).")
    if not args.no_gender and records:
        print("\nGender-tagging pass:")
@@ -188,28 +135,17 @@ def main():
        print(f"  distribution: {summary}")
    else:
        for r in records:
-            r.setdefault("gender", None)
+            r["gender"] = None
    # If --shrink was NOT used and somehow id >= count rows are still in
    # `existing` (which can only happen if the early gate was bypassed),
    # carry them forward so we don't quietly drop them.
    if not args.shrink:
        for rid, row in existing.items():
            if rid >= args.count and rid not in {r["id"] for r in records}:
                records.append(row)
        records.sort(key=lambda r: r.get("id", 0))
    # Strip transient flags before persisting
    for r in records:
        r.pop("cached", None)
        r.pop("error", None)
-    # Atomic write — if a re-run is interrupted, manifest stays intact.
+    manifest = os.path.join(out, "manifest.jsonl")
-    tmp = manifest + ".tmp"
+    with open(manifest, "w") as f:
    with open(tmp, "w") as f:
        for r in records:
            f.write(json.dumps(r) + "\n")
    os.replace(tmp, manifest)
    print(f"\nManifest: {manifest}  ({len(records)} entries)")
    # Quick checksum manifest for downstream debugging
@@ -1,230 +0,0 @@
 #!/usr/bin/env python3
 """
 render_role_pool.py — pre-render a role-aware face pool by hitting
 serve_imagegen.py (localhost:3600/generate) with prompts pulled from
 the bun server's /headshots/_scenes endpoint (single source of truth
 for SCENES + SCENES_VERSION).
 Layout:
    data/headshots_role_pool/
      {band}/
        {gender}_{race}/
          face_00.webp
          face_01.webp
          ...
      manifest.jsonl
 Each entry in manifest.jsonl:
    {"band": "warehouse", "gender": "man", "race": "caucasian",
     "file": "warehouse/man_caucasian/face_03.webp",
     "seed": 184729338, "scenes_version": "v1"}
 Idempotent: a file at the target path is skipped. Re-run with --force
 to regenerate. SCENES_VERSION is captured per render so the server's
 pool route can refuse stale renders if the version drifts.
 """
 from __future__ import annotations
 import argparse
 import base64
 import json
 import os
 import sys
 import time
 import urllib.request
 import urllib.error
 DEFAULT_BANDS = ["warehouse", "production", "trades", "driver", "lead"]
 DEFAULT_GENDERS = ["man", "woman"]
 DEFAULT_RACES = ["caucasian", "east_asian", "south_asian", "middle_eastern", "black", "hispanic"]
 def race_text(r: str) -> str:
    return {
        "caucasian": "",
        "east_asian": "East Asian",
        "south_asian": "South Asian",
        "middle_eastern": "Middle Eastern",
        "black": "Black",
        "hispanic": "Hispanic",
    }.get(r, "")
 def fetch_scenes(mcp_url: str) -> tuple[str, dict]:
    """Pull canonical SCENES from the bun server. Single source of truth."""
    req = urllib.request.Request(f"{mcp_url}/headshots/_scenes")
    with urllib.request.urlopen(req, timeout=10) as resp:
        data = json.loads(resp.read())
    return data["version"], data["scenes"]
 def render(comfy_url: str, prompt: str, seed: int, steps: int, timeout: int, dim: int) -> bytes | None:
    payload = json.dumps({
        "prompt": prompt,
        "width": dim,
        "height": dim,
        "steps": steps,
        "seed": seed,
    }).encode()
    req = urllib.request.Request(
        f"{comfy_url}/generate",
        data=payload,
        headers={"Content-Type": "application/json"},
    )
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            data = json.loads(resp.read())
    except urllib.error.HTTPError as e:
        print(f"  HTTP {e.code} from comfy: {e.read()[:200]}", file=sys.stderr)
        return None
    except Exception as e:
        print(f"  comfy error: {type(e).__name__}: {e}", file=sys.stderr)
        return None
    img_b64 = data.get("image")
    if not img_b64:
        print(f"  comfy response missing 'image' field: {list(data.keys())}", file=sys.stderr)
        return None
    return base64.b64decode(img_b64)
 def main():
    p = argparse.ArgumentParser()
    p.add_argument("--out", default=os.path.join(os.path.dirname(__file__), "..", "..", "data", "headshots_role_pool"))
    p.add_argument("--per-bucket", type=int, default=10, help="how many faces per (band × gender × race)")
    p.add_argument("--mcp", default="http://localhost:3700")
    p.add_argument("--comfy", default="http://localhost:3600")
    p.add_argument("--steps", type=int, default=8)
    p.add_argument("--bands", nargs="*", default=DEFAULT_BANDS)
    p.add_argument("--genders", nargs="*", default=DEFAULT_GENDERS)
    p.add_argument("--races", nargs="*", default=DEFAULT_RACES)
    p.add_argument("--force", action="store_true", help="regenerate existing files")
    p.add_argument("--age", type=int, default=32)
    p.add_argument("--timeout", type=int, default=120, help="per-render timeout (1024² takes ~5s on A4000)")
    p.add_argument("--dim", type=int, default=1024, help="square render dimension (v2 default 1024, v1 was 512)")
    args = p.parse_args()
    out_root = os.path.realpath(args.out)
    os.makedirs(out_root, exist_ok=True)
    print(f"Fetching canonical SCENES from {args.mcp}/headshots/_scenes…")
    try:
        version, scenes = fetch_scenes(args.mcp)
    except Exception as e:
        print(f"FATAL: could not fetch scenes ({e}). Is the mcp-server up?", file=sys.stderr)
        sys.exit(1)
    print(f"  SCENES_VERSION={version}, {len(scenes)} bands available: {list(scenes.keys())}")
    # v2+ files live at {out}/{version}/{band}/{g}_{r}/face_NN.webp.
    # v1 lived at {out}/{band}/... — keep that layout intact for
    # rollback; the server route reads both and prefers current.
    out = out_root if version == "v1" else os.path.join(out_root, version)
    os.makedirs(out, exist_ok=True)
    print(f"  writing to: {out}")
    print(f"  render dim: {args.dim}×{args.dim}")
    # Reject any --bands not in the server's SCENES
    unknown = [b for b in args.bands if b not in scenes]
    if unknown:
        print(f"FATAL: unknown bands {unknown}. Server has: {list(scenes.keys())}", file=sys.stderr)
        sys.exit(1)
    manifest_rows = []
    todo = [
        (band, g, r, n)
        for band in args.bands
        for g in args.genders
        for r in args.races
        for n in range(args.per_bucket)
    ]
    print(f"\nPlanning: {len(todo)} renders ({len(args.bands)} bands × {len(args.genders)} genders × {len(args.races)} races × {args.per_bucket} faces).")
    print(f"Estimated GPU time at 1.5s/render = {len(todo) * 1.5 / 60:.1f} min.\n")
    t0 = time.time()
    rendered = 0
    skipped = 0
    failed = 0
    for i, (band, g, r, n) in enumerate(todo):
        bucket_dir = os.path.join(out, band, f"{g}_{r}")
        os.makedirs(bucket_dir, exist_ok=True)
        fname = f"face_{n:02d}.webp"
        full = os.path.join(bucket_dir, fname)
        rel = os.path.relpath(full, out)
        if os.path.exists(full) and os.path.getsize(full) > 1024 and not args.force:
            skipped += 1
            manifest_rows.append({
                "band": band, "gender": g, "race": r, "file": rel,
                "seed": None, "scenes_version": version, "cached": True,
            })
            continue
        scene_def = scenes[band]
        scene_clause = scene_def["scene"]
        race_clause = race_text(r)
        gender_clause = g  # "man" / "woman"
        # Match the bun server's prompt builder exactly. If you tweak
        # one, tweak the other (or factor a /prompt-builder endpoint).
        # The {role} slot is intentionally a band-typical title here
        # — the pre-rendered face is shared across roles in the same
        # band, so we use the band's archetypal role. Specific roles
        # still hit the on-demand /headshots/generate/:key path with
        # their actual title.
        archetype_role = {
            "warehouse": "warehouse worker",
            "production": "production worker",
            "trades": "skilled tradesperson",
            "driver": "delivery driver",
            "lead": "shift supervisor",
        }.get(band, "warehouse worker")
        prompt = (
            f"professional headshot portrait of a {args.age}-year-old "
            f"{race_clause} {gender_clause} {archetype_role}, {scene_clause}, "
            f"neutral confident expression, sharp focus, photorealistic"
        )
        # Deterministic seed per slot — same (band, g, r, n) always
        # gets the same face. Mixing scenes_version means a SCENES
        # tweak shifts every face slightly; that's the right behavior
        # (it's how cache invalidation propagates to the pool too).
        seed_str = f"{band}|{g}|{r}|{n}|{version}"
        seed_h = 5381
        for ch in seed_str:
            seed_h = ((seed_h << 5) + seed_h + ord(ch)) & 0x7fffffff
        seed = seed_h
        bytes_ = render(args.comfy, prompt, seed, args.steps, args.timeout, args.dim)
        if bytes_ is None:
            failed += 1
            continue
        with open(full, "wb") as f:
            f.write(bytes_)
        rendered += 1
        manifest_rows.append({
            "band": band, "gender": g, "race": r, "file": rel,
            "seed": seed, "scenes_version": version, "cached": False,
        })
        if (i + 1) % 10 == 0 or (i + 1) == len(todo):
            elapsed = time.time() - t0
            done = i + 1
            rate = done / elapsed if elapsed > 0 else 0
            eta = (len(todo) - done) / rate if rate > 0 else 0
            print(f"  [{done}/{len(todo)}]  rendered={rendered} skipped={skipped} failed={failed}  "
                  f"rate={rate:.2f}/s  eta={eta:.0f}s")
    # Atomic manifest write
    manifest_path = os.path.join(out, "manifest.jsonl")
    tmp = manifest_path + ".tmp"
    with open(tmp, "w") as f:
        for row in manifest_rows:
            f.write(json.dumps(row) + "\n")
    os.replace(tmp, manifest_path)
    print(f"\nDone. {rendered} new, {skipped} cached, {failed} failed in {time.time()-t0:.1f}s")
    print(f"Manifest: {manifest_path} ({len(manifest_rows)} entries)")
    print(f"\nNext: poke {args.mcp}/headshots/__reload to pick up the new pool.")
 if __name__ == "__main__":
    main()
@@ -1,169 +0,0 @@
 #!/usr/bin/env python3
 """
 tag_face_pool.py — run deepface gender + race classification over the
 synthetic face pool produced by fetch_face_pool.py and rewrite
 manifest.jsonl with `gender` (man / woman) and `race` (asian / black /
 hispanic / indian / middle_eastern / white) tags.
 Run with the venv that has deepface installed:
    /home/profit/.local/share/deepface-venv/bin/python \
        scripts/staffing/tag_face_pool.py
 Idempotent: rows that already have BOTH gender and race tagged are
 skipped. Pass --force to re-tag everything.
 Mapping deepface buckets → /headshots/ ?e= values:
  asian        → split by manual region (deepface doesn't differentiate
                  East / South Asian; we lump as 'east_asian' since the
                  StyleGAN training set leans East Asian)
  indian       → south_asian
  middle eastern → middle_eastern
  black        → black
  hispanic     → hispanic
  white        → caucasian
 """
 from __future__ import annotations
 import argparse
 import json
 import os
 import sys
 import time
 DEEPFACE_RACE_TO_HINT = {
    "asian": "east_asian",
    "indian": "south_asian",
    "middle eastern": "middle_eastern",
    "black": "black",
    "latino hispanic": "hispanic",
    "hispanic": "hispanic",
    "white": "caucasian",
 }
 def main():
    p = argparse.ArgumentParser()
    p.add_argument(
        "--out",
        default=os.path.join(os.path.dirname(__file__), "..", "..", "data", "headshots"),
    )
    p.add_argument("--force", action="store_true", help="re-tag rows that already have gender+race")
    p.add_argument("--limit", type=int, default=0, help="cap how many faces to process this run (0 = all)")
    p.add_argument("--min-age", type=int, default=22, help="exclude faces estimated below this age (kids/teens). Staffing context = legal-age workers only.")
    args = p.parse_args()
    out = os.path.realpath(args.out)
    manifest_path = os.path.join(out, "manifest.jsonl")
    if not os.path.exists(manifest_path):
        print(f"manifest not found: {manifest_path}", file=sys.stderr)
        sys.exit(1)
    print(f"loading deepface (cold start ~10-15s for first model build)…")
    from deepface import DeepFace  # type: ignore
    rows = []
    with open(manifest_path) as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            rows.append(json.loads(line))
    print(f"manifest: {len(rows)} rows")
    todo = [
        r for r in rows
        if args.force or r.get("gender") is None or r.get("race") is None or r.get("age") is None
    ]
    if args.limit > 0:
        todo = todo[: args.limit]
    print(f"to tag: {len(todo)} faces")
    if not todo:
        print("nothing to do.")
        return
    counts_g = {}
    counts_r = {}
    failed = 0
    t0 = time.time()
    for i, r in enumerate(todo):
        full = os.path.join(out, r["file"])
        try:
            ana = DeepFace.analyze(
                img_path=full,
                actions=["gender", "race", "age"],
                enforce_detection=False,
                silent=True,
            )
            if isinstance(ana, list):
                ana = ana[0] if ana else {}
            g_raw = (ana.get("dominant_gender") or "").lower().strip()
            r["gender"] = (
                "man" if g_raw.startswith("man") else
                "woman" if g_raw.startswith("woman") else
                None
            )
            r_raw = (ana.get("dominant_race") or "").lower().strip()
            r["race"] = DEEPFACE_RACE_TO_HINT.get(r_raw, None)
            if r["race"] is None and r_raw:
                r["race_raw"] = r_raw
            # Age estimation — exclude minors / teens. Staffing context
            # uses adult workers only. Threshold is 22 by default
            # (legal + a buffer because age estimation is noisy).
            try:
                age = int(round(float(ana.get("age") or 0)))
            except Exception:
                age = 0
            r["age"] = age
            if age and age < args.min_age:
                r["excluded"] = "minor"
            else:
                r.pop("excluded", None)
            counts_g[r["gender"] or "unknown"] = counts_g.get(r["gender"] or "unknown", 0) + 1
            counts_r[r["race"] or r_raw or "unknown"] = counts_r.get(r["race"] or r_raw or "unknown", 0) + 1
        except Exception as e:
            r["tag_error"] = f"{type(e).__name__}: {e}"
            failed += 1
        if (i + 1) % 25 == 0 or (i + 1) == len(todo):
            elapsed = time.time() - t0
            rate = (i + 1) / elapsed if elapsed > 0 else 0
            eta = (len(todo) - i - 1) / rate if rate > 0 else 0
            print(f"  [{i+1}/{len(todo)}]  rate={rate:.1f}/s  eta={eta:.0f}s  failed={failed}")
            print(f"    gender: {counts_g}")
            print(f"    race  : {counts_r}")
    # Write updated manifest atomically
    tmp = manifest_path + ".tmp"
    with open(tmp, "w") as f:
        for r in rows:
            f.write(json.dumps(r) + "\n")
    os.replace(tmp, manifest_path)
    final_g = {}
    final_r = {}
    excluded = 0
    age_hist = {"<18": 0, "18-22": 0, "22-30": 0, "30-40": 0, "40-50": 0, "50-60": 0, "60+": 0, "unknown": 0}
    for r in rows:
        if r.get("excluded"):
            excluded += 1
            continue
        final_g[r.get("gender") or "untagged"] = final_g.get(r.get("gender") or "untagged", 0) + 1
        final_r[r.get("race") or "untagged"] = final_r.get(r.get("race") or "untagged", 0) + 1
        a = r.get("age") or 0
        if a == 0: age_hist["unknown"] += 1
        elif a < 18: age_hist["<18"] += 1
        elif a < 22: age_hist["18-22"] += 1
        elif a < 30: age_hist["22-30"] += 1
        elif a < 40: age_hist["30-40"] += 1
        elif a < 50: age_hist["40-50"] += 1
        elif a < 60: age_hist["50-60"] += 1
        else: age_hist["60+"] += 1
    print(f"\nDone. {len(rows)} rows, {excluded} excluded as <{args.min_age}, {failed} tag errors, {time.time()-t0:.1f}s")
    print(f"  final gender: {final_g}")
    print(f"  final race  : {final_r}")
    print(f"  age dist   : {age_hist}")
    print(f"\nNext: poke /headshots/__reload to refresh the in-memory pool.")
 if __name__ == "__main__":
    main()
@@ -1143,15 +1143,9 @@ Format each as a code-fenced block with the byte offset within the shard:
 EXACT LINE OF SOURCE — DO NOT PARAPHRASE, DO NOT TRUNCATE
 \`\`\`
 Pick the most reviewer-relevant lines: route definitions (e.g. \`@app.route(...)\`), function signatures, security-sensitive calls (auth/SQL/exec/template/secrets), hardcoded credentials/defaults, exception handlers, sensitive imports. The reviewer will REFUSE to act on any claim not backed by a verbatim anchor — so anchors are how you prove findings are real.`;
    // 2026-04-28: gpt-oss:120b → gemini-3-flash-preview via Ollama
    // Pro. Tree-split MAP fires once per shard (potentially 5-20×
    // per file), so latency dominates total scrum time. Gemini 3
    // flash returns shard digests substantially faster than the old
    // 120B free model while staying strong enough for byte-anchored
    // extraction.
    const r = await chat({
      provider: "ollama_cloud",
-      model: "gemini-3-flash-preview",
+      model: "gpt-oss:120b",
      prompt,
      max_tokens: 900,
    });
@@ -1201,14 +1195,9 @@ COPY EVERY anchor block from the piece notes IN ORDER, character-perfect. DO NOT
 Output the anchor blocks under their original \`\`\`@offset...\`\`\` fences, each on its own with a blank line between. The reviewer rejects findings that don't quote a string from this anchors block, so completeness here directly determines review quality.`;
  // 2026-04-28: gpt-oss:120b → gemini-3-flash-preview via Ollama
  // Pro. The reducer runs once per file (vs once per shard for MAP)
  // but on a much larger context (all shard digests stacked), so
  // throughput per token still matters. Same model as MAP for
  // consistency in tree-split outputs.
  const reduced = await chat({
    provider: "ollama_cloud",
-    model: "gemini-3-flash-preview",
+    model: "gpt-oss:120b",
    prompt: reducePrompt,
    max_tokens: 2400,
  });