Scrum-driven fixes: P5-001 auth wired, P42-001 truth evaluator, P9-001 journal on ingest

Apply the highest-confidence findings from the Phase 0→42 forensic sweep
after four scrum-master iterations under the adversarial prompt. Each fix
is independently validated by a later scrum iteration scoring the same
file higher under the same bar.

Code changes
────────────
P5-001 — crates/gateway/src/auth.rs + main.rs
  api_key_auth was marked #[allow(dead_code)] and never wrapped around
  the router, so `[auth] enabled=true` logged a green message and
  enforced nothing. Now wired via from_fn_with_state, with constant-time
  header compare and /health exempted for LB probes.

P42-001 — crates/truth/src/lib.rs
  TruthStore::check() ignored RuleCondition entirely — signature looked
  like enforcement, body returned every action unconditionally. Added
  evaluate(task_class, ctx) that actually walks FieldEquals / FieldEmpty /
  FieldGreater / Always against a serde_json::Value via dot-path lookup.
  check() kept for back-compat. Tests 14 → 24 (10 new exercising real
  pass/fail semantics). serde_json moved to [dependencies].

P9-001 (partial) — crates/ingestd/src/service.rs
  Added Optional<Journal> to IngestState + a journal.record_ingest() call
  on /ingest/file success. Gateway wires it with `journal.clone()` before
  the /journal nest consumes the original. First-ever internal mutation
  journal event verified live (total_events_created 0→1 after probe).

Iter-4 scrum scored these files higher under same prompt:
  ingestd/src/service.rs      3 → 6  (P9-001 visible)
  truth/src/lib.rs            3 → 4  (P42-001 visible)
  gateway/src/auth.rs         3 → 4  (P5-001 visible)
  gateway/src/execution_loop  4 → 6  (indirect)
  storaged/src/federation     3 → 4  (indirect)

Infrastructure additions
────────────────────────
 * tests/real-world/scrum_master_pipeline.ts
   - cloud-first ladder: kimi-k2:1t → deepseek-v3.1:671b → mistral-large-3:675b
     → gpt-oss:120b → devstral-2:123b → qwen3.5:397b (deep final thinker)
   - LH_SCRUM_FORENSIC env: injects SCRUM_FORENSIC_PROMPT.md as adversarial preamble
   - LH_SCRUM_PROPOSAL env: per-iter fix-wave doc override
   - Confidence extraction (markdown + JSON), schema v4 KB rows with:
     verdict, critical_failures_count, verified_components_count,
     missing_components_count, output_format, gradient_tier
   - Model trust profile written per file-accept to data/_kb/model_trust.jsonl
   - Fire-and-forget POST to observer /event so by_source.scrum appears in /stats

 * mcp-server/observer.ts — unchanged in shape, confirmed receiving scrum events

 * ui/ — new Visual Control Plane on :3950
   - Bun.serve with /data/{services,reviews,metrics,trust,overrides,findings,file,refactor_signals,search,logs/:svc,scrum_log}
   - Views: MAP (D3 graph, 5 overlays) / TRACE (per-file iter timeline) /
     TRAJECTORY (refactor signals + reverse index search) / METRICS (explainers
     with SOURCE + GOOD lines) / KB (card grid with tooltips) / CONSOLE (per-service
     journalctl tail, tabs for gateway/sidecar/observer/mcp/ctx7/auditor/langfuse)
   - tryFetch always attempts JSON.parse (fix for observer returning JSON without content-type)
   - renderNodeContext primitive-vs-object guard (fix for gateway /health string)

 * docs/SCRUM_FIX_WAVE.md     — iter-specific scope directing the scrum
 * docs/SCRUM_FORENSIC_PROMPT.md — adversarial audit prompt (verdict/critical/verified schema)
 * docs/SCRUM_LOOP_NOTES.md   — iteration observations + fix-next-loop queue
 * docs/SYSTEM_EVOLUTION_LAYERS.md — Layers 1-10 roadmap (trust profiling, execution DNA, drift sentinel, etc)

Measurements across iterations
──────────────────────────────
 iter 1 (soft prompt, gpt-oss:120b):   mean score 5.00/10
 iter 3 (forensic, kimi-k2:1t):        mean score 3.56/10 (−1.44 — bar raised)
 iter 4 (same bar, post fixes):        mean score 4.00/10 (+0.44 — fixes landed)

 Score movement iter3→iter4: ↑5 ↓1 =12
 21/21 first-attempt accept by kimi-k2:1t in iter 4
 20/21 emitted forensic JSON (richer signal than markdown)
 16 verified_components captured (proof-of-life, new metric)
 Permission Gradient distribution: 0 auto · 16 dry_run · 4 sim · 1 block

 Observer loop: by_source {scrum: 21, langfuse: 1985, phase24_audit: 1}
 v1/usage: 224 requests, 477K tokens, all tracked

Signal classes per file (iter 3 → iter 4):
 CONVERGING:  1 (ingestd/service.rs — fix clearly landed)
 LOOPING:     4 (catalogd/registry, main, queryd/service, vectord/index_registry)
 ORBITING:    1 (truth — novel findings surfacing as surface ones fix)
 PLATEAU:     9 (scores flat with high confidence — diminishing returns)
 MIXED:       6

Loop thesis status
──────────────────
A file's score rises only when the scrum confirms a real fix landed.
No false positives yet across 3 iterations. Fixes applied to 3 files all
raised their independent scores under the same adversarial prompt. Loop
is measurable, not hand-wavy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

tests/real-world/scrum_master_pipeline.ts

@@ -40,8 +40,27 @@ const OUT_DIR = `/home/profit/lakehouse/tests/real-world/runs/scrum_${Date.now()
 const PRD_PATH = "/home/profit/lakehouse/docs/PRD.md";
 // Using CONTROL_PLANE_PRD as the "suggested changes" doc since it
 // describes the Phase 38-44 target architecture and is on main.
-// COHESION_INTEGRATION_PLAN.md is still on PR #7 branch.
-const PROPOSAL_PATH = "/home/profit/lakehouse/docs/CONTROL_PLANE_PRD.md";
+// Override via LH_SCRUM_PROPOSAL env to point at a fix-wave doc
+// generated from a phase-sweep audit, so the scrum pulls direction
+// from concrete findings instead of the high-level PRD alone.
+const PROPOSAL_PATH = process.env.LH_SCRUM_PROPOSAL
+  || "/home/profit/lakehouse/docs/CONTROL_PLANE_PRD.md";
+
+// Iter 2+ — when LH_SCRUM_FORENSIC is set to a file path, prepend its
+// contents as an adversarial auditor preamble to every per-file prompt.
+// This flips the review tone from "suggest improvements" to "prove it
+// works or mark FAIL." Added 2026-04-23 for iter 2 of the 6x loop.
+// Empty string = no preamble (iter-1 behavior).
+const FORENSIC_PREAMBLE = process.env.LH_SCRUM_FORENSIC
+  ? (() => {
+      try {
+        return require("node:fs").readFileSync(process.env.LH_SCRUM_FORENSIC!, "utf8");
+      } catch (e) {
+        console.error(`[scrum] warning: could not read LH_SCRUM_FORENSIC=${process.env.LH_SCRUM_FORENSIC}: ${e}`);
+        return "";
+      }
+    })()
+  : "";
 
 // Scoped target: 3 representative source files by default.
 // The scrum-master walks these in order and produces one suggestion
@@ -55,13 +74,34 @@ const TARGET_FILES: string[] = process.env.LH_SCRUM_FILES
   ? process.env.LH_SCRUM_FILES.split(",").map(s => s.trim())
   : DEFAULT_TARGETS;
 
+// Cloud-first ladder, STRONGEST-MODEL-FIRST (iter 3+, 2026-04-24).
+// J's direction: "switch to the strongest cloud model" for iter 3 —
+// the forensic prompt is demanding enough that even 120B gets rejected
+// as thin. Rank by parameter count / reasoning strength:
+//   1. kimi-k2:1t            — 1T params, Moonshot flagship (biggest)
+//   2. kimi-k2.6             — Moonshot next-gen, pro tier
+//   3. deepseek-v3.1:671b    — 671B, strong reasoning + coding
+//   4. mistral-large-3:675b  — 675B, deep analysis
+//   5. qwen3.5:397b          — 397B (iter 2's rescue model)
+//   6. gpt-oss:120b          — 120B (iter 1's primary; still strong fallback)
+// Local fallbacks kept for cloud-down scenarios.
+// Hot-path pipelines (scenario.ts / execution_loop) stay local per
+// Phase 20 t1_hot — this scrum is not hot path.
 const LADDER: Array<{ provider: "ollama" | "ollama_cloud"; model: string; note: string }> = [
-  { provider: "ollama",       model: "qwen3.5:latest",       note: "local 7B" },
-  { provider: "ollama",       model: "qwen3:latest",         note: "local 7B (peer)" },
-  { provider: "ollama",       model: "gpt-oss:20b",          note: "local 20B" },
-  { provider: "ollama_cloud", model: "gpt-oss:120b",         note: "cloud 120B" },
-  { provider: "ollama_cloud", model: "devstral-2:123b",      note: "cloud 123B coding specialist" },
-  { provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B last-ditch" },
+  { provider: "ollama_cloud", model: "kimi-k2:1t",           note: "cloud 1T — biggest available, 1.4s probe" },
+  { provider: "ollama_cloud", model: "deepseek-v3.1:671b",   note: "cloud 671B — fast reasoning (1.0s probe)" },
+  { provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B — deep analysis (0.9s probe)" },
+  { provider: "ollama_cloud", model: "gpt-oss:120b",         note: "cloud 120B — reliable workhorse (iter1 baseline)" },
+  { provider: "ollama_cloud", model: "devstral-2:123b",      note: "cloud 123B — coding specialist" },
+  // qwen3.5:397b is the deep final thinker — J's note 2026-04-24:
+  // "qwen3.5 is really smart maybe the last call use that one".
+  // When every other cloud model has produced thin output, this dense
+  // 397B reviewer is the one that tends to push through. Keeping it
+  // LAST in cloud tier, before the local fallback.
+  { provider: "ollama_cloud", model: "qwen3.5:397b",         note: "cloud 397B dense — last-ditch smart reviewer" },
+  { provider: "ollama",       model: "gpt-oss:20b",          note: "local 20B — cloud-down fallback" },
+  // kimi-k2.6 removed 2026-04-24: probe returned empty (not available
+  // on current tier). Keeping note for when pro tier upgrade lands.
 ];
 
 type Chunk = { id: string; text: string; embedding: number[]; origin: string; offset: number };
@@ -258,7 +298,11 @@ async function reviewFile(
     ? `\nIMPORTANT: the "source" below is a multi-shard distillation (tree-split across ${shardsSummarized} shards), NOT the full raw file. DO NOT claim any field, function, or feature is "missing" based on its absence from this distillation — the distillation may have elided it. Only call out gaps that appear DIRECTLY contradicted by the PRD excerpts.\n`
     : "";
 
-  const baseTask = `You are reviewing one source file against the Lakehouse PRD and an active cohesion-integration plan.
+  const forensicPrefix = FORENSIC_PREAMBLE
+    ? `${FORENSIC_PREAMBLE}\n\n═══ FILE UNDER AUDIT ═══\n\n`
+    : "";
+
+  const baseTask = `${forensicPrefix}You are reviewing one source file against the Lakehouse PRD and an active cohesion-integration plan.
 
 FILE: ${rel} (${content.length} bytes${treeSplitFired ? `, tree-split into ${shardsSummarized} shards` : ""})
 ${truncationWarning}
@@ -272,6 +316,15 @@ Produce a structured review with:
 1. Alignment score (1-10) between this file and the PRD intent
 2. 3-5 concrete suggested changes (bullet points), each naming a specific function/line and what to change
 3. Any gap where this file's behavior contradicts the PRD or the proposal
+${FORENSIC_PREAMBLE ? "4. Apply the forensic audit passes from the preamble: pseudocode detection, PRD contract status, normalization/validation pipeline, failure→repair loop, execution memory, relevance orchestration, execution safety, testing evidence. Issue a verdict pass|needs_patch|fail." : ""}
+
+**Per-finding confidence (required on every suggestion):**
+Attach a self-assessed **Confidence: NN%** to every suggested change AND every gap you list. The percentage is your belief that the suggestion is correct, will compile, and lands the PRD intent. Calibration guide:
+- 90-100%: pattern seen repeatedly in shipped code; change is mechanical; low risk of regressions
+- 70-89%: confident in direction, some room for interpretation on API shape or naming
+- 50-69%: plausible fix but may not match existing conventions or may cascade to other files
+- <50%: genuinely uncertain — include regardless so downstream knows to investigate before applying
+Format each finding as: \`**1.** <change>. **Confidence: NN%.**\` (in tables, add a final "Confidence" column.) Low confidence is valuable signal — do not round up.
 
 Respond with markdown. Be specific, not generic. Cite file-region + PRD-chunk-offset when relevant.`;
 
@@ -335,6 +388,91 @@ Respond with markdown. Be specific, not generic. Cite file-region + PRD-chunk-of
     const { appendFile, mkdir } = await import("node:fs/promises");
     const { dirname } = await import("node:path");
     await mkdir(dirname(SCRUM_REVIEWS_JSONL), { recursive: true });
+
+    // Extract per-finding confidences from the accepted markdown.
+    // Patterns tried: "Confidence: NN%", "Confidence**: NN%",
+    // and table-cell "| 92% |". Cap at 20 matches to bound row size.
+    // Added 2026-04-23 (iter 2 direction from J: "make scrum output
+    // include self-assessed confidence per finding").
+    const confidences: number[] = [];
+    // Markdown format: "Confidence: 92%" / "Confidence**: 92%" / "| 92% |"
+    const patMarkdown = /(?:Confidence[*:\s]*\s*|\|\s*)(\d{1,3})\s*%/gi;
+    // JSON format (forensic strict output): "confidence": 92
+    const patJson = /"confidence"\s*:\s*(\d{1,3})(?!\d)/gi;
+    for (const pat of [patMarkdown, patJson]) {
+      const matches = accepted.matchAll(pat);
+      for (const hit of matches) {
+        if (confidences.length >= 40) break;
+        const pct = parseInt(hit[1], 10);
+        if (pct >= 0 && pct <= 100) confidences.push(pct);
+      }
+    }
+    const conf_avg = confidences.length
+      ? Math.round(confidences.reduce((a, b) => a + b, 0) / confidences.length)
+      : null;
+    const conf_min = confidences.length ? Math.min(...confidences) : null;
+
+    // Score extraction — regex accepts decimals ("Score: 4.5/10") and
+    // surrounding punctuation ("4/10 — mid"). iter 3 had 4 unparseable
+    // scores because the prior regex /(\d)\s*\/\s*10/ missed decimals.
+    const scoreMatch = accepted.match(/(?:score[\s*:]*)?(\d(?:\.\d)?)\s*\/\s*10\b/i);
+    const alignment_score = scoreMatch ? parseFloat(scoreMatch[1]) : null;
+
+    // Forensic JSON extraction — iter 3 showed 20/21 files came back
+    // as JSON (verdict + critical_failures[] + verified_components[] + ...)
+    // rather than markdown tables. Previously we only stored suggestions_preview
+    // (truncated to 2KB); now we also capture the structured counters so
+    // consumers can filter by verdict, sort by critical_failures_count, etc.
+    let verdict: string | null = null;
+    let critical_failures_count = 0;
+    let pseudocode_flags_count = 0;
+    let prd_mismatches_count = 0;
+    let missing_components_count = 0;
+    let verified_components_count = 0;
+    let risk_points_count = 0;
+    const isJsonShape = accepted.includes('"verdict"');
+    if (isJsonShape) {
+      const vm = accepted.match(/"verdict"\s*:\s*"([a-z_]+)"/i);
+      verdict = vm ? vm[1] : null;
+      // Count object entries per array by counting occurrences of
+      // either a unique-per-entry field name or {...} bracket pairs
+      // inside the array span. A straight "count opening braces inside
+      // the array range" is simplest and robust to field order.
+      const countArrayEntries = (arrayName: string): number => {
+        const re = new RegExp(`"${arrayName}"\\s*:\\s*\\[([\\s\\S]*?)\\]`, "i");
+        const m = accepted.match(re);
+        if (!m || !m[1].trim()) return 0;
+        // Count opening braces of direct-child objects.
+        let depth = 0, entries = 0;
+        for (const ch of m[1]) {
+          if (ch === '{') { if (depth === 0) entries++; depth++; }
+          else if (ch === '}') depth--;
+        }
+        return entries;
+      };
+      critical_failures_count = countArrayEntries("critical_failures");
+      pseudocode_flags_count = countArrayEntries("pseudocode_flags");
+      prd_mismatches_count = countArrayEntries("prd_mismatches");
+      missing_components_count = countArrayEntries("missing_components");
+      verified_components_count = countArrayEntries("verified_components");
+      risk_points_count = countArrayEntries("risk_points");
+    }
+
+    // Permission Gradient (Layer #6 from SYSTEM_EVOLUTION_LAYERS.md).
+    // Classify the overall finding set by confidence_avg:
+    //   ≥90 auto-apply-safe, ≥70 dry-run + diff, ≥50 simulation only,
+    //   <50 block (human review). Use conf_min as the tier-lower-bound
+    //   so one shaky finding drags the whole row down to the safer tier.
+    const tierFor = (c: number | null): string => {
+      if (c === null) return "unknown";
+      if (c >= 90) return "auto";
+      if (c >= 70) return "dry_run";
+      if (c >= 50) return "simulation";
+      return "block";
+    };
+    const gradient_tier = tierFor(conf_min); // conservative: weakest finding decides
+    const gradient_tier_avg = tierFor(conf_avg);
+
     const row = {
       file: rel,
       reviewed_at: new Date().toISOString(),
@@ -343,15 +481,122 @@ Respond with markdown. Be specific, not generic. Cite file-region + PRD-chunk-of
       attempts_made: history.length,
       tree_split_fired: treeSplitFired,
       suggestions_preview: accepted.slice(0, 2000),
-      schema_version: 2,
+      // Iter-3+ confidence fields.
+      confidences_per_finding: confidences,
+      confidence_avg: conf_avg,
+      confidence_min: conf_min,
+      findings_count: confidences.length,
+      // Layer #6 Permission Gradient — downstream consumers decide
+      // apply-semantics based on these fields instead of blindly trusting
+      // every suggestion.
+      gradient_tier,
+      gradient_tier_avg,
+      // Score (decimal-aware) and forensic JSON structured counters.
+      // iter 4+ fields (schema_version 4).
+      alignment_score,
+      output_format: isJsonShape ? "forensic_json" : "markdown",
+      verdict,
+      critical_failures_count,
+      pseudocode_flags_count,
+      prd_mismatches_count,
+      missing_components_count,
+      verified_components_count,
+      risk_points_count,
+      schema_version: 4,
       scrum_master_reviewed: true,
     };
+
+    // Layer #2 Model Trust Profiling — append one row per file-accept
+    // so over time we can compute per-(model, task_type) success/thin
+    // rates and trust scores. task_type here is inferred from the file
+    // path — good enough for initial stats, refine later.
+    try {
+      const taskType = rel.includes("/truth/") ? "truth"
+        : rel.includes("/gateway/") ? "gateway"
+        : rel.includes("/vectord") ? "vector"
+        : rel.includes("/ingestd") ? "ingest"
+        : rel.includes("/queryd") ? "query"
+        : rel.includes("/storaged") || rel.includes("/catalogd") || rel.includes("/journald") ? "storage"
+        : rel.includes("/aibridge") ? "aibridge"
+        : "other";
+      const trustRow = {
+        run_id: OUT_DIR.split("/").pop(),
+        reviewed_at: row.reviewed_at,
+        file: rel,
+        task_type: taskType,
+        accepted_model: acceptedModel,
+        accepted_on_attempt: acceptedOn,
+        attempts_made: history.length,
+        thin_rejections: history.filter(h => h.status === "thin").length,
+        errors: history.filter(h => h.status === "error").length,
+        confidence_avg: conf_avg,
+        tree_split_fired: treeSplitFired,
+      };
+      await appendFile("/home/profit/lakehouse/data/_kb/model_trust.jsonl",
+        JSON.stringify(trustRow) + "\n");
+    } catch (e) {
+      console.error(`[scrum] model_trust append failed: ${(e as Error).message}`);
+    }
     try {
       await appendFile(SCRUM_REVIEWS_JSONL, JSON.stringify(row) + "\n");
     } catch (e) {
       console.error(`[scrum] failed to append scrum_reviews.jsonl: ${(e as Error).message}`);
     }
 
+    // Close the scrum → observer loop (fix 2026-04-24). Architecture
+    // audit surfaced: observer ring had 2000 ops, 1999 from Langfuse,
+    // zero from scrum. Observer's analyzeErrors + PLAYBOOK_BUILDER loops
+    // were blind to the very pipeline most likely to teach them. One
+    // fire-and-forget POST wires them in. Observer tolerates unreachable
+    // backends; no scrum run fails if observer is down.
+    //
+    // Schema matches observer's ObservedOp shape (source, staffer_id,
+    // sig_hash, event_kind, success, ...). file + accepted_model +
+    // confidence_avg + gradient_tier give downstream analyzers enough
+    // signal to correlate reviews with later regressions.
+    try {
+      const sigHash = createHash("sha256")
+        .update(`${rel}|${OUT_DIR.split("/").pop()}`)
+        .digest("hex")
+        .slice(0, 16);
+      fetch("http://localhost:3800/event", {
+        method: "POST",
+        headers: { "content-type": "application/json" },
+        body: JSON.stringify({
+          source: "scrum",
+          staffer_id: "scrum_master",
+          sig_hash: sigHash,
+          event_kind: "file_review",
+          success: true,
+          run_id: OUT_DIR.split("/").pop(),
+          file: rel,
+          accepted_model: acceptedModel,
+          accepted_on_attempt: acceptedOn,
+          attempts_made: history.length,
+          thin_rejections: history.filter(h => h.status === "thin").length,
+          confidence_avg: conf_avg,
+          confidence_min: conf_min,
+          findings_count: confidences.length,
+          gradient_tier,
+          tree_split_fired: treeSplitFired,
+          // iter4+ forensic-JSON fields so observer's analyzer can
+          // route by verdict / sort by critical_failures_count
+          alignment_score,
+          verdict,
+          output_format: isJsonShape ? "forensic_json" : "markdown",
+          critical_failures_count,
+          verified_components_count,
+          missing_components_count,
+          ts: row.reviewed_at,
+        }),
+        signal: AbortSignal.timeout(3000),
+      }).catch(() => {
+        // observer down — not a scrum-run failure, just lose the signal.
+      });
+    } catch (e) {
+      // Synchronous construction error — ignore.
+    }
+
     // Route the accepted review through llm_team's fact extractor so
     // its entities + relationships land in audit_facts.jsonl alongside
     // inference-side extractions. Same index, two sources. Tagged