scrum + observer: hand-review wire — judgment moved out of the inner loop

Pre-2026-04-25 the scrum_master applied a hardcoded grounding-rate gate
inline. That baked policy into the wrong layer — semantic judgment about
whether a review is grounded belongs in the observer (which has Langfuse
traces, sees every response across the system, and can call cloud LLMs
for real evaluation). Scrum should report DATA, observer DECIDES.

What landed:
- scrum_master_pipeline.ts: removed the inline grounding-pct threshold;
  every accepted candidate now POSTs to observer's /review endpoint with
  {response, source_content, grounding_stats, model, attempt}. Observer
  returns {verdict: accept|reject|cycle, confidence, notes}. On observer
  failure, scrum falls open to accept (observer is policy, not blocker).
- mcp-server/observer.ts: new POST /review endpoint with two-tier
  evaluator. Tier 1: cloud LLM (qwen3-coder:480b at temp=0) hand-reviews
  with full context — response + source excerpt + grounding stats — and
  emits structured verdict JSON. Tier 2: deterministic heuristic over
  grounding pct + total quotes when cloud throttles, marked source:
  "heuristic" so consumers can tune it later by comparing against cloud.
- Every verdict persists to data/_kb/observer_reviews.jsonl with full
  input snapshot so cloud vs heuristic can be A/B compared once cloud
  quota refreshes.

Verified end-to-end: smoke loop iter 1 — observer returned `cycle` on
21% grounding (cycled to next rung), `reject` on 17% (gave up). Iter 2
— `reject` on 12% and 14%. Both UNRESOLVED with honest signal instead
of polluting pathway memory with hallucinated patterns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

tests/real-world/scrum_master_pipeline.ts

@@ -301,6 +301,52 @@ async function recordPathwayReplay(pathwayId: string, succeeded: boolean): Promi
   }
 }
 
+// Observer hand-review — the policy layer that decides whether a
+// candidate response is grounded enough to accept. Lives in mcp-server's
+// observer (port 3800) so it sits OUTSIDE the scrum loop's epistemic
+// scope. Synchronous so scrum can act on the verdict immediately.
+//
+// Returns {verdict, confidence, notes}. verdict ∈ {accept, reject, cycle}.
+// On unreachable observer (network, timeout, parse failure), falls open
+// to {verdict: "accept"} — the observer is the policy layer, not a hard
+// dependency. Pipeline keeps moving when the observer is down.
+const OBSERVER_URL = process.env.LH_OBSERVER_URL ?? "http://localhost:3800";
+
+interface ObserverVerdict {
+  verdict: "accept" | "reject" | "cycle";
+  confidence?: number;
+  notes?: string;
+  source?: "cloud" | "heuristic";
+}
+
+async function observerHandReview(input: {
+  file_path: string;
+  model: string;
+  response: string;
+  source_content: string;
+  grounding_stats: { total: number; grounded: number; groundedPct: number | null };
+  attempt: number;
+}): Promise<ObserverVerdict> {
+  try {
+    const r = await fetch(`${OBSERVER_URL}/review`, {
+      method: "POST",
+      headers: { "content-type": "application/json" },
+      body: JSON.stringify(input),
+      signal: AbortSignal.timeout(60000),
+    });
+    if (!r.ok) {
+      // Observer down or rejected the request — fall open to accept so
+      // the loop keeps moving. Log so we notice degradation.
+      console.error(`[scrum] observer review unreachable (${r.status}), falling open to accept`);
+      return { verdict: "accept", notes: `observer ${r.status}`, source: "heuristic" };
+    }
+    return (await r.json()) as ObserverVerdict;
+  } catch (e: any) {
+    console.error(`[scrum] observer review failed (${e.message}), falling open to accept`);
+    return { verdict: "accept", notes: `observer error: ${e.message}`, source: "heuristic" };
+  }
+}
+
 // ADR-021 Phase C: pre-review enrichment. Fetch aggregated bug
 // fingerprints for this narrow fingerprint (same key as hot-swap —
 // task_class + file_prefix + signal_class) so the reviewer prompt
@@ -1057,22 +1103,41 @@ Respond with markdown. Be specific, not generic. Cite file-region + PRD-chunk-of
       log(`    ✗ thin/unstructured (${r.content.length} chars)`);
       continue;
     }
+    // Compute grounding stats as DATA — feed to observer for hand-review.
+    // We no longer gate locally on a hardcoded threshold; that judgment
+    // belongs to the observer (which has Langfuse traces + can call cloud
+    // models for semantic review). Local stats are still informational
+    // and get appended as a footer for humans.
+    const groundingStats = verifyAnchorGrounding(r.content, content);
+
+    // Observer hand-review — synchronous call to mcp-server :3800. Observer
+    // returns {verdict: accept|reject|cycle, confidence, notes}. If the
+    // observer is unreachable or errors, fall through to acceptance (the
+    // observer is the policy layer; pipeline keeps moving when it's down).
+    const obsVerdict = await observerHandReview({
+      file_path: rel,
+      model: `${rung.provider}/${rung.model}`,
+      response: r.content,
+      source_content: content,
+      grounding_stats: groundingStats,
+      attempt: n,
+    });
+    if (obsVerdict.verdict === "reject" || obsVerdict.verdict === "cycle") {
+      const reason = `observer ${obsVerdict.verdict}: ${obsVerdict.notes ?? "no notes"} (conf=${obsVerdict.confidence ?? "?"})`;
+      history.push({ n, model: rung.model, status: "thin", chars: r.content.length, error: reason });
+      pathwayAttempts.push({ rung: i + 1, model: rung.model, latency_ms: attemptMs, accepted: false, reject_reason: reason });
+      log(`    ✗ ${reason} — cycling ladder`);
+      continue;
+    }
     history.push({ n, model: rung.model, status: "accepted", chars: r.content.length });
     pathwayAttempts.push({ rung: i + 1, model: rung.model, latency_ms: attemptMs, accepted: true, reject_reason: null });
     accepted = r.content;
     acceptedModel = `${rung.provider}/${rung.model}`;
     acceptedOn = n;
-    // Post-acceptance: when tree-split fired, run the anchor-grounding
-    // verifier and append a footer with the grounding rate. The footer
-    // surfaces ungrounded quotes so humans can spot hallucinated
-    // findings at a glance — prevents the 0/10 confabulation pattern
-    // observed on llm_team_ui.py 2026-04-24.
-    if (treeSplitFired) {
-      const stats = verifyAnchorGrounding(accepted, content);
-      log(`    ⚓ anchor grounding: ${stats.grounded}/${stats.total} quotes matched source` +
-        (stats.groundedPct !== null ? ` (${stats.groundedPct}%)` : ""));
-      accepted = appendGroundingFooter(accepted, stats);
-    }
+    log(`    ⚓ anchor grounding: ${groundingStats.grounded}/${groundingStats.total} quotes matched source` +
+      (groundingStats.groundedPct !== null ? ` (${groundingStats.groundedPct}%)` : "") +
+      ` · observer ${obsVerdict.verdict}` + (obsVerdict.confidence ? ` (conf=${obsVerdict.confidence})` : ""));
+    accepted = appendGroundingFooter(accepted, groundingStats);
     log(`    ✓ ACCEPTED on attempt ${n} (${rung.model}, ${r.content.length} chars)`);
     break;
   }