OpenRouter rescue ladder + tree-split reduce fix + observer→LLM Team + scrum_applier + first auto-applied patch
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."

## Infrastructure (scrum loop hardening)

crates/gateway/src/v1/openrouter.rs — new OpenRouter provider
  Direct HTTPS to openrouter.ai/api/v1/chat/completions with OpenAI-compatible shape.
  Key resolution: OPENROUTER_API_KEY env → /home/profit/.env → /root/llm_team_config.json
  (shares LLM Team UI's quota). Added after iter 5 hit repeated Ollama Cloud 502s on
  kimi-k2:1t — different provider backbone as rescue rung. Unit tests pin the URL
  stripping and OpenAI wire shape.

crates/gateway/src/v1/mod.rs + main.rs
  Added `"openrouter" | "openrouter_free"` arm to /v1/chat dispatch.
  V1State.openrouter_key loaded at startup via openrouter::resolve_openrouter_key()
  mirroring the Ollama Cloud pattern. Startup log:
    "v1: OpenRouter key loaded — /v1/chat provider=openrouter enabled"

tests/real-world/scrum_master_pipeline.ts
  * 9-rung ladder — kimi-k2:1t → qwen3-coder:480b → deepseek-v3.1:671b →
    mistral-large-3:675b → gpt-oss:120b → qwen3.5:397b → openrouter/gpt-oss-120b:free
    → openrouter/gemma-3-27b-it:free → local qwen3.5:latest.
    Added qwen3-coder:480b as rung 2 after live probes confirmed it rescues
    kimi-k2:1t 502s cleanly (0.9s latency, substantive reviews).
    Dropped devstral-2 (displaced by qwen3-coder); dropped kimi-k2.6 (not available);
    dropped minimax-m2.7 (returned 0 chars / 400 thinking tokens).
    Local fallback promoted qwen3.5:latest per J's direction 2026-04-24.
  * MAX_ATTEMPTS bumped 6 → 9 to accommodate the rescue tier.
  * Tree-split scratchpad fixed — was concatenating shard markers directly
    into the reviewer input, causing kimi-k2:1t to write titles like
    "Forensic Audit Report – file.rs (shard 3)". Now uses internal §N§
    markers during accumulation and runs a proper reduce step that
    collapses per-shard digests into ONE coherent file-level synthesis
    with markers stripped. Matches the Phase 21 aibridge::tree_split
    map→reduce design. Fallback to stripped scratchpad if reducer returns thin.

tests/real-world/scrum_applier.ts — NEW (737 lines)
  The auto-apply pipeline. Reads scrum_reviews.jsonl, filters rows where
  gradient_tier ∈ {auto, dry_run} AND confidence_avg ≥ MIN_CONF (default 90),
  asks the reviewer model for concrete old_string/new_string patch JSON,
  applies via text replacement, runs cargo check after each file, commits
  if green and reverts if red. Deny-list: /etc/, config/, ops/, auditor/,
  docs/, data/, mcp-server/, ui/, sidecar/, scripts/. Hard caps: per-patch
  confidence ≥ MIN_CONF, old_string must be exactly unique, max 20 lines per
  patch. Never runs on main without explicit LH_APPLIER_BRANCH override.
  Audit trail in data/_kb/auto_apply.jsonl.

  Empirical behavior (dry-run over iter 4 reviews):
    5 eligible files → 1 green commit-ready, 2 build-red reverts, 2 all-rejected
  The build-green gate caught 2 bad patches before they'd have merged.

mcp-server/observer.ts — LLM Team code_review escalation
  When a sig_hash accumulates ≥3 failures (ESCALATION_THRESHOLD), fire-and-forget
  POST /api/run?mode=code_review at localhost:5000 with the failure cluster context.
  Parses facts/entities/relationships/file_hints from the response. Writes to a
  new data/_kb/observer_escalations.jsonl surface. Answers J's vision of the
  observer triggering richer LLM Team calls when failures pile up.
  Non-blocking: runs parallel to existing qwen2.5 analyzer, never replaces it.
  Tracks escalated sig_hashes in a session-local Set to avoid re-hammering
  LLM Team when a cluster persists across observer cycles.

crates/aibridge/src/context.rs
  First auto-applied patch produced by scrum_applier.ts (dry-run path —
  applier writes files in dry-run mode but doesn't commit; bug noted for
  iter 6 fix). Adds #[deprecated] annotation to the inline estimate_tokens
  helper pointing callers to the centralized shared::model_matrix::ModelMatrix
  entry point (P21-002 — duplicate token-estimator surfaces). Cargo check
  passes with the annotation (verified by applier's own build gate).

## Visual Control Plane (UI)

ui/server.ts — Bun.serve on :3950 with /data/* fan-out:
  /data/services, /data/reviews, /data/metrics, /data/trust, /data/overrides,
  /data/findings, /data/outcomes, /data/audit_facts, /data/file/:path,
  /data/refactor_signals, /data/search?q=, /data/signal_classes,
  /data/logs/:svc (journalctl tail per systemd unit), /data/scrum_log.
  Bug fix: tryFetch always attempts JSON.parse before falling back to text
  — observer's Bun.serve returns JSON without application/json content-type,
  which was displaying stats as a raw string ("0 ops" on map) before.

ui/index.html + ui.css — dark neo-brutalist shell. 6 views:
  MAP (D3 force-graph + overlays) / TRACE (per-file iter history) /
  TRAJECTORY (signal-class cards + refactor-signals table + reverse-index
  search box) / METRICS (every card has SOURCE + GOOD lines explaining
  where the number comes from and what target trajectory means) /
  KB (card grid with tooltips on every field) / CONSOLE (per-service
  journalctl tabs).

ui/ui.js — polling client, D3 wiring, signal-class panel, refactor-signals
  table, reverse-index search, per-service console tabs. Bug fix:
  renderNodeContext had Object.entries() iterating string characters when
  /health returned a plain string — now guards with typeof check so
  "lakehouse ok" renders as one row instead of "0 l / 1 a / 2 k / ...".

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-24 03:45:35 -05:00
co-authored by Claude Opus 4.7
parent 39a2856851
commit 8b77d67c9c
10 changed files with 937 additions and 30 deletions
+90
View File
@@ -103,6 +103,93 @@ const REFACTOR_PHRASES = [
"split this file", "too large",
];
// Signal-class classifier — per file, given 2+ consecutive iterations'
// reviews, tag the file's behavior:
// CONVERGING — resolved > novel, score ↑
// LOOPING — 3+ same findings repeat, novel = 0, score flat
// ORBITING — novel findings each iter, no resolved (healthy depth)
// PLATEAU — score flat + findings flat (diminishing returns)
// MIXED — partial/unclear
// This is the foundation for iter-6+ auto-routing: each class gets a
// different sub-pipeline (specialist model, reviewer rotation, etc).
const SIGNAL_PHRASES = [
"pseudocode", "placeholder", "stub", "unwired", "missing", "dead code", "orphaned",
"duplicate", "redundant", "refactor", "rewrite", "remove", "unused", "unnecessary",
];
async function signalClasses(): Promise<any> {
const runsDir = `${REPO}/tests/real-world/runs`;
// Load every review, group by file, sort by timestamp
const perFile: Record<string, Array<{run: string, phrases: Set<string>, score: number | null, conf_avg: number | null, findings: number, ts: number}>> = {};
try {
const dirs = await Array.fromAsync(new Bun.Glob("scrum_*").scan({ cwd: runsDir, onlyFiles: false }));
for (const d of dirs) {
const files = await Array.fromAsync(new Bun.Glob("review_*.json").scan({ cwd: `${runsDir}/${d}` }));
for (const f of files) {
try {
const p = `${runsDir}/${d}/${f}`;
const j = JSON.parse(await Bun.file(p).text());
const key = j.file?.replace("/home/profit/lakehouse/", "") ?? "?";
const sug = (j.suggestions ?? "").toLowerCase();
const phrases = new Set<string>();
for (const ph of SIGNAL_PHRASES) if (sug.includes(ph)) phrases.add(ph);
const scoreMatch = sug.match(/(\d(?:\.\d)?)\s*\/\s*10\b/);
const score = scoreMatch ? parseFloat(scoreMatch[1]) : null;
const mconf = [...sug.matchAll(/(?:confidence[*:\s]*\s*|\|\s*)(\d{1,3})\s*%/gi)].map(m=>parseInt(m[1],10));
const jconf = [...sug.matchAll(/"confidence"\s*:\s*(\d{1,3})(?!\d)/gi)].map(m=>parseInt(m[1],10));
const all = [...mconf, ...jconf].filter(x => 0 <= x && x <= 100);
const conf_avg = all.length ? Math.round(all.reduce((a,b)=>a+b,0)/all.length) : null;
const ts = (await Bun.file(p).stat()).mtime.getTime();
(perFile[key] ??= []).push({ run: d, phrases, score, conf_avg, findings: all.length, ts });
} catch {}
}
}
} catch (e) {
return { error: String(e), classes: {} };
}
const classes: Record<string, any> = {};
for (const [file, runs] of Object.entries(perFile)) {
runs.sort((a, b) => a.ts - b.ts);
if (runs.length < 2) { classes[file] = { cls: "NEW", runs: runs.length }; continue; }
const last = runs[runs.length - 1];
const prev = runs[runs.length - 2];
const novel = [...last.phrases].filter(p => !prev.phrases.has(p));
const resolved = [...prev.phrases].filter(p => !last.phrases.has(p));
const looping = [...prev.phrases].filter(p => last.phrases.has(p));
const dScore = (last.score != null && prev.score != null) ? last.score - prev.score : null;
const dConf = (last.conf_avg != null && prev.conf_avg != null) ? last.conf_avg - prev.conf_avg : null;
const dFindings = last.findings - prev.findings;
let cls: string;
if (dScore != null && dScore > 0 && resolved.length > novel.length) cls = "CONVERGING";
else if (looping.length >= 3 && novel.length === 0 && (dScore == null || Math.abs(dScore) < 0.5)) cls = "LOOPING";
else if (novel.length >= 2 && resolved.length === 0) cls = "ORBITING";
else if (Math.abs(dFindings) <= 1 && (dScore == null || Math.abs(dScore) < 0.5)) cls = "PLATEAU";
else cls = "MIXED";
classes[file] = {
cls,
runs: runs.length,
iter_span: `${runs[0].run}…${last.run}`,
prev_score: prev.score,
last_score: last.score,
delta_score: dScore,
delta_conf: dConf,
delta_findings: dFindings,
novel,
resolved,
looping,
};
}
// Summary counts
const counts: Record<string, number> = {};
for (const v of Object.values(classes)) counts[v.cls] = (counts[v.cls] ?? 0) + 1;
return { generated_at: new Date().toISOString(), counts, classes };
}
async function refactorSignals(): Promise<any> {
// Walk every accepted review across all scrum runs. For each file,
// count how many times its suggestions mention a refactor phrase.
@@ -269,6 +356,9 @@ Bun.serve({
if (path === "/data/refactor_signals") {
return Response.json(await refactorSignals());
}
if (path === "/data/signal_classes") {
return Response.json(await signalClasses());
}
if (path === "/data/search") {
const q = url.searchParams.get("q") ?? "";
return Response.json(await reverseIndex(q, 30));
+33
View File
@@ -405,3 +405,36 @@ main {
.traj-hit-meta { font-family: var(--mono); font-size: 10px; color: var(--fg-muted); }
.traj-hit-snip { font-family: var(--mono); font-size: 11px; color: var(--fg-dim); line-height: 1.5; }
/* signal classes */
.signal-class-row { display: flex; gap: 8px; margin-bottom: 12px; flex-wrap: wrap; }
.signal-chip {
font-family: var(--mono); font-size: 10px; letter-spacing: 0.08em;
padding: 4px 10px; border: 1px solid var(--border); font-weight: 700;
}
.signal-converging { color: var(--green); border-color: var(--green); }
.signal-looping { color: var(--red); border-color: var(--red); }
.signal-orbiting { color: var(--purple); border-color: var(--purple); }
.signal-plateau { color: var(--yellow); border-color: var(--yellow); }
.signal-mixed { color: var(--blue); border-color: var(--blue); }
.signal-new { color: var(--fg-muted); border-color: var(--fg-muted); }
.signal-grid {
display: grid; grid-template-columns: repeat(auto-fill, minmax(480px, 1fr)); gap: 8px;
}
.signal-card {
border: 1px solid var(--border); background: var(--bg-1);
padding: 10px 12px; cursor: pointer; border-left: 3px solid var(--fg-muted);
}
.signal-card.signal-converging { border-left-color: var(--green); }
.signal-card.signal-looping { border-left-color: var(--red); }
.signal-card.signal-orbiting { border-left-color: var(--purple); }
.signal-card.signal-plateau { border-left-color: var(--yellow); }
.signal-card.signal-mixed { border-left-color: var(--blue); }
.signal-card:hover { border-color: var(--border-hi); background: var(--bg-2); }
.signal-card-top { display: flex; gap: 10px; align-items: center; margin-bottom: 6px; }
.signal-card-file { font-family: var(--mono); font-size: 11px; color: var(--fg); }
.signal-card-body { font-family: var(--mono); font-size: 10px; color: var(--fg-dim); line-height: 1.55; }
.signal-card-body > div { margin-top: 2px; }
.signal-novel { color: var(--purple); }
.signal-resolved { color: var(--green); }
.signal-loop { color: var(--red); }
+48
View File
@@ -444,6 +444,54 @@ async function drawTrajectory() {
const statsEl = document.getElementById("traj-stats");
clear(statsEl);
// SECTION 0 — signal classes (CONVERGING/LOOPING/ORBITING/PLATEAU/MIXED)
try {
const sc = await fetch("/data/signal_classes").then(r => r.json());
body.append(el("div", { className: "traj-section-head", text: "SIGNAL CLASSES · iter-to-iter behavior per file" }));
body.append(el("div", { className: "traj-section-explain", text:
"Each file compared iter-to-iter: CONVERGING = fix landed (resolved > novel + score↑), " +
"LOOPING = same findings repeating (deadlock candidate for hyper-focus), " +
"ORBITING = novel findings every iter (healthy depth-first), " +
"PLATEAU = score+findings flat (diminishing returns, needs different angle), " +
"MIXED = partial movement, NEW = only 1 iter so far."
}));
const classRow = el("div", { className: "signal-class-row" });
for (const [cls, n] of Object.entries(sc.counts ?? {})) {
const chip = el("span", { className: `signal-chip signal-${cls.toLowerCase()}`, text: `${cls} ${n}` });
classRow.append(chip);
}
body.append(classRow);
const grid = el("div", { className: "signal-grid" });
const sorted = Object.entries(sc.classes ?? {}).sort((a, b) => {
const order = { CONVERGING: 0, LOOPING: 1, ORBITING: 2, MIXED: 3, PLATEAU: 4, NEW: 5 };
return (order[a[1].cls] ?? 9) - (order[b[1].cls] ?? 9);
});
for (const [file, info] of sorted) {
const card = el("div", { className: `signal-card signal-${info.cls.toLowerCase()}` });
card.append(el("div", { className: "signal-card-top" },
el("span", { className: `signal-chip signal-${info.cls.toLowerCase()}`, text: info.cls }),
el("span", { className: "signal-card-file", text: file })
));
const body2 = el("div", { className: "signal-card-body" });
if (info.prev_score != null || info.last_score != null) {
body2.append(el("div", { text: `score ${info.prev_score ?? "?"} → ${info.last_score ?? "?"} (Δ ${info.delta_score != null ? (info.delta_score > 0 ? "+" : "") + info.delta_score.toFixed(1) : "?"})` }));
}
if (info.novel?.length) body2.append(el("div", { className: "signal-novel", text: `NEW: ${info.novel.join(", ")}` }));
if (info.resolved?.length) body2.append(el("div", { className: "signal-resolved", text: `RESOLVED: ${info.resolved.join(", ")}` }));
if (info.looping?.length) body2.append(el("div", { className: "signal-loop", text: `LOOPING: ${info.looping.join(", ")}` }));
card.append(body2);
card.addEventListener("click", () => {
state.selected = { type: "file", id: `/home/profit/lakehouse/${file}` };
renderContext();
document.querySelector('#views button[data-view="trace"]').click();
});
grid.append(card);
}
body.append(grid);
} catch (e) {
body.append(el("div", { className: "ctx-hint", text: `signal classes error: ${e}` }));
}
// SECTION 1 — refactor signals
const sig = await fetch("/data/refactor_signals").then(r => r.json());
const sigs = sig.signals ?? [];