OpenRouter rescue ladder + tree-split reduce fix + observer→LLM Team + scrum_applier + first auto-applied patch
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
## Infrastructure (scrum loop hardening)
crates/gateway/src/v1/openrouter.rs — new OpenRouter provider
Direct HTTPS to openrouter.ai/api/v1/chat/completions with OpenAI-compatible shape.
Key resolution: OPENROUTER_API_KEY env → /home/profit/.env → /root/llm_team_config.json
(shares LLM Team UI's quota). Added after iter 5 hit repeated Ollama Cloud 502s on
kimi-k2:1t — different provider backbone as rescue rung. Unit tests pin the URL
stripping and OpenAI wire shape.
crates/gateway/src/v1/mod.rs + main.rs
Added `"openrouter" | "openrouter_free"` arm to /v1/chat dispatch.
V1State.openrouter_key loaded at startup via openrouter::resolve_openrouter_key()
mirroring the Ollama Cloud pattern. Startup log:
"v1: OpenRouter key loaded — /v1/chat provider=openrouter enabled"
tests/real-world/scrum_master_pipeline.ts
* 9-rung ladder — kimi-k2:1t → qwen3-coder:480b → deepseek-v3.1:671b →
mistral-large-3:675b → gpt-oss:120b → qwen3.5:397b → openrouter/gpt-oss-120b:free
→ openrouter/gemma-3-27b-it:free → local qwen3.5:latest.
Added qwen3-coder:480b as rung 2 after live probes confirmed it rescues
kimi-k2:1t 502s cleanly (0.9s latency, substantive reviews).
Dropped devstral-2 (displaced by qwen3-coder); dropped kimi-k2.6 (not available);
dropped minimax-m2.7 (returned 0 chars / 400 thinking tokens).
Local fallback promoted qwen3.5:latest per J's direction 2026-04-24.
* MAX_ATTEMPTS bumped 6 → 9 to accommodate the rescue tier.
* Tree-split scratchpad fixed — was concatenating shard markers directly
into the reviewer input, causing kimi-k2:1t to write titles like
"Forensic Audit Report – file.rs (shard 3)". Now uses internal §N§
markers during accumulation and runs a proper reduce step that
collapses per-shard digests into ONE coherent file-level synthesis
with markers stripped. Matches the Phase 21 aibridge::tree_split
map→reduce design. Fallback to stripped scratchpad if reducer returns thin.
tests/real-world/scrum_applier.ts — NEW (737 lines)
The auto-apply pipeline. Reads scrum_reviews.jsonl, filters rows where
gradient_tier ∈ {auto, dry_run} AND confidence_avg ≥ MIN_CONF (default 90),
asks the reviewer model for concrete old_string/new_string patch JSON,
applies via text replacement, runs cargo check after each file, commits
if green and reverts if red. Deny-list: /etc/, config/, ops/, auditor/,
docs/, data/, mcp-server/, ui/, sidecar/, scripts/. Hard caps: per-patch
confidence ≥ MIN_CONF, old_string must be exactly unique, max 20 lines per
patch. Never runs on main without explicit LH_APPLIER_BRANCH override.
Audit trail in data/_kb/auto_apply.jsonl.
Empirical behavior (dry-run over iter 4 reviews):
5 eligible files → 1 green commit-ready, 2 build-red reverts, 2 all-rejected
The build-green gate caught 2 bad patches before they'd have merged.
mcp-server/observer.ts — LLM Team code_review escalation
When a sig_hash accumulates ≥3 failures (ESCALATION_THRESHOLD), fire-and-forget
POST /api/run?mode=code_review at localhost:5000 with the failure cluster context.
Parses facts/entities/relationships/file_hints from the response. Writes to a
new data/_kb/observer_escalations.jsonl surface. Answers J's vision of the
observer triggering richer LLM Team calls when failures pile up.
Non-blocking: runs parallel to existing qwen2.5 analyzer, never replaces it.
Tracks escalated sig_hashes in a session-local Set to avoid re-hammering
LLM Team when a cluster persists across observer cycles.
crates/aibridge/src/context.rs
First auto-applied patch produced by scrum_applier.ts (dry-run path —
applier writes files in dry-run mode but doesn't commit; bug noted for
iter 6 fix). Adds #[deprecated] annotation to the inline estimate_tokens
helper pointing callers to the centralized shared::model_matrix::ModelMatrix
entry point (P21-002 — duplicate token-estimator surfaces). Cargo check
passes with the annotation (verified by applier's own build gate).
## Visual Control Plane (UI)
ui/server.ts — Bun.serve on :3950 with /data/* fan-out:
/data/services, /data/reviews, /data/metrics, /data/trust, /data/overrides,
/data/findings, /data/outcomes, /data/audit_facts, /data/file/:path,
/data/refactor_signals, /data/search?q=, /data/signal_classes,
/data/logs/:svc (journalctl tail per systemd unit), /data/scrum_log.
Bug fix: tryFetch always attempts JSON.parse before falling back to text
— observer's Bun.serve returns JSON without application/json content-type,
which was displaying stats as a raw string ("0 ops" on map) before.
ui/index.html + ui.css — dark neo-brutalist shell. 6 views:
MAP (D3 force-graph + overlays) / TRACE (per-file iter history) /
TRAJECTORY (signal-class cards + refactor-signals table + reverse-index
search box) / METRICS (every card has SOURCE + GOOD lines explaining
where the number comes from and what target trajectory means) /
KB (card grid with tooltips on every field) / CONSOLE (per-service
journalctl tabs).
ui/ui.js — polling client, D3 wiring, signal-class panel, refactor-signals
table, reverse-index search, per-service console tabs. Bug fix:
renderNodeContext had Object.entries() iterating string characters when
/health returned a plain string — now guards with typeof check so
"lakehouse ok" renders as one row instead of "0 l / 1 a / 2 k / ...".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,339 @@
|
||||
// scrum_applier.ts — the auto-apply pipeline.
|
||||
//
|
||||
// Turns the scrum master's signal into real commits. Reads
|
||||
// data/_kb/scrum_reviews.jsonl, filters to rows where the scrum's
|
||||
// own confidence is high enough to trust auto-apply (gradient_tier
|
||||
// auto OR confidence_avg ≥ 90), asks a patch-emitting model to
|
||||
// produce concrete old_string/new_string pairs, applies them via
|
||||
// text replacement, runs `cargo check` after each, commits on green
|
||||
// and reverts on red.
|
||||
//
|
||||
// Runs on its own branch (never on main). Every action is recorded
|
||||
// in data/_kb/auto_apply.jsonl so the auditor and future iterations
|
||||
// can see what landed and what reverted.
|
||||
//
|
||||
// Usage:
|
||||
// bun run tests/real-world/scrum_applier.ts # dry-run, print only
|
||||
// LH_APPLIER_COMMIT=1 bun run tests/real-world/scrum_applier.ts # actually apply
|
||||
//
|
||||
// Env:
|
||||
// LH_APPLIER_BRANCH — branch name (default: "scrum/auto-apply-${Date.now()}")
|
||||
// LH_APPLIER_MIN_CONF — minimum confidence_avg, default 90
|
||||
// LH_APPLIER_MAX_FILES — cap on files per run (default 5, keeps diffs reviewable)
|
||||
// LH_APPLIER_COMMIT — "1" to actually commit; otherwise dry-run
|
||||
// LH_APPLIER_MODEL — patch-emitting model (default: kimi-k2:1t)
|
||||
|
||||
import { readFile, writeFile, appendFile } from "node:fs/promises";
|
||||
import { existsSync } from "node:fs";
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
const REPO = "/home/profit/lakehouse";
|
||||
const GATEWAY = "http://localhost:3100";
|
||||
const SCRUM_REVIEWS = `${REPO}/data/_kb/scrum_reviews.jsonl`;
|
||||
const AUDIT_LOG = `${REPO}/data/_kb/auto_apply.jsonl`;
|
||||
|
||||
const MIN_CONF = Number(process.env.LH_APPLIER_MIN_CONF ?? 90);
|
||||
const MAX_FILES = Number(process.env.LH_APPLIER_MAX_FILES ?? 5);
|
||||
const COMMIT = process.env.LH_APPLIER_COMMIT === "1";
|
||||
const MODEL = process.env.LH_APPLIER_MODEL ?? "kimi-k2:1t";
|
||||
const BRANCH = process.env.LH_APPLIER_BRANCH ?? `scrum/auto-apply-${Date.now().toString(36)}`;
|
||||
|
||||
// Deny-list — anything whose path starts with one of these is skipped
|
||||
// regardless of how confident the scrum is. Config / systemd / docs /
|
||||
// auditor itself are off limits for auto-apply; they need a human.
|
||||
const DENY_PREFIXES = [
|
||||
"config/",
|
||||
"ops/",
|
||||
"auditor/",
|
||||
"docs/",
|
||||
"data/",
|
||||
"/etc/",
|
||||
"mcp-server/",
|
||||
"ui/",
|
||||
"sidecar/",
|
||||
"scripts/",
|
||||
];
|
||||
|
||||
function log(msg: string) { console.log(`[applier] ${msg}`); }
|
||||
|
||||
async function sh(cmd: string[], cwd = REPO): Promise<{ stdout: string; stderr: string; code: number }> {
|
||||
return new Promise((resolve) => {
|
||||
const p = spawn(cmd[0], cmd.slice(1), { cwd, stdio: ["ignore", "pipe", "pipe"] });
|
||||
let out = ""; let err = "";
|
||||
p.stdout.on("data", (d) => { out += d.toString(); });
|
||||
p.stderr.on("data", (d) => { err += d.toString(); });
|
||||
p.on("close", (code) => resolve({ stdout: out, stderr: err, code: code ?? 1 }));
|
||||
});
|
||||
}
|
||||
|
||||
async function auditLog(row: Record<string, any>) {
|
||||
const line = JSON.stringify({ ...row, ts: new Date().toISOString() }) + "\n";
|
||||
await appendFile(AUDIT_LOG, line);
|
||||
}
|
||||
|
||||
async function chat(opts: {
|
||||
provider: "ollama_cloud" | "openrouter" | "ollama";
|
||||
model: string;
|
||||
prompt: string;
|
||||
max_tokens?: number;
|
||||
}): Promise<{ content: string; error?: string }> {
|
||||
try {
|
||||
const r = await fetch(`${GATEWAY}/v1/chat`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
provider: opts.provider,
|
||||
model: opts.model,
|
||||
messages: [{ role: "user", content: opts.prompt }],
|
||||
max_tokens: opts.max_tokens ?? 1500,
|
||||
temperature: 0.1,
|
||||
}),
|
||||
signal: AbortSignal.timeout(180000),
|
||||
});
|
||||
if (!r.ok) return { content: "", error: `${r.status}: ${(await r.text()).slice(0, 300)}` };
|
||||
const j: any = await r.json();
|
||||
return { content: j.choices?.[0]?.message?.content ?? "" };
|
||||
} catch (e) {
|
||||
return { content: "", error: String(e) };
|
||||
}
|
||||
}
|
||||
|
||||
interface ScrumReview {
|
||||
file: string;
|
||||
reviewed_at: string;
|
||||
accepted_model: string;
|
||||
suggestions_preview: string;
|
||||
confidences_per_finding?: number[];
|
||||
confidence_avg?: number | null;
|
||||
confidence_min?: number | null;
|
||||
gradient_tier?: string;
|
||||
gradient_tier_avg?: string;
|
||||
verdict?: string;
|
||||
critical_failures_count?: number;
|
||||
schema_version?: number;
|
||||
}
|
||||
|
||||
async function loadLatestReviews(): Promise<Map<string, ScrumReview>> {
|
||||
// Map of file → latest review for that file. Ordered by reviewed_at.
|
||||
if (!existsSync(SCRUM_REVIEWS)) return new Map();
|
||||
const text = await readFile(SCRUM_REVIEWS, "utf8");
|
||||
const rows: ScrumReview[] = text.split("\n").filter(Boolean).map(l => {
|
||||
try { return JSON.parse(l); } catch { return null; }
|
||||
}).filter((r): r is ScrumReview => r !== null);
|
||||
// Keep the LATEST review per file.
|
||||
const latest = new Map<string, ScrumReview>();
|
||||
for (const r of rows) {
|
||||
if (!r.file) continue;
|
||||
const prev = latest.get(r.file);
|
||||
if (!prev || (r.reviewed_at > prev.reviewed_at)) latest.set(r.file, r);
|
||||
}
|
||||
return latest;
|
||||
}
|
||||
|
||||
function passesConfidenceGate(r: ScrumReview): boolean {
|
||||
const avg = r.confidence_avg ?? 0;
|
||||
const min = r.confidence_min ?? 0;
|
||||
// Must be auto or dry_run tier AND confidence_min ≥ MIN_CONF.
|
||||
// min is the conservative tier-lower-bound (one weak finding drags
|
||||
// the whole file to "simulation" or "block" tier).
|
||||
if (r.gradient_tier === "block" || r.gradient_tier === "simulation") return false;
|
||||
return avg >= MIN_CONF && min >= 70;
|
||||
}
|
||||
|
||||
function passesDenyList(file: string): boolean {
|
||||
return !DENY_PREFIXES.some((p) => file.startsWith(p) || file === p.replace(/\/$/, ""));
|
||||
}
|
||||
|
||||
interface Patch {
|
||||
file: string;
|
||||
old_string: string;
|
||||
new_string: string;
|
||||
rationale: string;
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
async function requestPatches(file: string, source: string, review: string): Promise<Patch[]> {
|
||||
const prompt = `You previously produced this review of ${file}:
|
||||
|
||||
─── REVIEW ───
|
||||
${review}
|
||||
─── END REVIEW ───
|
||||
|
||||
The review is high-confidence and the file is eligible for auto-apply. Produce CONCRETE PATCHES as JSON so they can be applied via string replacement.
|
||||
|
||||
RULES:
|
||||
1. Output ONE JSON object with a "patches" array. NO prose, no markdown fences.
|
||||
2. Each patch is {"old_string": "...", "new_string": "...", "rationale": "short", "confidence": 0-100}.
|
||||
3. "old_string" MUST appear EXACTLY ONCE in the file (verbatim, including whitespace). If no unique anchor exists, SKIP that suggestion.
|
||||
4. Mechanical changes only: wire a function call, add a field, remove #[allow(dead_code)], add a missing use import, rename one call-site. NO architectural rewrites. NO new modules.
|
||||
5. Each "new_string" MUST compile in isolation with the same surrounding code. Don't introduce new dependencies.
|
||||
6. If you cannot produce at least one high-confidence mechanical patch, output {"patches": []}.
|
||||
7. Max 3 patches per file.
|
||||
|
||||
─── SOURCE (${source.length} bytes) ───
|
||||
${source.slice(0, 14000)}
|
||||
─── END SOURCE ───
|
||||
|
||||
Emit ONLY the JSON object.`;
|
||||
|
||||
const r = await chat({ provider: "ollama_cloud", model: MODEL, prompt, max_tokens: 2500 });
|
||||
if (r.error || !r.content) return [];
|
||||
|
||||
// Strip markdown fences if model wrapped the JSON.
|
||||
let raw = r.content.trim();
|
||||
const fenceStart = raw.match(/^```(?:json)?\s*/);
|
||||
if (fenceStart) raw = raw.slice(fenceStart[0].length);
|
||||
if (raw.endsWith("```")) raw = raw.slice(0, -3).trim();
|
||||
// Find first { and last } to extract JSON block if there's prose.
|
||||
const first = raw.indexOf("{");
|
||||
const last = raw.lastIndexOf("}");
|
||||
if (first >= 0 && last > first) raw = raw.slice(first, last + 1);
|
||||
|
||||
try {
|
||||
const obj = JSON.parse(raw);
|
||||
const patches: Patch[] = (obj.patches ?? []).filter((p: any) =>
|
||||
typeof p?.old_string === "string" &&
|
||||
typeof p?.new_string === "string" &&
|
||||
p.old_string !== p.new_string &&
|
||||
p.old_string.length > 0 &&
|
||||
typeof p?.confidence === "number"
|
||||
).map((p: any) => ({
|
||||
file,
|
||||
old_string: p.old_string,
|
||||
new_string: p.new_string,
|
||||
rationale: String(p.rationale ?? ""),
|
||||
confidence: p.confidence,
|
||||
}));
|
||||
return patches;
|
||||
} catch (e) {
|
||||
log(` ${file}: patch JSON parse failed — ${String(e).slice(0, 100)}`);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function applyPatches(file: string, patches: Patch[]): Promise<{ applied: number; rejected: Array<{patch: Patch; reason: string}> }> {
|
||||
const full = `${REPO}/${file}`;
|
||||
let source = await readFile(full, "utf8");
|
||||
const rejected: Array<{patch: Patch; reason: string}> = [];
|
||||
let applied = 0;
|
||||
for (const p of patches) {
|
||||
// Confidence gate at the individual-patch level.
|
||||
if (p.confidence < MIN_CONF) { rejected.push({patch: p, reason: `confidence ${p.confidence} < ${MIN_CONF}`}); continue; }
|
||||
// Uniqueness gate.
|
||||
const occurrences = source.split(p.old_string).length - 1;
|
||||
if (occurrences === 0) { rejected.push({patch: p, reason: "old_string not found"}); continue; }
|
||||
if (occurrences > 1) { rejected.push({patch: p, reason: `old_string appears ${occurrences}× (not unique)`}); continue; }
|
||||
// Size gate — no patch touches > 20 lines (diff discipline).
|
||||
const oldLines = p.old_string.split("\n").length;
|
||||
const newLines = p.new_string.split("\n").length;
|
||||
if (Math.max(oldLines, newLines) > 20) { rejected.push({patch: p, reason: `patch too large (${Math.max(oldLines,newLines)} lines)`}); continue; }
|
||||
source = source.replace(p.old_string, p.new_string);
|
||||
applied++;
|
||||
}
|
||||
if (applied > 0) await writeFile(full, source);
|
||||
return { applied, rejected };
|
||||
}
|
||||
|
||||
async function cargoCheck(): Promise<boolean> {
|
||||
const r = await sh(["cargo", "check", "--workspace"]);
|
||||
return r.code === 0;
|
||||
}
|
||||
|
||||
async function gitCommit(file: string, patches: Patch[]): Promise<boolean> {
|
||||
if (!COMMIT) { log(` (dry-run) would commit ${file}`); return true; }
|
||||
const addR = await sh(["git", "add", file]);
|
||||
if (addR.code !== 0) { log(` git add failed: ${addR.stderr.slice(0, 200)}`); return false; }
|
||||
const msg = `auto-apply: ${patches.length} high-confidence fix${patches.length === 1 ? "" : "es"} in ${file}\n\n${patches.map(p => `- ${p.rationale} (conf ${p.confidence}%)`).join("\n")}\n\n🤖 scrum_applier.ts`;
|
||||
const commitR = await sh(["git", "commit", "-m", msg]);
|
||||
if (commitR.code !== 0) { log(` git commit failed: ${commitR.stderr.slice(0, 200)}`); return false; }
|
||||
log(` ✓ committed ${file}`);
|
||||
return true;
|
||||
}
|
||||
|
||||
async function revertFile(file: string): Promise<void> {
|
||||
await sh(["git", "checkout", "--", file]);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
log(`starting · min_conf=${MIN_CONF} max_files=${MAX_FILES} model=${MODEL} commit=${COMMIT}`);
|
||||
|
||||
if (COMMIT) {
|
||||
const headR = await sh(["git", "rev-parse", "--abbrev-ref", "HEAD"]);
|
||||
const currentBranch = headR.stdout.trim();
|
||||
if (currentBranch === "main") {
|
||||
log(`refusing to run on main — create a branch first or set LH_APPLIER_BRANCH`);
|
||||
const coR = await sh(["git", "checkout", "-b", BRANCH]);
|
||||
if (coR.code !== 0) { log(`could not create branch ${BRANCH}: ${coR.stderr.slice(0, 200)}`); process.exit(1); }
|
||||
log(`working branch: ${BRANCH}`);
|
||||
} else {
|
||||
log(`working branch: ${currentBranch}`);
|
||||
}
|
||||
}
|
||||
|
||||
const reviews = await loadLatestReviews();
|
||||
log(`loaded ${reviews.size} latest reviews`);
|
||||
|
||||
const eligible = [...reviews.values()].filter(r =>
|
||||
passesConfidenceGate(r) && passesDenyList(r.file)
|
||||
).sort((a, b) => (b.confidence_avg ?? 0) - (a.confidence_avg ?? 0));
|
||||
|
||||
log(`${eligible.length} pass confidence gate + deny-list`);
|
||||
log(`taking top ${Math.min(MAX_FILES, eligible.length)} by confidence`);
|
||||
|
||||
let committedFiles = 0;
|
||||
let revertedFiles = 0;
|
||||
|
||||
for (const r of eligible.slice(0, MAX_FILES)) {
|
||||
log(`${r.file} (conf_avg=${r.confidence_avg} tier=${r.gradient_tier})`);
|
||||
const full = `${REPO}/${r.file}`;
|
||||
if (!existsSync(full)) { log(` skip — file not found on disk`); continue; }
|
||||
|
||||
const source = await readFile(full, "utf8");
|
||||
const patches = await requestPatches(r.file, source, r.suggestions_preview ?? "");
|
||||
|
||||
if (patches.length === 0) {
|
||||
log(` no patches produced`);
|
||||
await auditLog({ action: "no_patches", file: r.file, reviewer_model: r.accepted_model });
|
||||
continue;
|
||||
}
|
||||
|
||||
log(` ${patches.length} candidate patches`);
|
||||
const { applied, rejected } = await applyPatches(r.file, patches);
|
||||
log(` applied ${applied}, rejected ${rejected.length}`);
|
||||
for (const rj of rejected) log(` ✗ ${rj.reason}`);
|
||||
|
||||
if (applied === 0) {
|
||||
await auditLog({ action: "all_rejected", file: r.file, rejected: rejected.map(x => x.reason) });
|
||||
continue;
|
||||
}
|
||||
|
||||
log(` running cargo check...`);
|
||||
const green = await cargoCheck();
|
||||
if (!green) {
|
||||
log(` ✗ build red — reverting ${r.file}`);
|
||||
await revertFile(r.file);
|
||||
revertedFiles++;
|
||||
await auditLog({ action: "build_red_reverted", file: r.file, patches_applied: applied });
|
||||
continue;
|
||||
}
|
||||
|
||||
log(` ✓ build green`);
|
||||
const ok = await gitCommit(r.file, patches.slice(0, applied));
|
||||
if (ok) {
|
||||
committedFiles++;
|
||||
await auditLog({
|
||||
action: COMMIT ? "committed" : "dry_run_committed",
|
||||
file: r.file,
|
||||
patches_applied: applied,
|
||||
patches_rejected: rejected.length,
|
||||
confidence_avg: r.confidence_avg,
|
||||
gradient_tier: r.gradient_tier,
|
||||
reviewer_model: r.accepted_model,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
log(`DONE · committed=${committedFiles} reverted=${revertedFiles}`);
|
||||
}
|
||||
|
||||
await main();
|
||||
@@ -26,7 +26,7 @@ const SIDECAR = "http://localhost:3200";
|
||||
const CHUNK_SIZE = 800;
|
||||
const CHUNK_OVERLAP = 120;
|
||||
const TOP_K_CONTEXT = 5;
|
||||
const MAX_ATTEMPTS = 6;
|
||||
const MAX_ATTEMPTS = 9;
|
||||
// Files larger than this get tree-split instead of truncated. Fixes the
|
||||
// 6KB false-positive class (model claiming a field is "missing" when
|
||||
// it exists past the context cutoff).
|
||||
@@ -87,21 +87,27 @@ const TARGET_FILES: string[] = process.env.LH_SCRUM_FILES
|
||||
// Local fallbacks kept for cloud-down scenarios.
|
||||
// Hot-path pipelines (scenario.ts / execution_loop) stay local per
|
||||
// Phase 20 t1_hot — this scrum is not hot path.
|
||||
const LADDER: Array<{ provider: "ollama" | "ollama_cloud"; model: string; note: string }> = [
|
||||
{ provider: "ollama_cloud", model: "kimi-k2:1t", note: "cloud 1T — biggest available, 1.4s probe" },
|
||||
{ provider: "ollama_cloud", model: "deepseek-v3.1:671b", note: "cloud 671B — fast reasoning (1.0s probe)" },
|
||||
{ provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B — deep analysis (0.9s probe)" },
|
||||
{ provider: "ollama_cloud", model: "gpt-oss:120b", note: "cloud 120B — reliable workhorse (iter1 baseline)" },
|
||||
{ provider: "ollama_cloud", model: "devstral-2:123b", note: "cloud 123B — coding specialist" },
|
||||
// qwen3.5:397b is the deep final thinker — J's note 2026-04-24:
|
||||
// "qwen3.5 is really smart maybe the last call use that one".
|
||||
// When every other cloud model has produced thin output, this dense
|
||||
// 397B reviewer is the one that tends to push through. Keeping it
|
||||
// LAST in cloud tier, before the local fallback.
|
||||
{ provider: "ollama_cloud", model: "qwen3.5:397b", note: "cloud 397B dense — last-ditch smart reviewer" },
|
||||
{ provider: "ollama", model: "gpt-oss:20b", note: "local 20B — cloud-down fallback" },
|
||||
// kimi-k2.6 removed 2026-04-24: probe returned empty (not available
|
||||
// on current tier). Keeping note for when pro tier upgrade lands.
|
||||
const LADDER: Array<{ provider: "ollama" | "ollama_cloud" | "openrouter"; model: string; note: string }> = [
|
||||
{ provider: "ollama_cloud", model: "kimi-k2:1t", note: "cloud 1T — biggest available, 1.4s probe" },
|
||||
{ provider: "ollama_cloud", model: "qwen3-coder:480b", note: "cloud 480B — coding specialist, 0.9s probe" },
|
||||
{ provider: "ollama_cloud", model: "deepseek-v3.1:671b", note: "cloud 671B — fast reasoning (1.0s probe)" },
|
||||
{ provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B — deep analysis (0.9s probe)" },
|
||||
{ provider: "ollama_cloud", model: "gpt-oss:120b", note: "cloud 120B — reliable workhorse (iter1 baseline)" },
|
||||
{ provider: "ollama_cloud", model: "qwen3.5:397b", note: "cloud 397B dense — deep final thinker (J 2026-04-24)" },
|
||||
// Free-tier rescue — different provider backbone, different quota.
|
||||
// Added 2026-04-24 after iter 5 hit repeated Ollama Cloud 502s on
|
||||
// kimi-k2:1t. These have lower parameter counts than the Ollama
|
||||
// Cloud rungs but high availability: if upstream is down, we still
|
||||
// land a review instead of giving up.
|
||||
{ provider: "openrouter", model: "openai/gpt-oss-120b:free", note: "OpenRouter free 120B — substantive rescue, 2.8s probe" },
|
||||
{ provider: "openrouter", model: "google/gemma-3-27b-it:free", note: "OpenRouter free 27B — fastest rescue, 1.4s probe" },
|
||||
{ provider: "ollama", model: "qwen3.5:latest", note: "local qwen3.5 — best local model per J (2026-04-24), last-resort if all cloud down" },
|
||||
// Dropped from the ladder after 2026-04-24 probe:
|
||||
// - kimi-k2.6 — not available on current tier (empty response)
|
||||
// - devstral-2:123b — displaced by qwen3-coder:480b (better coding specialist)
|
||||
// - minimax-m2.7 — 400 thinking tokens, 0 content output
|
||||
// - openrouter qwen3-coder:free / llama-3.3 / hermes-3 — provider errors
|
||||
// - openrouter minimax-m2.5:free — 45s timeout
|
||||
];
|
||||
|
||||
type Chunk = { id: string; text: string; embedding: number[]; origin: string; offset: number };
|
||||
@@ -203,10 +209,18 @@ function retrieveTopK(query_emb: number[], pool: Chunk[], k: number): Chunk[] {
|
||||
.map(x => ({ ...x.c, _score: x.score } as any));
|
||||
}
|
||||
|
||||
// Tree-split a large file: shard it, summarize each shard against
|
||||
// the review question, merge into a scratchpad. Uses cloud because
|
||||
// the summarization step needs quality > speed. Returns the
|
||||
// scratchpad (full-file distillation) and the cloud-call count.
|
||||
// Tree-split a large file: shard it, summarize each shard into a
|
||||
// running scratchpad, THEN run a reduce step that collapses the
|
||||
// scratchpad into one file-level synthesis with shard boundaries
|
||||
// stripped. Returns the synthesis (not the raw scratchpad) so the
|
||||
// final reviewer never sees "--- shard N ---" markers and can't
|
||||
// leak them into its review title.
|
||||
//
|
||||
// Phase 21 design (aibridge/src/tree_split.rs) with the map → reduce
|
||||
// shape. Earlier version concatenated per-shard digests directly into
|
||||
// the reviewer prompt, which led to kimi-k2:1t writing review titles
|
||||
// like "Forensic Audit Report – file.rs (shard 3)" because the shard
|
||||
// markers bled through. Fix 2026-04-24 adds the reduce step.
|
||||
async function treeSplitFile(
|
||||
filePath: string,
|
||||
content: string,
|
||||
@@ -216,20 +230,25 @@ async function treeSplitFile(
|
||||
const end = Math.min(i + FILE_SHARD_SIZE, content.length);
|
||||
shards.push({ from: i, to: end, text: content.slice(i, end) });
|
||||
}
|
||||
let scratchpad = "";
|
||||
|
||||
// MAP — each shard produces a digest that feeds the next shard's
|
||||
// context. Internal markers are kept to help the reducer align
|
||||
// overlapping observations across shards; they're stripped before
|
||||
// the reviewer sees anything.
|
||||
let workingScratchpad = "";
|
||||
let cloud_calls = 0;
|
||||
log(` tree-split: ${content.length} chars → ${shards.length} shards of ${FILE_SHARD_SIZE}`);
|
||||
for (const [si, shard] of shards.entries()) {
|
||||
const prompt = `You are summarizing ONE SHARD of a source file as part of a multi-shard review. File: ${filePath}. Shard ${si + 1}/${shards.length} (bytes ${shard.from}..${shard.to}).
|
||||
const prompt = `You are writing a SECTION of a full-file summary. File: ${filePath}. This is one piece (bytes ${shard.from}..${shard.to}) of a larger source file you are NOT seeing in its entirety right now.
|
||||
|
||||
─────── shard source ───────
|
||||
─────── source ───────
|
||||
${shard.text}
|
||||
─────── end shard ───────
|
||||
─────── end source ───────
|
||||
|
||||
Scratchpad of prior shards (if empty, this is shard 1):
|
||||
${scratchpad || "(empty)"}
|
||||
Prior-piece notes so far (if empty, this is the first piece):
|
||||
${workingScratchpad || "(empty)"}
|
||||
|
||||
Extract ONLY facts useful for reviewing this file against its PRD: function names + purposes, struct fields + types, invariants, edge cases, TODO markers, error-handling style. Under 150 words. No prose outside the extracted facts.`;
|
||||
Extract facts about the code in this piece that will help review the FULL file later: function + struct names with brief purpose, struct fields + types, invariants, TODOs, error-handling style, obvious gaps. Under 150 words. Flat facts only, no headings, no phrases like "this shard" or "in my section".`;
|
||||
const r = await chat({
|
||||
provider: "ollama_cloud",
|
||||
model: "gpt-oss:120b",
|
||||
@@ -238,10 +257,47 @@ Extract ONLY facts useful for reviewing this file against its PRD: function name
|
||||
});
|
||||
cloud_calls += 1;
|
||||
if (r.content) {
|
||||
scratchpad += `\n--- shard ${si + 1} (bytes ${shard.from}..${shard.to}) ---\n${r.content.trim()}`;
|
||||
// Keep internal alignment markers for the reducer; stripped later.
|
||||
workingScratchpad += `\n§${si + 1}§\n${r.content.trim()}`;
|
||||
}
|
||||
}
|
||||
return { scratchpad, shards: shards.length, cloud_calls };
|
||||
|
||||
// REDUCE — collapse the per-shard digests into one coherent
|
||||
// file-level summary. The reducer sees all digests at once and
|
||||
// produces a single narrative the reviewer can treat as "the file".
|
||||
// Shard markers are NOT in the output. This is what fixes the
|
||||
// shard-leakage bug that affected both the scrum and the auditor.
|
||||
const reducePrompt = `You are producing a SINGLE coherent summary of a Rust/TypeScript source file from a set of prior-piece notes. The notes were taken while walking the file in order but should be merged into one description of the whole file.
|
||||
|
||||
FILE: ${filePath} (${content.length} bytes, ${shards.length} pieces)
|
||||
|
||||
PRIOR-PIECE NOTES (markers §N§ delimit pieces but are artifacts — do not mention them):
|
||||
${workingScratchpad}
|
||||
|
||||
Produce ONE coherent file-level summary:
|
||||
1. One-sentence purpose of the file.
|
||||
2. Key public types / functions / constants (names + one-line purpose each).
|
||||
3. Known gaps, TODOs, or error-handling inconsistencies the notes surfaced.
|
||||
4. Obvious invariants the file relies on.
|
||||
|
||||
Do NOT say "piece 1" or "shard N" or "section" — present the summary as if you read the whole file at once. Under 600 words.`;
|
||||
|
||||
const reduced = await chat({
|
||||
provider: "ollama_cloud",
|
||||
model: "gpt-oss:120b",
|
||||
prompt: reducePrompt,
|
||||
max_tokens: 900,
|
||||
});
|
||||
cloud_calls += 1;
|
||||
const synthesis = reduced.content?.trim() ?? "";
|
||||
|
||||
// Safety: if the reducer returned thin output, fall back to the
|
||||
// raw scratchpad stripped of markers — better than nothing.
|
||||
const final = synthesis.length > 200
|
||||
? synthesis
|
||||
: workingScratchpad.replace(/§\d+§\n/g, "").trim();
|
||||
|
||||
return { scratchpad: final, shards: shards.length, cloud_calls };
|
||||
}
|
||||
|
||||
async function reviewFile(
|
||||
|
||||
Reference in New Issue
Block a user