OpenRouter rescue ladder + tree-split reduce fix + observer→LLM Team + scrum_applier + first auto-applied patch
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
## Infrastructure (scrum loop hardening)
crates/gateway/src/v1/openrouter.rs — new OpenRouter provider
Direct HTTPS to openrouter.ai/api/v1/chat/completions with OpenAI-compatible shape.
Key resolution: OPENROUTER_API_KEY env → /home/profit/.env → /root/llm_team_config.json
(shares LLM Team UI's quota). Added after iter 5 hit repeated Ollama Cloud 502s on
kimi-k2:1t — different provider backbone as rescue rung. Unit tests pin the URL
stripping and OpenAI wire shape.
crates/gateway/src/v1/mod.rs + main.rs
Added `"openrouter" | "openrouter_free"` arm to /v1/chat dispatch.
V1State.openrouter_key loaded at startup via openrouter::resolve_openrouter_key()
mirroring the Ollama Cloud pattern. Startup log:
"v1: OpenRouter key loaded — /v1/chat provider=openrouter enabled"
tests/real-world/scrum_master_pipeline.ts
* 9-rung ladder — kimi-k2:1t → qwen3-coder:480b → deepseek-v3.1:671b →
mistral-large-3:675b → gpt-oss:120b → qwen3.5:397b → openrouter/gpt-oss-120b:free
→ openrouter/gemma-3-27b-it:free → local qwen3.5:latest.
Added qwen3-coder:480b as rung 2 after live probes confirmed it rescues
kimi-k2:1t 502s cleanly (0.9s latency, substantive reviews).
Dropped devstral-2 (displaced by qwen3-coder); dropped kimi-k2.6 (not available);
dropped minimax-m2.7 (returned 0 chars / 400 thinking tokens).
Local fallback promoted qwen3.5:latest per J's direction 2026-04-24.
* MAX_ATTEMPTS bumped 6 → 9 to accommodate the rescue tier.
* Tree-split scratchpad fixed — was concatenating shard markers directly
into the reviewer input, causing kimi-k2:1t to write titles like
"Forensic Audit Report – file.rs (shard 3)". Now uses internal §N§
markers during accumulation and runs a proper reduce step that
collapses per-shard digests into ONE coherent file-level synthesis
with markers stripped. Matches the Phase 21 aibridge::tree_split
map→reduce design. Fallback to stripped scratchpad if reducer returns thin.
tests/real-world/scrum_applier.ts — NEW (737 lines)
The auto-apply pipeline. Reads scrum_reviews.jsonl, filters rows where
gradient_tier ∈ {auto, dry_run} AND confidence_avg ≥ MIN_CONF (default 90),
asks the reviewer model for concrete old_string/new_string patch JSON,
applies via text replacement, runs cargo check after each file, commits
if green and reverts if red. Deny-list: /etc/, config/, ops/, auditor/,
docs/, data/, mcp-server/, ui/, sidecar/, scripts/. Hard caps: per-patch
confidence ≥ MIN_CONF, old_string must be exactly unique, max 20 lines per
patch. Never runs on main without explicit LH_APPLIER_BRANCH override.
Audit trail in data/_kb/auto_apply.jsonl.
Empirical behavior (dry-run over iter 4 reviews):
5 eligible files → 1 green commit-ready, 2 build-red reverts, 2 all-rejected
The build-green gate caught 2 bad patches before they'd have merged.
mcp-server/observer.ts — LLM Team code_review escalation
When a sig_hash accumulates ≥3 failures (ESCALATION_THRESHOLD), fire-and-forget
POST /api/run?mode=code_review at localhost:5000 with the failure cluster context.
Parses facts/entities/relationships/file_hints from the response. Writes to a
new data/_kb/observer_escalations.jsonl surface. Answers J's vision of the
observer triggering richer LLM Team calls when failures pile up.
Non-blocking: runs parallel to existing qwen2.5 analyzer, never replaces it.
Tracks escalated sig_hashes in a session-local Set to avoid re-hammering
LLM Team when a cluster persists across observer cycles.
crates/aibridge/src/context.rs
First auto-applied patch produced by scrum_applier.ts (dry-run path —
applier writes files in dry-run mode but doesn't commit; bug noted for
iter 6 fix). Adds #[deprecated] annotation to the inline estimate_tokens
helper pointing callers to the centralized shared::model_matrix::ModelMatrix
entry point (P21-002 — duplicate token-estimator surfaces). Cargo check
passes with the annotation (verified by applier's own build gate).
## Visual Control Plane (UI)
ui/server.ts — Bun.serve on :3950 with /data/* fan-out:
/data/services, /data/reviews, /data/metrics, /data/trust, /data/overrides,
/data/findings, /data/outcomes, /data/audit_facts, /data/file/:path,
/data/refactor_signals, /data/search?q=, /data/signal_classes,
/data/logs/:svc (journalctl tail per systemd unit), /data/scrum_log.
Bug fix: tryFetch always attempts JSON.parse before falling back to text
— observer's Bun.serve returns JSON without application/json content-type,
which was displaying stats as a raw string ("0 ops" on map) before.
ui/index.html + ui.css — dark neo-brutalist shell. 6 views:
MAP (D3 force-graph + overlays) / TRACE (per-file iter history) /
TRAJECTORY (signal-class cards + refactor-signals table + reverse-index
search box) / METRICS (every card has SOURCE + GOOD lines explaining
where the number comes from and what target trajectory means) /
KB (card grid with tooltips on every field) / CONSOLE (per-service
journalctl tabs).
ui/ui.js — polling client, D3 wiring, signal-class panel, refactor-signals
table, reverse-index search, per-service console tabs. Bug fix:
renderNodeContext had Object.entries() iterating string characters when
/health returned a plain string — now guards with typeof check so
"lakehouse ok" renders as one row instead of "0 l / 1 a / 2 k / ...".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,7 +26,7 @@ const SIDECAR = "http://localhost:3200";
|
||||
const CHUNK_SIZE = 800;
|
||||
const CHUNK_OVERLAP = 120;
|
||||
const TOP_K_CONTEXT = 5;
|
||||
const MAX_ATTEMPTS = 6;
|
||||
const MAX_ATTEMPTS = 9;
|
||||
// Files larger than this get tree-split instead of truncated. Fixes the
|
||||
// 6KB false-positive class (model claiming a field is "missing" when
|
||||
// it exists past the context cutoff).
|
||||
@@ -87,21 +87,27 @@ const TARGET_FILES: string[] = process.env.LH_SCRUM_FILES
|
||||
// Local fallbacks kept for cloud-down scenarios.
|
||||
// Hot-path pipelines (scenario.ts / execution_loop) stay local per
|
||||
// Phase 20 t1_hot — this scrum is not hot path.
|
||||
const LADDER: Array<{ provider: "ollama" | "ollama_cloud"; model: string; note: string }> = [
|
||||
{ provider: "ollama_cloud", model: "kimi-k2:1t", note: "cloud 1T — biggest available, 1.4s probe" },
|
||||
{ provider: "ollama_cloud", model: "deepseek-v3.1:671b", note: "cloud 671B — fast reasoning (1.0s probe)" },
|
||||
{ provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B — deep analysis (0.9s probe)" },
|
||||
{ provider: "ollama_cloud", model: "gpt-oss:120b", note: "cloud 120B — reliable workhorse (iter1 baseline)" },
|
||||
{ provider: "ollama_cloud", model: "devstral-2:123b", note: "cloud 123B — coding specialist" },
|
||||
// qwen3.5:397b is the deep final thinker — J's note 2026-04-24:
|
||||
// "qwen3.5 is really smart maybe the last call use that one".
|
||||
// When every other cloud model has produced thin output, this dense
|
||||
// 397B reviewer is the one that tends to push through. Keeping it
|
||||
// LAST in cloud tier, before the local fallback.
|
||||
{ provider: "ollama_cloud", model: "qwen3.5:397b", note: "cloud 397B dense — last-ditch smart reviewer" },
|
||||
{ provider: "ollama", model: "gpt-oss:20b", note: "local 20B — cloud-down fallback" },
|
||||
// kimi-k2.6 removed 2026-04-24: probe returned empty (not available
|
||||
// on current tier). Keeping note for when pro tier upgrade lands.
|
||||
const LADDER: Array<{ provider: "ollama" | "ollama_cloud" | "openrouter"; model: string; note: string }> = [
|
||||
{ provider: "ollama_cloud", model: "kimi-k2:1t", note: "cloud 1T — biggest available, 1.4s probe" },
|
||||
{ provider: "ollama_cloud", model: "qwen3-coder:480b", note: "cloud 480B — coding specialist, 0.9s probe" },
|
||||
{ provider: "ollama_cloud", model: "deepseek-v3.1:671b", note: "cloud 671B — fast reasoning (1.0s probe)" },
|
||||
{ provider: "ollama_cloud", model: "mistral-large-3:675b", note: "cloud 675B — deep analysis (0.9s probe)" },
|
||||
{ provider: "ollama_cloud", model: "gpt-oss:120b", note: "cloud 120B — reliable workhorse (iter1 baseline)" },
|
||||
{ provider: "ollama_cloud", model: "qwen3.5:397b", note: "cloud 397B dense — deep final thinker (J 2026-04-24)" },
|
||||
// Free-tier rescue — different provider backbone, different quota.
|
||||
// Added 2026-04-24 after iter 5 hit repeated Ollama Cloud 502s on
|
||||
// kimi-k2:1t. These have lower parameter counts than the Ollama
|
||||
// Cloud rungs but high availability: if upstream is down, we still
|
||||
// land a review instead of giving up.
|
||||
{ provider: "openrouter", model: "openai/gpt-oss-120b:free", note: "OpenRouter free 120B — substantive rescue, 2.8s probe" },
|
||||
{ provider: "openrouter", model: "google/gemma-3-27b-it:free", note: "OpenRouter free 27B — fastest rescue, 1.4s probe" },
|
||||
{ provider: "ollama", model: "qwen3.5:latest", note: "local qwen3.5 — best local model per J (2026-04-24), last-resort if all cloud down" },
|
||||
// Dropped from the ladder after 2026-04-24 probe:
|
||||
// - kimi-k2.6 — not available on current tier (empty response)
|
||||
// - devstral-2:123b — displaced by qwen3-coder:480b (better coding specialist)
|
||||
// - minimax-m2.7 — 400 thinking tokens, 0 content output
|
||||
// - openrouter qwen3-coder:free / llama-3.3 / hermes-3 — provider errors
|
||||
// - openrouter minimax-m2.5:free — 45s timeout
|
||||
];
|
||||
|
||||
type Chunk = { id: string; text: string; embedding: number[]; origin: string; offset: number };
|
||||
@@ -203,10 +209,18 @@ function retrieveTopK(query_emb: number[], pool: Chunk[], k: number): Chunk[] {
|
||||
.map(x => ({ ...x.c, _score: x.score } as any));
|
||||
}
|
||||
|
||||
// Tree-split a large file: shard it, summarize each shard against
|
||||
// the review question, merge into a scratchpad. Uses cloud because
|
||||
// the summarization step needs quality > speed. Returns the
|
||||
// scratchpad (full-file distillation) and the cloud-call count.
|
||||
// Tree-split a large file: shard it, summarize each shard into a
|
||||
// running scratchpad, THEN run a reduce step that collapses the
|
||||
// scratchpad into one file-level synthesis with shard boundaries
|
||||
// stripped. Returns the synthesis (not the raw scratchpad) so the
|
||||
// final reviewer never sees "--- shard N ---" markers and can't
|
||||
// leak them into its review title.
|
||||
//
|
||||
// Phase 21 design (aibridge/src/tree_split.rs) with the map → reduce
|
||||
// shape. Earlier version concatenated per-shard digests directly into
|
||||
// the reviewer prompt, which led to kimi-k2:1t writing review titles
|
||||
// like "Forensic Audit Report – file.rs (shard 3)" because the shard
|
||||
// markers bled through. Fix 2026-04-24 adds the reduce step.
|
||||
async function treeSplitFile(
|
||||
filePath: string,
|
||||
content: string,
|
||||
@@ -216,20 +230,25 @@ async function treeSplitFile(
|
||||
const end = Math.min(i + FILE_SHARD_SIZE, content.length);
|
||||
shards.push({ from: i, to: end, text: content.slice(i, end) });
|
||||
}
|
||||
let scratchpad = "";
|
||||
|
||||
// MAP — each shard produces a digest that feeds the next shard's
|
||||
// context. Internal markers are kept to help the reducer align
|
||||
// overlapping observations across shards; they're stripped before
|
||||
// the reviewer sees anything.
|
||||
let workingScratchpad = "";
|
||||
let cloud_calls = 0;
|
||||
log(` tree-split: ${content.length} chars → ${shards.length} shards of ${FILE_SHARD_SIZE}`);
|
||||
for (const [si, shard] of shards.entries()) {
|
||||
const prompt = `You are summarizing ONE SHARD of a source file as part of a multi-shard review. File: ${filePath}. Shard ${si + 1}/${shards.length} (bytes ${shard.from}..${shard.to}).
|
||||
const prompt = `You are writing a SECTION of a full-file summary. File: ${filePath}. This is one piece (bytes ${shard.from}..${shard.to}) of a larger source file you are NOT seeing in its entirety right now.
|
||||
|
||||
─────── shard source ───────
|
||||
─────── source ───────
|
||||
${shard.text}
|
||||
─────── end shard ───────
|
||||
─────── end source ───────
|
||||
|
||||
Scratchpad of prior shards (if empty, this is shard 1):
|
||||
${scratchpad || "(empty)"}
|
||||
Prior-piece notes so far (if empty, this is the first piece):
|
||||
${workingScratchpad || "(empty)"}
|
||||
|
||||
Extract ONLY facts useful for reviewing this file against its PRD: function names + purposes, struct fields + types, invariants, edge cases, TODO markers, error-handling style. Under 150 words. No prose outside the extracted facts.`;
|
||||
Extract facts about the code in this piece that will help review the FULL file later: function + struct names with brief purpose, struct fields + types, invariants, TODOs, error-handling style, obvious gaps. Under 150 words. Flat facts only, no headings, no phrases like "this shard" or "in my section".`;
|
||||
const r = await chat({
|
||||
provider: "ollama_cloud",
|
||||
model: "gpt-oss:120b",
|
||||
@@ -238,10 +257,47 @@ Extract ONLY facts useful for reviewing this file against its PRD: function name
|
||||
});
|
||||
cloud_calls += 1;
|
||||
if (r.content) {
|
||||
scratchpad += `\n--- shard ${si + 1} (bytes ${shard.from}..${shard.to}) ---\n${r.content.trim()}`;
|
||||
// Keep internal alignment markers for the reducer; stripped later.
|
||||
workingScratchpad += `\n§${si + 1}§\n${r.content.trim()}`;
|
||||
}
|
||||
}
|
||||
return { scratchpad, shards: shards.length, cloud_calls };
|
||||
|
||||
// REDUCE — collapse the per-shard digests into one coherent
|
||||
// file-level summary. The reducer sees all digests at once and
|
||||
// produces a single narrative the reviewer can treat as "the file".
|
||||
// Shard markers are NOT in the output. This is what fixes the
|
||||
// shard-leakage bug that affected both the scrum and the auditor.
|
||||
const reducePrompt = `You are producing a SINGLE coherent summary of a Rust/TypeScript source file from a set of prior-piece notes. The notes were taken while walking the file in order but should be merged into one description of the whole file.
|
||||
|
||||
FILE: ${filePath} (${content.length} bytes, ${shards.length} pieces)
|
||||
|
||||
PRIOR-PIECE NOTES (markers §N§ delimit pieces but are artifacts — do not mention them):
|
||||
${workingScratchpad}
|
||||
|
||||
Produce ONE coherent file-level summary:
|
||||
1. One-sentence purpose of the file.
|
||||
2. Key public types / functions / constants (names + one-line purpose each).
|
||||
3. Known gaps, TODOs, or error-handling inconsistencies the notes surfaced.
|
||||
4. Obvious invariants the file relies on.
|
||||
|
||||
Do NOT say "piece 1" or "shard N" or "section" — present the summary as if you read the whole file at once. Under 600 words.`;
|
||||
|
||||
const reduced = await chat({
|
||||
provider: "ollama_cloud",
|
||||
model: "gpt-oss:120b",
|
||||
prompt: reducePrompt,
|
||||
max_tokens: 900,
|
||||
});
|
||||
cloud_calls += 1;
|
||||
const synthesis = reduced.content?.trim() ?? "";
|
||||
|
||||
// Safety: if the reducer returned thin output, fall back to the
|
||||
// raw scratchpad stripped of markers — better than nothing.
|
||||
const final = synthesis.length > 200
|
||||
? synthesis
|
||||
: workingScratchpad.replace(/§\d+§\n/g, "").trim();
|
||||
|
||||
return { scratchpad: final, shards: shards.length, cloud_calls };
|
||||
}
|
||||
|
||||
async function reviewFile(
|
||||
|
||||
Reference in New Issue
Block a user