6 Commits

Include renames

Author	SHA1	Message	Date
rootandClaude Opus 4.7	19a65b87e3	auditor: 3 fixes from Opus self-audit on 454da15 + tree-split deletion ... The post-fix audit on commit 454da15 produced a fresh BLOCK and re-flagged the dead tree-split as still dead. This commit lands the BLOCK fix and the deletion. LANDED: 1. kimi_architect.ts:113 BLOCK — MAX_TOKENS=128_000 exceeds Anthropic Opus 4.x's 32K output cap. Worked silently (Anthropic clamps server-side) but was technically invalid. Replaced single-default with `maxTokensFor(model)` returning per-model caps: claude-opus-* -> 32_000 (Opus extended-output) claude-haiku-* -> 8_192 (Haiku/Sonnet default) claude-sonnet-* -> 8_192 kimi-* -> 128_000 (reasoning_content needs headroom) gpt-5*/o-series -> 32_000 default -> 16_000 (conservative) LH_AUDITOR_KIMI_MAX_TOKENS env override still works (forces value regardless of model). 2. inference.ts dead-code removal — Opus flagged tree-split as still dead post-2026-04-27 mode-runner rebuild. Removed 156 lines: runCloudInference (lines 464-503) legacy /v1/chat caller treeSplitDiff (lines 547-619) shard-and-summarize fn callCloud (lines 621-651) helper for treeSplitDiff SHARD_MODEL const qwen3-coder:480b SHARD_CONCURRENCY const 6 DIFF_SHARD_SIZE const 4500 CURATION_THRESHOLD const 30000 No live callers — verified by grep before deletion. The mode runner's matrix retrieval against lakehouse_answers_v1 supplies the cross-PR context that tree-split was synthesizing from scratch. 3. inference.ts:38-49 stale comment about "curate via tree-split" replaced with current "matrix retrieval supplies cross-PR context" semantics. Block was already physically gone but the comment describing it remained, contradicting the actual code path. SKIPPED (defensible / minor): - WARN: outage sentinel TTL refresh on continued failure — intentional (refresh keeps cache valid while upstream is still down) - WARN: enrichment counts use Math.max — defensible (consensus enrichment IS the max of the three runs) - WARN: parseFindings regex eats severity into rationale on multi- paragraph inputs — minor, hasn't affected grounding rate - WARN: selectModel uses pre-truncation diff.length — defensible (promotion is "is this audit worth Opus", not "what does the model see") - INFO×3: static.ts state reset, parentStruct walk bound, appendMetrics 0-finding rows — all defensible per current intent Verification: bun build auditor/checks/{inference,kimi_architect}.ts compiles systemctl restart lakehouse-auditor.service active Net: -184 lines, +29 lines (155 net deletion). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 07:20:03 -05:00
rootandClaude Opus 4.7	454da15301	auditor + aibridge: 6 fixes from Opus 4.7 self-audit on PR #11 ... The kimi_architect auditor on commit 00c8408 ran with auto-promotion to claude-opus-4-7 (diff > 100k chars), produced 10 grounded findings, 1 BLOCK + 6 WARN + 3 INFO. This commit lands 6 of them; 3 are skipped (false positives or out-of-scope cleanup deferred). LANDED: 1. kimi_architect.ts:144 empty-parse cache poisoning. When parseFindings returns 0 findings (markdown shape changed, prompt too big, regex missed every block), the verdict was still persisted with empty findings, and the 24h TTL cache short-circuited every subsequent audit with a useless "0 findings" hit. Fix: only persist when findings.length > 0; metrics still appended unconditionally. 2. kimi_architect.ts:122 outage negative-cache. When callKimi throws (network error, gateway 502, rate limit), we returned skipFinding but didn't note the outage anywhere. Every audit cycle within the 24h TTL hammered the dead upstream. Fix: write a sentinel file `<verdict>.outage` on failure with 10-min TTL; future calls within that window short-circuit immediately. 3. kimi_architect.ts:331 mkdir(join(p, "..")) -> dirname(p). The "/.." idiom resolved correctly via Node path normalization but was non-idiomatic and breaks if the path ever has trailing dots. Both Haiku and Opus self-audits flagged it. 4. inference.ts:202 N=3 consensus latency double/triple-count. `totalLatencyMs += run.latency_ms` summed across THREE parallel `Promise.all` calls — wall-clock is bounded by the slowest, not the sum. Renamed to `maxLatencyMs` using `Math.max`. Telemetry now reports actual wall-clock instead of 3x reality. 5. continuation.rs:198,199,230,231 i64/u64 -> u32 saturating cast. `resp.tokens_evaluated as u32` truncates bits when source > u32::MAX instead of saturating. Fix: u32::try_from(...).unwrap_or(u32::MAX) wraps the cast in a real saturate. Applied to both the empty-retry loop and the structural-completion continuation loop. SKIPPED: - BLOCK at Cargo.lock:8911 "validator-not-in-workspace" — confabulation. The diff Opus saw was truncated mid-line; validator IS in Cargo.toml workspace members. Real-world MAX_DIFF_CHARS=180k edge case to watch as we feed more big diffs. - WARN at kimi_architect.ts:248 regex absolute-path edge case — minor, doesn't affect grounding rate observed so far. - INFO at inference.ts:606 "dead reconstruction loop" — Opus misread. The Promise.all worker fills `summaries[]`; the second loop builds a sequential `scratchpad` string from those. Two distinct operations, not redundant. Verification: bun build auditor/checks/{kimi_architect,inference}.ts compiles cargo check -p aibridge green cargo build --release -p gateway green systemctl restart lakehouse.service lakehouse-auditor.service active Next audit cycle (~90s after push) will run on the new diff and exercise the negative-cache + dirname + maxLatencyMs paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 07:10:43 -05:00
rootandClaude Opus 4.7	20a039c379	auditor: rebuild on mode runner + drop tree-split (use distillation substrate) ... Architectural simplification leveraging Phase 5 distillation work: the auditor no longer pre-extracts facts via per-shard summaries because lakehouse_answers_v1 (gold-standard prior PR audits + observer escalations corpus) supplies cross-PR context through the mode runner's matrix retrieval. Same signal, ~50× fewer cloud calls per audit. Per-audit cost: Before: 168 gpt-oss:120b shard summaries + 3 final inference calls After: 3 deepseek-v3.1:671b mode-runner calls (full retrieval included) Wall-clock on PR #11 (1.36MB diff): Before: ~25 minutes After: 88 seconds (3/3 consensus succeeded) Files: auditor/checks/inference.ts - Default MODEL kimi-k2:1t → deepseek-v3.1:671b. kimi-k2 is hitting sustained Ollama Cloud 500 ISE (verified via repeated trivial probes; multi-hour outage). deepseek is the proven drop-in from Phase 5 distillation acceptance testing. - Dropped treeSplitDiff invocation. Diff truncates to MAX_DIFF_CHARS and goes straight to /v1/mode/execute task_class=pr_audit; mode runner pulls cross-PR context from lakehouse_answers_v1 via matrix retrieval. SHARD_MODEL retained for legacy callCloud compatibility (default qwen3-coder:480b if it ever runs). - extractAndPersistFacts now reads from truncated diff (no scratchpad post-tree-split-removal). auditor/checks/static.ts - serde-derived struct exemption (commit 107a682 shipped this; this commit is the rest of the auditor rebuild it landed alongside) - multi-line template literal awareness in isInsideQuotedString — tracks backtick state across lines so todo!() inside docstrings doesn't trip BLOCK_PATTERNS. crates/gateway/src/v1/mode.rs - pr_audit native runner mode added to VALID_MODES + is_native_mode + flags_for_mode + framing_text. PrAudit framing produces strict JSON {claim_verdicts, unflagged_gaps} for the auditor to parse. config/modes.toml - pr_audit task class with default_model=deepseek-v3.1:671b and matrix_corpus=lakehouse_answers_v1. Documents kimi-k2 outage with link to the swap rationale. Real-data audit on PR #11 head 1b433a9 (which is the PR with all the distillation work + auditor rebuild itself): - Pipeline ran to completion (88s for inference; full audit ~3 min) - 3/3 consensus runs succeeded on deepseek-v3.1:671b - 156 findings: 12 block, 23 warn, 121 info - Block findings are legitimate signal: 12 reviewer claims like "Invariants enforced (proven by tests + real run):" that the truncated diff can't directly verify. The auditor is correctly flagging claim-vs-diff divergence — exactly its job. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-26 23:32:44 -05:00
profit	7c1745611a	Audit pipeline PR #9: determinism + fact extraction + verifier gate + KB stats + context injection (PR #9) ... Bundles PR #9's work for the audit pipeline: - N=3 consensus on cloud inference (gpt-oss:120b parallel) with qwen3-coder:480b tie-breaker - audit_discrepancies.jsonl logs N-run disagreements - scrum_master reviews route through llm_team fact extraction; source="scrum_review" - Verifier-gated persistence: drops INCORRECT, keeps UNVERIFIABLE/UNCHECKED; schema_version:2 - scrum_master_reviewed flag on accepted reviews - auditor/kb_stats.ts: on-demand observability script - claim_parser history/proof pattern class (verified-on-PR, was-flipping, the-proven-X) - claim_parser quoted-string guard (mirrors static.ts fix) - fact_extractor project context injection via docs/AUDITOR_CONTEXT.md - Fixed verifier-verdict parser to handle multiple gemma2 output formats Empirical: 3-run determinism test on unchanged PR #9 SHA showed 7/7 warn findings stable; block count oscillation eliminated; llm_team quality scores 8-9 on context-injected extract runs. See PR #9 for full run-by-run commit history.	2026-04-23 05:29:38 +00:00
profit	156dae6732	Auditor self-test branch: real-world pipelines + cohesion Phase C + KB index (PR #8) ... Bundles 12 commits validating the auditor + scrum_master architecture end-to-end: - enrich_prd_pipeline / hard_task_escalation / scrum_master_pipeline stress tests - Tree-split + scrum_reviews.jsonl + kb_query surfacing - Verdict → audit_lessons feedback loop (closed) - kb_index aggregator with confidence-based severity policy - 9-run + 5-run empirical tests proved the predictive-compounding property - Level 1 correction: temp=0 cloud inference for deterministic per-claim verdicts - audit_one.ts dry-run CLI - Fixes: static quoted-string guard, empirical-claim classification, symbol-resolver gate, repo-file size cap See PR #8 for run-by-run commit history.	2026-04-23 03:28:32 +00:00
profit	efc7b5ac44	Auditor: dynamic + inference checks ... auditor/checks/dynamic.ts — wraps runHybridFixture, maps layer results to Findings. Placeholder-style errors (404/unimplemented/ slice N) → info; other failures → warn. Always emits a summary finding with real numbers (shipped/placeholder phase counts + per- layer latency). Live-tested against current stack: 2 info findings, 0 warnings — all shipped layers actually work. auditor/checks/inference.ts — wraps the run_codereview reviewer pattern from llm_team_ui.py, adapted for claim-vs-diff verification. Calls /v1/chat provider=ollama_cloud model=gpt-oss:120b. Requests strict JSON response with claim_verdicts[] and unflagged_gaps[]. A strong claim marked "not backed" by cloud → BLOCK severity; moderate → warn; weak → info. Cloud-unreachable or unparseable-output → info (never blocks on the reviewer being down). Live-tested against PR #1 (this PR, 20 claims, 39KB diff): - 36.9s round-trip - 7 block + 23 warn + 2 info findings - gpt-oss:120b correctly flagged "Fully-functional auditor (tasks 1-9 complete)" as not-backed (only 6/10 tasks done at that commit) — accurate catch - Some false positives from the original 15KB truncation threshold (cloud missed gitea.ts, flagged "no Gitea client present") - Bumped MAX_DIFF_CHARS from 15000 to 40000 to fit the full PR diff in context; reviewer precision improves accordingly Tasks 5 + 6 completed. Remaining: #7 (KB query), #8 (verdict + Gitea poster), #9 (poller), #10 (end-to-end proof), #12 (upsert UPDATE-drops-doc_refs).	2026-04-22 03:54:18 -05:00