The auditor's own audit on commit ebd9ab7 produced 10 kimi_architect
findings; 2 are real correctness issues that this commit lands. The
other 8 are documented in the commit body as triaged-skip with
rationale (false flags, defensible by current intent, or edge cases).
LANDED:
1. auditor/index.ts — atomic state mutation on audit count.
`state.audit_count_per_pr[prKey] += 1` was held in memory until
the cycle's saveState at the end. If the daemon was killed mid-
cycle (SIGTERM, OOM, panic), the count was lost on restart while
the on-disk last_audited still showed the SHA as audited — the cap
silently leaked one audit per crash. Fix: persist state immediately
after each successful audit so the increment survives a crash.
saveState is idempotent + cheap (single JSON write); per-audit
cost negligible.
2. auditor/checks/inference.ts — Number-coerce mode runner telemetry.
`body?.latency_ms ?? 0` collapses null/undefined but passes through
non-numeric values (string, NaN, etc.) which would poison downstream
arithmetic in maxLatencyMs computation. Added a `num(v)` helper
that does `Number(v)` with `isFinite` fallback to 0. Applied to
latency_ms, enriched_prompt_chars, bug_fingerprints_count,
matrix_chunks_kept.
SKIPPED with rationale:
- WARN kimi_architect.ts:211 "metrics appended even on empty verdict":
this is intentional — observability shouldn't depend on whether
parseFindings succeeded. Comment in the file explicitly notes this.
- WARN static.ts:270 "escaped-backslash-before-backtick edge case":
real but extremely narrow (Rust raw strings with `\\\\\``). No
observed false positives in production audits; defer.
- INFO kimi_architect.ts:333 "sync existsSync in async fn": existsSync
is non-blocking syscall on Linux; not a real perf hit at audit
scale (10s of findings per call).
- INFO kimi_architect.ts:105 "audit_index modulo wraparound at 50+
audits": cap=3 means we never reach high counts on any PR.
- INFO inference.ts:366 "prompt injection delimiter risk": OUTPUT
FORMAT delimiter is in our prompt template, not user input; user
data goes inside content sections that don't contain the delimiter.
- WARN Cargo.lock:8739 "truth+validator no Cargo.toml in diff":
false flag — Cargo.toml IS in workspace members (lines 17-18 of
the workspace manifest).
- WARN config/modes.toml:1 "no schema validation": defensible — the
load path validates structure (deserialize_string_or_vec at
mode.rs:175) and falls back to safe default on parse error.
- INFO evidence_record.ts:124 "metadata accepts any keys": values are
constrained to `string | number | boolean`; key-name validation
not warranted for a domain-metadata field.
The 13 BLOCK-severity inference findings on this audit are all
"claim not backed" against historical commit messages from earlier
in the branch (8aa7ee9, bc698eb, 5bdd159, etc.). Those are
aspirational prose ("Verified end-to-end") that the deepseek
consensus can't verify from a static diff — known limitation, not
actionable as code fixes.
Verification:
bun build auditor/index.ts compiles
bun build auditor/checks/inference.ts compiles
systemctl restart lakehouse-auditor active
Cap remains active on PR #11 (3/3) — daemon will not audit this
fix-commit. Reset state.audit_count_per_pr.11 to verify the fixes
land clean on a fresh audit when ready.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The post-fix audit on commit 454da15 produced a fresh BLOCK and
re-flagged the dead tree-split as still dead. This commit lands the
BLOCK fix and the deletion.
LANDED:
1. kimi_architect.ts:113 BLOCK — MAX_TOKENS=128_000 exceeds Anthropic
Opus 4.x's 32K output cap. Worked silently (Anthropic clamps
server-side) but was technically invalid. Replaced single-default
with `maxTokensFor(model)` returning per-model caps:
claude-opus-* -> 32_000 (Opus extended-output)
claude-haiku-* -> 8_192 (Haiku/Sonnet default)
claude-sonnet-* -> 8_192
kimi-* -> 128_000 (reasoning_content needs headroom)
gpt-5*/o-series -> 32_000
default -> 16_000 (conservative)
LH_AUDITOR_KIMI_MAX_TOKENS env override still works (forces value
regardless of model).
2. inference.ts dead-code removal — Opus flagged tree-split as still
dead post-2026-04-27 mode-runner rebuild. Removed 156 lines:
runCloudInference (lines 464-503) legacy /v1/chat caller
treeSplitDiff (lines 547-619) shard-and-summarize fn
callCloud (lines 621-651) helper for treeSplitDiff
SHARD_MODEL const qwen3-coder:480b
SHARD_CONCURRENCY const 6
DIFF_SHARD_SIZE const 4500
CURATION_THRESHOLD const 30000
No live callers — verified by grep before deletion. The mode
runner's matrix retrieval against lakehouse_answers_v1 supplies
the cross-PR context that tree-split was synthesizing from scratch.
3. inference.ts:38-49 stale comment about "curate via tree-split"
replaced with current "matrix retrieval supplies cross-PR context"
semantics. Block was already physically gone but the comment
describing it remained, contradicting the actual code path.
SKIPPED (defensible / minor):
- WARN: outage sentinel TTL refresh on continued failure — intentional
(refresh keeps cache valid while upstream is still down)
- WARN: enrichment counts use Math.max — defensible (consensus
enrichment IS the max of the three runs)
- WARN: parseFindings regex eats severity into rationale on multi-
paragraph inputs — minor, hasn't affected grounding rate
- WARN: selectModel uses pre-truncation diff.length — defensible
(promotion is "is this audit worth Opus", not "what does the model
see")
- INFO×3: static.ts state reset, parentStruct walk bound,
appendMetrics 0-finding rows — all defensible per current intent
Verification:
bun build auditor/checks/{inference,kimi_architect}.ts compiles
systemctl restart lakehouse-auditor.service active
Net: -184 lines, +29 lines (155 net deletion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kimi_architect auditor on commit 00c8408 ran with auto-promotion
to claude-opus-4-7 (diff > 100k chars), produced 10 grounded
findings, 1 BLOCK + 6 WARN + 3 INFO. This commit lands 6 of them; 3
are skipped (false positives or out-of-scope cleanup deferred).
LANDED:
1. kimi_architect.ts:144 empty-parse cache poisoning. When parseFindings
returns 0 findings (markdown shape changed, prompt too big, regex
missed every block), the verdict was still persisted with empty
findings, and the 24h TTL cache short-circuited every subsequent
audit with a useless "0 findings" hit. Fix: only persist when
findings.length > 0; metrics still appended unconditionally.
2. kimi_architect.ts:122 outage negative-cache. When callKimi throws
(network error, gateway 502, rate limit), we returned skipFinding
but didn't note the outage anywhere. Every audit cycle within the
24h TTL hammered the dead upstream. Fix: write a sentinel file
`<verdict>.outage` on failure with 10-min TTL; future calls within
that window short-circuit immediately.
3. kimi_architect.ts:331 mkdir(join(p, "..")) -> dirname(p). The
"/.." idiom resolved correctly via Node path normalization but
was non-idiomatic and breaks if the path ever has trailing dots.
Both Haiku and Opus self-audits flagged it.
4. inference.ts:202 N=3 consensus latency double/triple-count.
`totalLatencyMs += run.latency_ms` summed across THREE parallel
`Promise.all` calls — wall-clock is bounded by the slowest, not
the sum. Renamed to `maxLatencyMs` using `Math.max`. Telemetry now
reports actual wall-clock instead of 3x reality.
5. continuation.rs:198,199,230,231 i64/u64 -> u32 saturating cast.
`resp.tokens_evaluated as u32` truncates bits when source > u32::MAX
instead of saturating. Fix: u32::try_from(...).unwrap_or(u32::MAX)
wraps the cast in a real saturate. Applied to both the empty-retry
loop and the structural-completion continuation loop.
SKIPPED:
- BLOCK at Cargo.lock:8911 "validator-not-in-workspace" — confabulation.
The diff Opus saw was truncated mid-line; validator IS in
Cargo.toml workspace members. Real-world MAX_DIFF_CHARS=180k
edge case to watch as we feed more big diffs.
- WARN at kimi_architect.ts:248 regex absolute-path edge case — minor,
doesn't affect grounding rate observed so far.
- INFO at inference.ts:606 "dead reconstruction loop" — Opus misread.
The Promise.all worker fills `summaries[]`; the second loop builds
a sequential `scratchpad` string from those. Two distinct
operations, not redundant.
Verification:
bun build auditor/checks/{kimi_architect,inference}.ts compiles
cargo check -p aibridge green
cargo build --release -p gateway green
systemctl restart lakehouse.service lakehouse-auditor.service active
Next audit cycle (~90s after push) will run on the new diff and
exercise the negative-cache + dirname + maxLatencyMs paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lakehouse/auditor 13 blocking issues: cloud: claim not backed — "Invariants enforced (proven by tests + real run):"
Architectural simplification leveraging Phase 5 distillation work:
the auditor no longer pre-extracts facts via per-shard summaries
because lakehouse_answers_v1 (gold-standard prior PR audits + observer
escalations corpus) supplies cross-PR context through the mode runner's
matrix retrieval. Same signal, ~50× fewer cloud calls per audit.
Per-audit cost:
Before: 168 gpt-oss:120b shard summaries + 3 final inference calls
After: 3 deepseek-v3.1:671b mode-runner calls (full retrieval included)
Wall-clock on PR #11 (1.36MB diff):
Before: ~25 minutes
After: 88 seconds (3/3 consensus succeeded)
Files:
auditor/checks/inference.ts
- Default MODEL kimi-k2:1t → deepseek-v3.1:671b. kimi-k2 is hitting
sustained Ollama Cloud 500 ISE (verified via repeated trivial
probes; multi-hour outage). deepseek is the proven drop-in from
Phase 5 distillation acceptance testing.
- Dropped treeSplitDiff invocation. Diff truncates to MAX_DIFF_CHARS
and goes straight to /v1/mode/execute task_class=pr_audit; mode
runner pulls cross-PR context from lakehouse_answers_v1 via
matrix retrieval. SHARD_MODEL retained for legacy callCloud
compatibility (default qwen3-coder:480b if it ever runs).
- extractAndPersistFacts now reads from truncated diff (no
scratchpad post-tree-split-removal).
auditor/checks/static.ts
- serde-derived struct exemption (commit 107a682 shipped this; this
commit is the rest of the auditor rebuild it landed alongside)
- multi-line template literal awareness in isInsideQuotedString —
tracks backtick state across lines so todo!() inside docstrings
doesn't trip BLOCK_PATTERNS.
crates/gateway/src/v1/mode.rs
- pr_audit native runner mode added to VALID_MODES + is_native_mode
+ flags_for_mode + framing_text. PrAudit framing produces strict
JSON {claim_verdicts, unflagged_gaps} for the auditor to parse.
config/modes.toml
- pr_audit task class with default_model=deepseek-v3.1:671b and
matrix_corpus=lakehouse_answers_v1. Documents kimi-k2 outage
with link to the swap rationale.
Real-data audit on PR #11 head 1b433a9 (which is the PR with all the
distillation work + auditor rebuild itself):
- Pipeline ran to completion (88s for inference; full audit ~3 min)
- 3/3 consensus runs succeeded on deepseek-v3.1:671b
- 156 findings: 12 block, 23 warn, 121 info
- Block findings are legitimate signal: 12 reviewer claims like
"Invariants enforced (proven by tests + real run):" that the
truncated diff can't directly verify. The auditor is correctly
flagging claim-vs-diff divergence — exactly its job.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
profit