auditor: 3 fixes from Opus self-audit on 454da15 + tree-split deletion
lakehouse/auditor 14 blocking issues: cloud: claim not backed — "Verified end-to-end:"
lakehouse/auditor 14 blocking issues: cloud: claim not backed — "Verified end-to-end:"
The post-fix audit on commit 454da15 produced a fresh BLOCK and
re-flagged the dead tree-split as still dead. This commit lands the
BLOCK fix and the deletion.
LANDED:
1. kimi_architect.ts:113 BLOCK — MAX_TOKENS=128_000 exceeds Anthropic
Opus 4.x's 32K output cap. Worked silently (Anthropic clamps
server-side) but was technically invalid. Replaced single-default
with `maxTokensFor(model)` returning per-model caps:
claude-opus-* -> 32_000 (Opus extended-output)
claude-haiku-* -> 8_192 (Haiku/Sonnet default)
claude-sonnet-* -> 8_192
kimi-* -> 128_000 (reasoning_content needs headroom)
gpt-5*/o-series -> 32_000
default -> 16_000 (conservative)
LH_AUDITOR_KIMI_MAX_TOKENS env override still works (forces value
regardless of model).
2. inference.ts dead-code removal — Opus flagged tree-split as still
dead post-2026-04-27 mode-runner rebuild. Removed 156 lines:
runCloudInference (lines 464-503) legacy /v1/chat caller
treeSplitDiff (lines 547-619) shard-and-summarize fn
callCloud (lines 621-651) helper for treeSplitDiff
SHARD_MODEL const qwen3-coder:480b
SHARD_CONCURRENCY const 6
DIFF_SHARD_SIZE const 4500
CURATION_THRESHOLD const 30000
No live callers — verified by grep before deletion. The mode
runner's matrix retrieval against lakehouse_answers_v1 supplies
the cross-PR context that tree-split was synthesizing from scratch.
3. inference.ts:38-49 stale comment about "curate via tree-split"
replaced with current "matrix retrieval supplies cross-PR context"
semantics. Block was already physically gone but the comment
describing it remained, contradicting the actual code path.
SKIPPED (defensible / minor):
- WARN: outage sentinel TTL refresh on continued failure — intentional
(refresh keeps cache valid while upstream is still down)
- WARN: enrichment counts use Math.max — defensible (consensus
enrichment IS the max of the three runs)
- WARN: parseFindings regex eats severity into rationale on multi-
paragraph inputs — minor, hasn't affected grounding rate
- WARN: selectModel uses pre-truncation diff.length — defensible
(promotion is "is this audit worth Opus", not "what does the model
see")
- INFO×3: static.ts state reset, parentStruct walk bound,
appendMetrics 0-finding rows — all defensible per current intent
Verification:
bun build auditor/checks/{inference,kimi_architect}.ts compiles
systemctl restart lakehouse-auditor.service active
Net: -184 lines, +29 lines (155 net deletion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -69,18 +69,28 @@ function selectModel(diffLen: number): { provider: string; model: string; promot
|
||||
}
|
||||
return { provider: KIMI_PROVIDER, model: KIMI_MODEL, promoted: false };
|
||||
}
|
||||
// 128K — Kimi K2.6 emits reasoning_content that counts against this
|
||||
// budget but isn't surfaced in the OpenAI-shape `content` field.
|
||||
// Capping low silently produces empty content with finish_reason=length
|
||||
// when reasoning consumes the budget. 128K leaves ample room for both
|
||||
// reasoning and visible findings on any audit prompt we throw at it.
|
||||
// Override via LH_AUDITOR_KIMI_MAX_TOKENS only if you want to cap cost.
|
||||
// Model-aware max_tokens. Different upstream APIs cap at different
|
||||
// limits and reject requests that exceed them:
|
||||
// - Anthropic Opus 4.x: 32K output (with extended-output header)
|
||||
// - Anthropic Haiku 4.5: 8K output
|
||||
// - Kimi K2.6 (reasoning): 128K — needs headroom because
|
||||
// reasoning_content counts against the budget
|
||||
// - Default: 16K, conservative middle ground
|
||||
//
|
||||
// Bug fix 2026-04-27 (caught by Kimi's own self-audit): empty env var
|
||||
// like LH_AUDITOR_KIMI_MAX_TOKENS="" used to parse via Number("") → 0
|
||||
// because `??` only catches null/undefined. Use `||` so empty string,
|
||||
// 0, or NaN all fall back to the default.
|
||||
const MAX_TOKENS = Number(process.env.LH_AUDITOR_KIMI_MAX_TOKENS) || 128_000;
|
||||
// 2026-04-27 BLOCK from Opus self-audit: the prior single-default of
|
||||
// 128K worked silently (Anthropic clamps server-side) but was
|
||||
// technically invalid. Per-model caps make it explicit. Override via
|
||||
// LH_AUDITOR_KIMI_MAX_TOKENS to force a value (also fixes the empty-
|
||||
// env Number("") -> 0 trap by using `||` not `??`).
|
||||
const MAX_TOKENS_OVERRIDE = Number(process.env.LH_AUDITOR_KIMI_MAX_TOKENS) || 0;
|
||||
function maxTokensFor(model: string): number {
|
||||
if (MAX_TOKENS_OVERRIDE > 0) return MAX_TOKENS_OVERRIDE;
|
||||
if (model.startsWith("claude-opus")) return 32_000;
|
||||
if (model.startsWith("claude-haiku") || model.startsWith("claude-sonnet")) return 8_192;
|
||||
if (model.startsWith("kimi-")) return 128_000;
|
||||
if (model.startsWith("gpt-5") || model.startsWith("o1") || model.startsWith("o3") || model.startsWith("o4")) return 32_000;
|
||||
return 16_000;
|
||||
}
|
||||
|
||||
export interface KimiArchitectContext {
|
||||
pr_number: number;
|
||||
@@ -240,7 +250,7 @@ async function callKimi(prompt: string, provider: string, model: string): Promis
|
||||
provider,
|
||||
model,
|
||||
messages: [{ role: "user", content: prompt }],
|
||||
max_tokens: MAX_TOKENS,
|
||||
max_tokens: maxTokensFor(model),
|
||||
temperature: 0.2,
|
||||
});
|
||||
// curl via Bun.spawn — bypasses Bun fetch's ~300s intrinsic ceiling.
|
||||
|
||||
Reference in New Issue
Block a user