Apply B1+B2+B3+B4 from 2026-04-30 cross-lineage scrum
Cross-lineage scrum (Opus 4.7 / Kimi K2.6 / Qwen3-coder via chatd's
/v1/chat) on the harness's first 4 commits surfaced 5 real bugs;
this commit lands the 4 inside the LLM/validator stack. B5 (scanner
skip-list semantics) ships separately as it changes scan behavior
on every target repo.
B1 (Kimi BLOCK + Opus WARN convergent) — internal/validators:
evidencePresent had two flaws: (1) cursor advanced on match in the
trim-line fallback, breaking same-line repeated matches AND skipping
not-yet-considered lines so out-of-order evidence spuriously failed;
(2) strings.Contains on a single `}` trim-matched any closing brace
in the file, defeating the "evidence quotes real text" contract.
Fix: trivial-evidence guard FIRST (reject anything <4 non-whitespace
chars) + per-line search no longer advances a cursor. New regression
test TestEvidencePresent_RejectsTrivialMatches covers `}`, `{`, `)`,
empty, and out-of-order multi-line evidence (which now passes —
order isn't part of the contract).
B2 (Kimi WARN + Opus WARN convergent) — internal/pipeline:
WriteJSON error for rejected-findings.json was swallowed with
`if err == nil`, so a write failure left the validation phase
reporting status="ok" while the audit trail vanished. Mirror the
validated-findings branch: surface the error in
validatePhase.Errors + bump status to degraded + ExitCode=66.
B3 (Kimi BLOCK + Opus BLOCK convergent) — internal/llm/ollama.go:
HealthCheck.basic_prompt_ok was set to true on ANY non-empty
response, so a model emitting `<think>...` traces or apologies
passed silently. Now requires the response to contain "OK"
(uppercase, substring). Substring rather than equality lets minor
whitespace/punctuation variations through (some models add a
trailing period). Errors now record what the model actually said
when it fails the check.
B4 (Opus BLOCK only — same class as today's chatd Anthropic-temp
fix) — internal/llm/ollama.go: chatBody had `if opts.Temperature != 0`
which silently dropped Temperature=0 from the request, so HealthCheck
+ Reviewer (both pass Temperature=0 expecting determinism) actually
ran at Ollama's ~0.8 default. Always forward Temperature now. The
two callers always set explicit values, so "0 means 0" is correct;
if a future caller wants Ollama's default they'll switch
CompleteOptions.Temperature to *float64 like chatd did this morning.
Verified end-to-end: insecure-repo + --enable-llm still produces 25
confirmed findings (16 static + 9 LLM), 0 rejected. Validator unit
tests: 11 pass (added TestEvidencePresent_RejectsTrivialMatches).
Same-day-as-shipping scrum, same-day-as-shipping fixes. The
convergent-≥2 gate caught 3 of these; the 4th was Opus-only but
verified by reading the code (same idiom as today's chatd bug).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
4dc53c5798
commit
305f5b9ac0
+20
-5
@@ -97,11 +97,18 @@ func (o *OllamaProvider) HealthCheck(ctx context.Context, primary, fallback stri
|
||||
return st
|
||||
}
|
||||
|
||||
// 2. Basic completion
|
||||
// 2. Basic completion. Scrum fix B3 (Kimi BLOCK + Opus BLOCK,
|
||||
// 2026-04-30): checks that the response actually contains "OK"
|
||||
// (case-insensitive, substring) — pre-fix accepted any non-empty
|
||||
// string, so a thinking-model's `<think>...` trace or an apology
|
||||
// passed silently. Substring rather than equality because some
|
||||
// models surround the answer with whitespace or a trailing period.
|
||||
if got, err := o.Complete(ctx, probeModel, "Reply with the single word: OK", CompleteOptions{Temperature: 0, MaxTokens: 8, TimeoutSeconds: 30}); err != nil {
|
||||
st.Errors = append(st.Errors, "basic prompt: "+err.Error())
|
||||
} else if strings.TrimSpace(got) != "" {
|
||||
} else if upper := strings.ToUpper(strings.TrimSpace(got)); upper == "OK" || strings.Contains(upper, "OK") {
|
||||
st.BasicPromptOK = true
|
||||
} else {
|
||||
st.Errors = append(st.Errors, fmt.Sprintf("basic prompt: expected 'OK', got %q", abbrev(got, 80)))
|
||||
}
|
||||
|
||||
// 3. JSON-mode completion
|
||||
@@ -140,9 +147,17 @@ func (o *OllamaProvider) CompleteJSON(ctx context.Context, model, prompt string,
|
||||
|
||||
func (o *OllamaProvider) chatBody(model, prompt string, opts CompleteOptions, jsonMode bool) map[string]any {
|
||||
options := map[string]any{}
|
||||
if opts.Temperature != 0 {
|
||||
options["temperature"] = opts.Temperature
|
||||
}
|
||||
// Scrum fix B4 (Opus BLOCK, 2026-04-30): always forward the
|
||||
// caller-supplied Temperature, including 0. Pre-fix `if != 0`
|
||||
// silently dropped the field for callers wanting deterministic
|
||||
// generation, so Ollama's ~0.8 default applied to the JSON
|
||||
// probe + every reviewer call. CompleteOptions.Temperature is
|
||||
// still float64 (not *float64) — the harness's two callers
|
||||
// (HealthCheck, Reviewer) always set it explicitly, so "0
|
||||
// means 0" is the right semantic. If a future caller wants
|
||||
// "use Ollama default", they can set MaxTokens=0 + delete the
|
||||
// option (or we'll switch to *float64 like chatd did).
|
||||
options["temperature"] = opts.Temperature
|
||||
if opts.MaxTokens > 0 {
|
||||
options["num_predict"] = opts.MaxTokens
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user