profit/golangLAKEHOUSE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

matrix: strong-model downgrade gate — SPEC §3.4 component 4 of 5

Browse Source 
Pure-Go port of mode.rs::execute's pass5 downgrade gate (Rust
2026-04-26). Adds POST /v1/matrix/downgrade endpoint via matrixd.

The gate captures the pass5 finding: composing matrix corpora into
codereview_lakehouse on a strong model LOST 5/5 head-to-head reps
against matrix-free codereview_isolation on grok-4.1-fast (p=0.031).
Strong models have enough native capacity that bug fingerprints +
adversarial framing + file content carry them; matrix chunks
displace depth-of-analysis.

Logic (matches Rust mode.rs:614-632):
  if mode == codereview_lakehouse
     && !forced_mode
     && !LH_FORCE_FULL_ENRICHMENT
     && !is_weak_model(model)
  → flip to codereview_isolation, record downgraded_from

is_weak_model captures the empirical weak-list:
  - `:free` suffix or `:free/` infix (OpenRouter free tier)
  - qwen3.5:latest, qwen3:latest (local last-resort rungs)
  - everything else → strong by default

Tests:
  - 3 unit tests in internal/matrix/downgrade_test.go: IsWeakModel
    coverage, MaybeDowngrade truth table (5 rows), forced-mode
    precedence (forced beats every other bypass)
  - scripts/downgrade_smoke.sh: 6 assertions through gateway covering
    all 5 truth-table rows + empty-mode 400

14-smoke regression sweep all green (D1-D6, G1, G1P, G2,
storaged_cap, pathway, matrix, relevance, downgrade).

SPEC §3.4 progress: 4 of 5 components shipped (corpus builders,
multi-corpus retrieve+merge, relevance filter, downgrade gate).
Last component is learning-loop integration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-29 19:17:55 -05:00
co-authored by Claude Opus 4.7
parent 9588bd82ae
commit 3968ec8a7b
4 changed files with 428 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/matrix/downgrade.go
+137
View File
@@ -0,0 +1,137 @@
package matrix
// Strong-model auto-downgrade gate. Port of mode.rs::execute's
// downgrade block (Rust system, 2026-04-26 pass5).
//
// What it does: if the caller resolves `codereview_lakehouse` against
// a strong model and didn't force the mode, flip to
// `codereview_isolation` so we don't pollute the prompt with matrix
// chunks the model demonstrably does better without.
//
// Why: pass5 variance test on x-ai/grok-4.1-fast — composing matrix
// corpora into codereview_lakehouse LOST 5/5 head-to-head reps
// against matrix-free codereview_isolation, p=0.031. Strong models
// have enough native capacity that bug fingerprints + adversarial
// framing + file content carry them; matrix chunks displace
// depth-of-analysis.
//
// Defaults: assume "strong" (downgrade matrix off). The explicit
// IsWeakModel predicate keeps the weak-list small — anything
// matching `:free` (OpenRouter free tier) or the local last-resort
// rungs (qwen3.5/qwen3) stays on the full lakehouse path where
// matrix demonstrably helped during the 2026-04-26 free-tier
// bake-off.
import (
"os"
"strings"
)
// Mode constants — exported so callers don't string-literal them.
const (
ModeCodeReviewLakehouse = "codereview_lakehouse"
ModeCodeReviewIsolation = "codereview_isolation"
)
// EnvForceFullEnrichment is the env var that bypasses the gate for
// diagnostic runs ("LH_FORCE_FULL_ENRICHMENT=1" or "true").
const EnvForceFullEnrichment = "LH_FORCE_FULL_ENRICHMENT"
// IsWeakModel returns true for models matrix-corpus composition
// demonstrably helped during the 2026-04-26 pass5 bake-off. Strong
// models (default) get matrix dropped to avoid the "composed lost
// 5/5 vs isolation" effect.
//
// Weak signals:
// - `:free` suffix (OpenRouter free tier, e.g. `gpt-oss-120b:free`)
// - `:free/` infix (handles routing-prefixed names like `or:free/x`)
// - `qwen3.5:latest` / `qwen3:latest` — local last-resort rung
//
// Add new weak models by extending this function alongside variance
// data that justifies it.
func IsWeakModel(model string) bool {
if strings.HasSuffix(model, ":free") || strings.Contains(model, ":free/") {
return true
}
switch model {
case "qwen3.5:latest", "qwen3:latest":
return true
}
return false
}
// DowngradeInput is what MaybeDowngrade evaluates.
//
// ForcedMode: caller explicitly set their mode (mirrors Rust's
// req.force_mode.is_some()) — treated as opt-in to the chosen mode,
// skips the downgrade. Experiments need exact-mode control.
//
// ForceFullOverride: the LH_FORCE_FULL_ENRICHMENT escape hatch —
// usually populated from the env var via NewDowngradeInputFromEnv,
// but the field is explicit so callers can pass it from a config or
// test deterministically.
type DowngradeInput struct {
Mode string
Model string
ForcedMode bool
ForceFullOverride bool
}
// DowngradeDecision is the output. DowngradedFrom is non-empty
// only when a downgrade fired — callers should record it for audit
// (matches the Rust EnrichmentSources.downgraded_from field).
//
// Reason is a short human-readable string for logs/responses;
// useful for debugging "why did/didn't the gate fire."
type DowngradeDecision struct {
Mode string `json:"mode"`
DowngradedFrom string `json:"downgraded_from,omitempty"`
Reason string `json:"reason"`
}
// MaybeDowngrade applies the strong-model auto-downgrade gate.
// Pure function; no env reads. For env-driven callers see
// NewDowngradeInputFromEnv.
func MaybeDowngrade(in DowngradeInput) DowngradeDecision {
out := DowngradeDecision{Mode: in.Mode}
if in.Mode != ModeCodeReviewLakehouse {
out.Reason = "mode is not " + ModeCodeReviewLakehouse + "; gate not applicable"
return out
}
if in.ForcedMode {
out.Reason = "caller forced mode; skip downgrade"
return out
}
if in.ForceFullOverride {
out.Reason = EnvForceFullEnrichment + " bypass"
return out
}
if IsWeakModel(in.Model) {
out.Reason = "weak model; matrix composition demonstrably helped (2026-04-26 free-tier bake-off)"
return out
}
// Downgrade fires.
out.Mode = ModeCodeReviewIsolation
out.DowngradedFrom = ModeCodeReviewLakehouse
out.Reason = "strong model; matrix composes anti-additively (pass5: composed lost 5/5 vs isolation on grok-4.1-fast, p=0.031)"
return out
}
// NewDowngradeInputFromEnv is a convenience that reads
// LH_FORCE_FULL_ENRICHMENT from the process environment and returns
// a populated DowngradeInput. Most production callers want this;
// tests should construct DowngradeInput directly to avoid env
// pollution.
func NewDowngradeInputFromEnv(mode, model string, forcedMode bool) DowngradeInput {
return DowngradeInput{
Mode: mode,
Model: model,
ForcedMode: forcedMode,
ForceFullOverride: envForceFullEnrichment(),
}
}
func envForceFullEnrichment() bool {
v := strings.ToLower(strings.TrimSpace(os.Getenv(EnvForceFullEnrichment)))
return v == "1" || v == "true"
}
internal/matrix/downgrade_test.go
+100
View File
@@ -0,0 +1,100 @@
package matrix
import "testing"
func TestIsWeakModel(t *testing.T) {
cases := []struct {
model string
weak bool
}{
// :free suffix → weak
{"openai/gpt-4o:free", true},
{"meta-llama/llama-3-8b:free", true},
// :free/ infix (routing-prefixed names)
{"openrouter:free/anthropic/claude-3.5-sonnet", true},
// Local last-resort rungs
{"qwen3.5:latest", true},
{"qwen3:latest", true},
// Strong by default
{"x-ai/grok-4.1-fast", false},
{"opencode/claude-opus-4-7", false},
{"openai/gpt-5", false},
{"qwen3-coder:480b", false}, // not the :latest tag
{"", false},
}
for _, c := range cases {
got := IsWeakModel(c.model)
if got != c.weak {
t.Errorf("IsWeakModel(%q): want %v, got %v", c.model, c.weak, got)
}
}
}
func TestMaybeDowngrade_TruthTable(t *testing.T) {
cases := []struct {
name string
in DowngradeInput
want DowngradeDecision
}{
{
name: "downgrade fires: lakehouse mode + strong model + no force",
in: DowngradeInput{Mode: ModeCodeReviewLakehouse, Model: "x-ai/grok-4.1-fast"},
want: DowngradeDecision{
Mode: ModeCodeReviewIsolation,
DowngradedFrom: ModeCodeReviewLakehouse,
},
},
{
name: "no downgrade: forced mode bypasses gate",
in: DowngradeInput{Mode: ModeCodeReviewLakehouse, Model: "x-ai/grok-4.1-fast", ForcedMode: true},
want: DowngradeDecision{Mode: ModeCodeReviewLakehouse},
},
{
name: "no downgrade: env override bypasses gate",
in: DowngradeInput{Mode: ModeCodeReviewLakehouse, Model: "x-ai/grok-4.1-fast", ForceFullOverride: true},
want: DowngradeDecision{Mode: ModeCodeReviewLakehouse},
},
{
name: "no downgrade: weak model keeps lakehouse",
in: DowngradeInput{Mode: ModeCodeReviewLakehouse, Model: "openai/gpt-4o:free"},
want: DowngradeDecision{Mode: ModeCodeReviewLakehouse},
},
{
name: "no downgrade: non-lakehouse mode (gate not applicable)",
in: DowngradeInput{Mode: "codereview_isolation", Model: "x-ai/grok-4.1-fast"},
want: DowngradeDecision{Mode: "codereview_isolation"},
},
}
for _, c := range cases {
got := MaybeDowngrade(c.in)
if got.Mode != c.want.Mode {
t.Errorf("%s: Mode want %q, got %q", c.name, c.want.Mode, got.Mode)
}
if got.DowngradedFrom != c.want.DowngradedFrom {
t.Errorf("%s: DowngradedFrom want %q, got %q", c.name, c.want.DowngradedFrom, got.DowngradedFrom)
}
if got.Reason == "" {
t.Errorf("%s: Reason should be non-empty", c.name)
}
}
}
// TestMaybeDowngrade_ForcedTrumpsOthers verifies precedence: when
// multiple bypass conditions hit, ForcedMode wins (explicit caller
// intent always overrides). Caught a subtle ordering bug in the
// original Rust code where this was tested only by happy path.
func TestMaybeDowngrade_ForcedTrumpsOthers(t *testing.T) {
in := DowngradeInput{
Mode: ModeCodeReviewLakehouse,
Model: "qwen3.5:latest", // weak — would otherwise hit weak-bypass
ForcedMode: true,
ForceFullOverride: true,
}
got := MaybeDowngrade(in)
if got.Mode != ModeCodeReviewLakehouse {
t.Errorf("forced mode should keep mode: got %q", got.Mode)
}
if got.DowngradedFrom != "" {
t.Errorf("no downgrade expected; got DowngradedFrom=%q", got.DowngradedFrom)
}
}