vectord: v2 envelope + handleMerge robustness — actions post_role_gate_v1 scrum

3-lineage scrum on 434f466..0d4f033 surfaced one convergent finding
(Opus + Kimi) and 3 Opus-only real bugs. All actioned in this
commit. Two false positives (Kimi rollback misreading, Opus stale-
comment claim) verified + rejected — both required manual control-
flow inspection to refute, matching the documented Kimi-truncation
behavior in feedback_cross_lineage_review.md.

Convergent fix — DecodeIndex lost nil-meta items:
- Envelope version bumped 1 → 2.
- New v2 field: IDs []string carries the canonical ID set
  explicitly, independent of meta map's nil-vs-{} sparseness.
- DecodeIndex accepts both versions: v2 reads from env.IDs; v1
  falls back to meta-key inference (with the documented
  limitation that nil-meta items are invisible — preserved for
  backward-compat with already-persisted indexes).
- Encode emits v2 going forward.
- 2 new regression tests:
  - TestEncodeDecode_NilMetaItemsSurviveRoundTrip: items added
    with nil metadata MUST survive Encode → Decode and remain
    visible to IDs(). Pre-fix would have yielded IDs() == [].
  - TestDecodeIndex_V1BackwardCompat: hand-crafted v1 envelope
    still decodes (proves the fallback path).

Opus-only fixes:
- handleMerge: non-ErrIndexNotFound errors at h.reg.Get(name) /
  h.reg.Get(req.Dest) now return 500 + log instead of falling
  through with nil src/dest pointers (which would panic on the
  next deref). Real bug — only the sentinel error was handled.
- internal/drift/drift.go: mathLog wrapper removed; math.Log
  inlined. Wrapper added no value (math was already imported).
- internal/distillation/audit_baseline.go: BuildAuditDriftTable's
  bubble sort replaced with sort.Slice. Idiomatic + shorter.

Rejected after verification:
- Kimi WARN "missing rollback on partial merge": misread the
  control flow. Code at cmd/vectord/main.go:404-414 does NOT
  delete from src when dest.Add fails (continue before reaching
  src.Delete). Only successful Adds trigger Deletes.
- Opus INFO "TimestampUnixNano comment references missing field":
  field exists at scripts/multi_coord_stress/main.go:128. Opus
  saw only the diff context, not the full file.

Deferred (no fired trigger):
- Opus WARN "no per-index lock during merge": no concurrent merge
  callers today (operators run merge as deliberate one-shot job).
  Worth a lock if/when matrixd or chatd start auto-triggering.

Disposition: reports/scrum/_evidence/2026-05-01/verdicts/post_role_gate_v1_disposition.md.

Build + vet + tests green; 2 new regression tests + all prior tests
unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-05-01 01:20:37 -05:00
co-authored by Claude Opus 4.7
parent 0d4f033b34
commit eb0dfdff04
6 changed files with 124 additions and 31 deletions
+66
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"math"
"strings"
"sync"
"testing"
)
@@ -189,6 +190,71 @@ func TestDecodeIndex_VersionMismatch(t *testing.T) {
}
}
// TestEncodeDecode_NilMetaItemsSurviveRoundTrip locks the
// post_role_gate_v1 scrum convergent finding (Opus + Kimi):
// items added with nil metadata MUST survive Encode→Decode and
// remain visible to IDs(). Pre-fix v1 envelope inferred ids from
// meta keys, silently dropping nil-meta items. v2 envelope carries
// the IDs slice explicitly. Test creates a worst-case where every
// item has nil metadata — pre-fix would yield IDs() == [].
func TestEncodeDecode_NilMetaItemsSurviveRoundTrip(t *testing.T) {
src, _ := NewIndex(IndexParams{Name: "nil_meta_test", Dimension: 4, Distance: DistanceCosine})
for _, id := range []string{"a", "b", "c"} {
// nil meta — the case Opus + Kimi flagged.
if err := src.Add(id, []float32{1, 0, 0, 0}, nil); err != nil {
t.Fatalf("Add %s: %v", id, err)
}
}
if got := src.IDs(); len(got) != 3 {
t.Fatalf("pre-encode: expected 3 IDs, got %d", len(got))
}
var envBuf, graphBuf bytes.Buffer
if err := src.Encode(&envBuf, &graphBuf); err != nil {
t.Fatalf("Encode: %v", err)
}
dst, err := DecodeIndex(&envBuf, &graphBuf)
if err != nil {
t.Fatalf("DecodeIndex: %v", err)
}
if got := dst.IDs(); len(got) != 3 {
t.Errorf("post-decode: expected 3 IDs (nil-meta items must survive v2 round-trip), got %d %v", len(got), got)
}
}
// TestDecodeIndex_V1BackwardCompat locks the legacy-shape fallback:
// envelope without an explicit "ids" field is still loadable. The
// v2 → v1 fallback path infers ids from meta keys (with the
// documented limitation for nil-meta items, which this test does
// NOT exercise — it only proves v1 envelopes still load).
func TestDecodeIndex_V1BackwardCompat(t *testing.T) {
// Hand-craft a v1 envelope (no IDs field).
envJSON := `{"version":1,"params":{"name":"v1_test","dimension":4,"distance":"cosine","m":16,"ef_search":20},"metadata":{"id1":{"foo":"bar"}}}`
// Empty graph stream — DecodeIndex should still succeed and
// emit an Index with id1 in i.ids inferred from meta.
src, _ := NewIndex(IndexParams{Name: "tmp", Dimension: 4})
_ = src.Add("dummy", []float32{1, 0, 0, 0}, json.RawMessage(`{"x":1}`))
var graphBuf bytes.Buffer
if err := src.g.Export(&graphBuf); err != nil {
t.Fatalf("export tmp graph for v1 fixture: %v", err)
}
dst, err := DecodeIndex(strings.NewReader(envJSON), &graphBuf)
if err != nil {
t.Fatalf("v1 envelope must still load, got %v", err)
}
// ids should contain "id1" (from the v1 metadata-key fallback).
hasID1 := false
for _, id := range dst.IDs() {
if id == "id1" {
hasID1 = true
break
}
}
if !hasID1 {
t.Errorf("v1 fallback didn't restore id from meta keys, got IDs=%v", dst.IDs())
}
}
func TestRegistry_CreateGetDelete(t *testing.T) {
r := NewRegistry()
idx, err := r.Create(IndexParams{Name: "workers", Dimension: 4})