vectord: BatchAdd — single-lock variadic batch (Option A)
Replaces the per-item Add loop in the HTTP handler with one call to Index.BatchAdd, which acquires the write-lock once and pushes the whole batch through coder/hnsw's variadic Graph.Add. Pre-validation stays in the handler so per-item error messages keep their item-index precision. Microbench (internal/vectord/batch_bench_test.go) at d=768 cosine: N=16 SingleAdd 283µs/op → BatchAdd 170µs/op 1.66× N=128 SingleAdd 7.9ms/op → BatchAdd 7.5ms/op 1.05× N=1024 SingleAdd 87.5ms/op → BatchAdd 83.4ms/op 1.05× Win is biggest at staffing-driver batch sizes (N=16) where per-call lock + validation overhead is a meaningful fraction. At larger N the inner HNSW neighborhood search per insert dominates, which is the load-bearing finding for Option B (sharded indexes): the throughput ceiling lives inside the library, not at the lock, so sharding to N parallel Graphs is the only path to true concurrent-Add throughput. g1, g1p, g2 smokes all PASS post-change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -225,6 +225,66 @@ func validateVector(vec []float32, distance string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// BatchItem is one entry in a BatchAdd call. Same per-field
|
||||
// contract as Add: ID + Vector required, Metadata follows
|
||||
// upsert-style semantics (nil = leave existing alone).
|
||||
type BatchItem struct {
|
||||
ID string
|
||||
Vector []float32
|
||||
Metadata json.RawMessage
|
||||
}
|
||||
|
||||
// BatchAdd inserts a slice of items under a single write-lock, with
|
||||
// one variadic call into coder/hnsw's Graph.Add. Net win vs. a loop
|
||||
// of single Add calls: N→1 lock acquisitions per HTTP batch and one
|
||||
// variadic library call instead of N.
|
||||
//
|
||||
// Contract: items MUST be pre-validated by the caller (id non-empty,
|
||||
// vector dimension matches, vector finite + non-zero-norm under
|
||||
// cosine). Pre-validation lives in the HTTP handler so per-item
|
||||
// error messages stay precise; reproducing it here would force
|
||||
// position-encoded errors on every consumer.
|
||||
//
|
||||
// Intra-batch duplicate IDs are undefined behavior — coder/hnsw's
|
||||
// internal "node not added" length-invariant fires on the second
|
||||
// occurrence. Callers must de-dup before calling. The HTTP smoke
|
||||
// uses unique IDs so this isn't an exercised path; documented for
|
||||
// future callers.
|
||||
func (i *Index) BatchAdd(items []BatchItem) error {
|
||||
if len(items) == 0 {
|
||||
return nil
|
||||
}
|
||||
i.mu.Lock()
|
||||
defer i.mu.Unlock()
|
||||
|
||||
// Pre-pass: drop any existing IDs so coder/hnsw's variadic Add
|
||||
// never sees a re-add. Same library-quirk handling as single
|
||||
// Add — Len()==1 needs a full graph reset because Delete of the
|
||||
// last node leaves layers[0] entryless.
|
||||
for _, it := range items {
|
||||
if _, exists := i.g.Lookup(it.ID); exists {
|
||||
if i.g.Len() == 1 {
|
||||
i.resetGraphLocked()
|
||||
} else {
|
||||
i.g.Delete(it.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nodes := make([]hnsw.Node[string], len(items))
|
||||
for j, it := range items {
|
||||
nodes[j] = hnsw.MakeNode(it.ID, it.Vector)
|
||||
}
|
||||
i.g.Add(nodes...)
|
||||
|
||||
for _, it := range items {
|
||||
if it.Metadata != nil {
|
||||
i.meta[it.ID] = it.Metadata
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Delete removes id from the index. Returns true if present.
|
||||
func (i *Index) Delete(id string) bool {
|
||||
i.mu.Lock()
|
||||
|
||||
Reference in New Issue
Block a user