G1P: vectord persistence to storaged + scrum (3 fixes incl. 3-way convergent)

Adds optional persistence to vectord (G1's HNSW vector search). Single-
file framed format per index — eliminates the torn-write class that
the 3-way convergent scrum finding identified:

  _vectors/<name>.lhv1  — single binary blob:
      [4 bytes magic "LHV1"]
      [4 bytes envelope_len uint32 BE]
      [envelope bytes — JSON params + metadata + version]
      [graph bytes — raw hnsw.Graph.Export]

Pre-extraction: internal/catalogd/store_client.go → internal/storeclient/
shared package, since both catalogd and vectord need it. Same pattern as
the pre-D5 catalogclient extraction.

Optional via [vectord].storaged_url config (empty = ephemeral mode).
On startup: List + Load each persisted index. After Create / batch Add /
DELETE: Save (or Delete from storaged). Save failures are logged-not-
fatal — in-memory state is the source of truth in flight.

Acceptance smoke G1P 8/8 PASS — kill+restart preserves state, post-
restart search returns dist=0 (graph round-trips exactly), DELETE
removes the file, post-delete restart shows count=0.

All 8 smokes (D1-D6 + G1 + G1P) PASS deterministically. The g1_smoke
gained scripts/g1_smoke.toml that disables persistence so the
in-memory API test stays decoupled from any rehydrate-from-storaged
state contamination.

Cross-lineage scrum on shipped code:
  - Opus 4.7 (opencode):                     1 BLOCK + 5 WARN + 3 INFO
  - Kimi K2-0905 (openrouter):               1 BLOCK + 2 WARN
  - Qwen3-coder (openrouter):                2 BLOCK + 2 WARN + 1 INFO

Fixed (3 — 1 convergent + 2 single-reviewer):
  C1 (Opus + Kimi + Qwen 3-WAY CONVERGENT WARN): Save was non-atomic
    across two PUTs — envelope-succeeds + graph-fails left a half-
    saved index that passed the "both present" List filter and
    silently mismatched metadata against vectors on Load.
    Fix: collapse to single framed file (no torn-write window
    possible).
  O-B1 (Opus BLOCK): isNotFound substring-matched "key not found"
    against the wrapped error message — brittle, any 5xx body
    containing that text would silently misclassify as missing.
    Fix: errors.Is(err, storeclient.ErrKeyNotFound).
  O-I3 (Opus INFO): handleAdd pre-validation only covered id+dim;
    NaN/Inf/zero-norm could still fail mid-batch leaving partial
    commits. Fix: extend pre-validation to call ValidateVector
    (newly exported) per item before any commit.

Dismissed (3 false positives):
  K-B1 + Q-B1 ("safeKey double-escapes %2F segments") — false
    convergent. Wire-protocol escape is decoded by storaged's chi
    router on the way in; on-disk key is the original literal.
    %2F round-trips correctly through PathEscape → URL → chi decode
    → S3 key.
  Q-B2 ("List vulnerable to race conditions") — vectord is single-
    process; no concurrent Save against List in the same vectord.

Deferred (3): rehydrate per-index timeout (G2+ multi-index scale),
saveAfter request ctx (matches G0 timeout deferral), Encode RLock
during slow writer (documented as buffer-only API).

The C1 finding is the strongest signal of the cross-lineage filter:
three independent reviewers all flagged the same torn-write hazard.
Single-file framing eliminates the class — there's now no Persistor
state where envelope and graph can disagree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

cmd/catalogd/main.go

@@ -18,6 +18,7 @@ import (
 
 	"git.agentview.dev/profit/golangLAKEHOUSE/internal/catalogd"
 	"git.agentview.dev/profit/golangLAKEHOUSE/internal/shared"
+	"git.agentview.dev/profit/golangLAKEHOUSE/internal/storeclient"
 )
 
 func main() {
@@ -34,7 +35,7 @@ func main() {
 		os.Exit(1)
 	}
 
-	store := catalogd.NewStoreClient(cfg.Catalogd.StoragedURL)
+	store := storeclient.New(cfg.Catalogd.StoragedURL)
 	registry := catalogd.NewRegistry(store)
 
 	rehydrateCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)

cmd/vectord/main.go

@@ -1,15 +1,14 @@
-// vectord is the vector-search service. In-memory HNSW indexes
-// keyed by string IDs with optional opaque JSON metadata. Wraps
+// vectord is the vector-search service. HNSW indexes keyed by
+// string IDs with optional opaque JSON metadata. Wraps
 // github.com/coder/hnsw (pure Go, no cgo).
 //
-// G1 scope: in-memory only, single-process. Persistence to storaged
-// + rehydrate-on-restart is the next piece. The HTTP surface is
-// stable enough to build the staffing co-pilot's "find workers
-// like X" path on top right now — the indexes just have to be
-// rebuilt after a restart.
+// G1 + persistence: indexes are persisted to storaged at
+// _vectors/<name>.{json,hnsw} and rehydrated on startup. Setting
+// [vectord].storaged_url empty disables persistence (dev mode).
 package main
 
 import (
+	"context"
 	"encoding/json"
 	"errors"
 	"flag"
@@ -18,10 +17,12 @@ import (
 	"os"
 	"strconv"
 	"strings"
+	"time"
 
 	"github.com/go-chi/chi/v5"
 
 	"git.agentview.dev/profit/golangLAKEHOUSE/internal/shared"
+	"git.agentview.dev/profit/golangLAKEHOUSE/internal/storeclient"
 	"git.agentview.dev/profit/golangLAKEHOUSE/internal/vectord"
 )
 
@@ -43,6 +44,24 @@ func main() {
 
 	h := &handlers{reg: vectord.NewRegistry()}
 
+	// Persistence is optional — empty StoragedURL = dev/ephemeral mode.
+	if cfg.Vectord.StoragedURL != "" {
+		h.persist = vectord.NewPersistor(storeclient.New(cfg.Vectord.StoragedURL))
+
+		// Rehydrate any persisted indexes at startup. Failures are
+		// logged-not-fatal: storaged might be coming up after vectord,
+		// and an index that failed to load is still recoverable by
+		// re-ingesting on top of an empty registry.
+		ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+		n, err := h.rehydrate(ctx)
+		cancel()
+		if err != nil {
+			slog.Warn("rehydrate", "err", err, "loaded", n)
+		} else {
+			slog.Info("rehydrated", "indexes", n)
+		}
+	}
+
 	if err := shared.Run("vectord", cfg.Vectord.Bind, h.register); err != nil {
 		slog.Error("server", "err", err)
 		os.Exit(1)
@@ -50,7 +69,65 @@ func main() {
 }
 
 type handlers struct {
-	reg *vectord.Registry
+	reg     *vectord.Registry
+	persist *vectord.Persistor // nil when persistence is disabled
+}
+
+// rehydrate enumerates persisted indexes and loads each into the
+// registry. Returns the count of successfully loaded indexes plus
+// the first error (if any) — caller decides fatality.
+func (h *handlers) rehydrate(ctx context.Context) (int, error) {
+	if h.persist == nil {
+		return 0, nil
+	}
+	names, err := h.persist.List(ctx)
+	if err != nil {
+		return 0, err
+	}
+	loaded := 0
+	for _, name := range names {
+		idx, err := h.persist.Load(ctx, name)
+		if err != nil {
+			slog.Warn("rehydrate skip", "name", name, "err", err)
+			continue
+		}
+		// The registry's Create rebuilds an empty Index from params;
+		// we want the LOADED one (with vectors). Bypass via a
+		// helper that registers a pre-built Index.
+		if err := h.reg.RegisterPrebuilt(idx); err != nil {
+			slog.Warn("rehydrate register", "name", name, "err", err)
+			continue
+		}
+		loaded++
+	}
+	return loaded, nil
+}
+
+// saveAfter is the post-write persistence hook. Logs-not-fatal:
+// in-memory state is the source of truth in flight; a failed save
+// gets re-attempted on the next mutation, and the operator log
+// shows the storaged outage.
+func (h *handlers) saveAfter(idx *vectord.Index) {
+	if h.persist == nil {
+		return
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	if err := h.persist.Save(ctx, idx); err != nil {
+		slog.Warn("persist save", "name", idx.Params().Name, "err", err)
+	}
+}
+
+// deleteAfter mirrors saveAfter for the Delete path.
+func (h *handlers) deleteAfter(name string) {
+	if h.persist == nil {
+		return
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	if err := h.persist.Delete(ctx, name); err != nil {
+		slog.Warn("persist delete", "name", name, "err", err)
+	}
 }
 
 func (h *handlers) register(r chi.Router) {
@@ -125,6 +202,7 @@ func (h *handlers) handleCreate(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, "internal", http.StatusInternalServerError)
 		return
 	}
+	h.saveAfter(idx)
 	writeJSON(w, http.StatusCreated, indexInfo{Params: idx.Params(), Length: idx.Len()})
 }
 
@@ -156,6 +234,7 @@ func (h *handlers) handleDelete(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, "internal", http.StatusInternalServerError)
 		return
 	}
+	h.deleteAfter(name)
 	w.WriteHeader(http.StatusNoContent)
 }
 
@@ -174,12 +253,13 @@ func (h *handlers) handleAdd(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, "items must be non-empty", http.StatusBadRequest)
 		return
 	}
-	// Per scrum O-W4 (Opus): pre-validate all items before any Add,
-	// so a bad item at position N doesn't leave items 0..N-1 already
-	// committed and item N rejected. Both checks (empty id, dim
-	// mismatch) are local; running them up-front is O(N) extra work
-	// that the success path already paid in idx.Add.
-	dim := idx.Params().Dimension
+	// Per scrum O-W4 (Opus, D5): pre-validate all items before any
+	// Add, so a bad item at position N doesn't leave items 0..N-1
+	// committed and item N rejected. Per scrum O-I3 (Opus, G1P):
+	// extend pre-validation to cover NaN/Inf and zero-norm — these
+	// were caught inside idx.Add but only after partial commits.
+	params := idx.Params()
+	dim := params.Dimension
 	for j, it := range req.Items {
 		if it.ID == "" {
 			http.Error(w, "items["+strconv.Itoa(j)+"]: empty id", http.StatusBadRequest)
@@ -189,6 +269,10 @@ func (h *handlers) handleAdd(w http.ResponseWriter, r *http.Request) {
 			http.Error(w, "items["+strconv.Itoa(j)+"]: dim mismatch (index="+strconv.Itoa(dim)+", got="+strconv.Itoa(len(it.Vector))+")", http.StatusBadRequest)
 			return
 		}
+		if err := vectord.ValidateVector(it.Vector, params.Distance); err != nil {
+			http.Error(w, "items["+strconv.Itoa(j)+"]: "+err.Error(), http.StatusBadRequest)
+			return
+		}
 	}
 	for j, it := range req.Items {
 		if err := idx.Add(it.ID, it.Vector, it.Metadata); err != nil {
@@ -206,6 +290,9 @@ func (h *handlers) handleAdd(w http.ResponseWriter, r *http.Request) {
 			return
 		}
 	}
+	// One save per batch (post-loop), not per item. Per scrum
+	// O-W4-style discipline: HTTP-batch boundary is the natural unit.
+	h.saveAfter(idx)
 	writeJSON(w, http.StatusOK, map[string]any{"added": len(req.Items), "length": idx.Len()})
 }