Phase 16: Hot-swap generations + autotune agent loop

Closes the self-iteration loop from the PRD reframe: an agent can
tune HNSW configs autonomously and the winner flows through to the
next profile activation without human intervention.

Three primitives:

1. PromotionRegistry (vectord::promotion)
   - Per-index current + history at _hnsw_promotions/{index}.json
   - promote(index, entry) atomically swaps current, pushes prior
     onto history (capped at 50)
   - rollback() pops history back onto current; clears current if
     history exhausted
   - config_or(index, default) — the read side used at build time,
     returns promoted config if set else caller's default
   - Full cache + persistence; writes are durable on return

2. Autotune (vectord::autotune)
   - run_autotune(request, ...) — synchronous agent loop
   - Default grid: 5 configs covering the practical range
     (ec=20/40/80/80/160, es=30/30/30/60/30) with seed=42 for
     reproducibility
   - Every trial goes through the existing trial-journal pipeline
     so autotune runs land alongside manual trials in the
     "trials are data" log
   - Winner: max recall first, then min p50 latency; must clear
     min_recall gate (default 0.9) or no promotion happens
   - Config bounds (ec ∈ [10,400], es ∈ [10,200]) reject absurd
     values from the request's optional custom grid
   - On winner: promote with note "autotune winner: recall=X p50=Y"

3. Wiring
   - VectorState gains promotion_registry
   - activate_profile now calls promotion_registry.config_or(...)
     so newly-promoted configs are picked up on next activation —
     the "hot-swap" is: autotune promotes -> profile activates ->
     HNSW rebuilt with new config
   - New endpoints:
       POST /vectors/hnsw/promote/{index}/{trial_id}
             ?promoted_by=...&note=...
       POST /vectors/hnsw/rollback/{index}
       GET  /vectors/hnsw/promoted/{index}
       POST /vectors/hnsw/autotune  { index_name, harness,
                                      min_recall?, grid? }

End-to-end verified on threat_intel_v1 (54 vectors):
- autogen harness 'threat_intel_smoke' (10 queries)
- POST /autotune -> 5 trials in 620ms, winner ec=20 es=30
  recall=1.00 p50=64us auto-promoted
- Manual promote of ec=80 es=30 -> history depth 1
- Rollback -> back to ec=20 es=30 autotune winner
- Second rollback -> current cleared
- Re-promote + restart -> persistence verified
- Profile activation after promotion logged:
  "building HNSW ef_construction=80 ef_search=30 seed=Some(42)"
  proving the hot-swap loop is closed.

Deferred:
- Bayesian optimization (random-grid is fine at this config-space size)
- Append-triggered autotune (Phase 17.5 — refresh OnAppend policy
  can schedule autotune after appending sufficient new rows)
- Concurrent autotune per index guard (JobTracker integration)

PRD invariants satisfied: invariant 8 (hot-swappable indexes) is now
real code — promote is atomic, rollback is always available, the
active generation is a persistent pointer not a runtime convention.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

crates/vectord/src/promotion.rs

@@ -0,0 +1,187 @@
+//! Phase 16: Promoted HNSW configs — the "active generation" pointer.
+//!
+//! An index's HNSW config used at build time normally defaults to the
+//! system-wide default (`HnswConfig::default()`). An operator or the
+//! autotune agent can *promote* a specific trial's config — subsequent
+//! HNSW builds against that index use the promoted config instead.
+//!
+//! Every promotion is history-tracked so `rollback` can revert. The
+//! history file lives at `primary://_hnsw_promotions/{index_name}.json`
+//! and is small (< few KB) so we rewrite it on every promotion rather
+//! than append-log.
+//!
+//! Not included here:
+//! - Atomic graph rebuild on promote — promotion only updates the sticky
+//!   default. Next activation (or search that triggers lazy build) picks
+//!   up the new config. That's "zero-downtime swap after build" which is
+//!   what ADR-019 actually claimed; an instant-swap requires a
+//!   pre-built graph pool which we don't have yet.
+//! - Agent loop — lives in `vectord::autotune`.
+
+use chrono::{DateTime, Utc};
+use object_store::ObjectStore;
+use serde::{Deserialize, Serialize};
+use std::collections::HashMap;
+use std::sync::Arc;
+use storaged::ops;
+use tokio::sync::RwLock;
+
+use crate::trial::HnswConfig;
+
+const PROMOTION_PREFIX: &str = "_hnsw_promotions";
+
+/// One promotion record. The `trial_id` is the origin of the config —
+/// lets operators trace back "why was this config picked?" to the exact
+/// trial in the trial journal.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct PromotionEntry {
+    pub config: HnswConfig,
+    pub trial_id: String,
+    pub promoted_at: DateTime<Utc>,
+    #[serde(default)]
+    pub promoted_by: String,
+    #[serde(default)]
+    pub note: Option<String>,
+}
+
+/// Serialized form of an index's promotion history.
+#[derive(Debug, Clone, Serialize, Deserialize, Default)]
+pub struct PromotionFile {
+    pub index_name: String,
+    pub current: Option<PromotionEntry>,
+    #[serde(default)]
+    pub history: Vec<PromotionEntry>,
+}
+
+#[derive(Clone)]
+pub struct PromotionRegistry {
+    store: Arc<dyn ObjectStore>,
+    cache: Arc<RwLock<HashMap<String, PromotionFile>>>,
+}
+
+impl PromotionRegistry {
+    pub fn new(store: Arc<dyn ObjectStore>) -> Self {
+        Self {
+            store,
+            cache: Arc::new(RwLock::new(HashMap::new())),
+        }
+    }
+
+    fn key(index_name: &str) -> String {
+        // Sanitize for object-store safety.
+        let safe: String = index_name
+            .chars()
+            .map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c } else { '_' })
+            .collect();
+        format!("{PROMOTION_PREFIX}/{safe}.json")
+    }
+
+    /// Load (and cache) the promotion file for an index.
+    pub async fn load(&self, index_name: &str) -> Result<PromotionFile, String> {
+        if let Some(cached) = self.cache.read().await.get(index_name) {
+            return Ok(cached.clone());
+        }
+        let key = Self::key(index_name);
+        let file = match ops::get(&self.store, &key).await {
+            Ok(bytes) => serde_json::from_slice::<PromotionFile>(&bytes)
+                .map_err(|e| format!("parse promotion file: {e}"))?,
+            Err(_) => PromotionFile {
+                index_name: index_name.to_string(),
+                current: None,
+                history: Vec::new(),
+            },
+        };
+        self.cache.write().await.insert(index_name.to_string(), file.clone());
+        Ok(file)
+    }
+
+    /// Promote a config to the active slot. Pushes the current promotion
+    /// (if any) onto the history stack. Persists before returning — the
+    /// config is durable by the time this call completes.
+    pub async fn promote(
+        &self,
+        index_name: &str,
+        entry: PromotionEntry,
+    ) -> Result<PromotionFile, String> {
+        let mut file = self.load(index_name).await?;
+        if let Some(prior) = file.current.take() {
+            file.history.push(prior);
+            // Cap history to something sensible so this file doesn't grow
+            // unbounded. 50 entries = 50 promotions — way more than any
+            // sane workflow needs.
+            const HISTORY_CAP: usize = 50;
+            if file.history.len() > HISTORY_CAP {
+                let drop = file.history.len() - HISTORY_CAP;
+                file.history.drain(0..drop);
+            }
+        }
+        file.current = Some(entry);
+        file.index_name = index_name.to_string();
+
+        let key = Self::key(index_name);
+        let json = serde_json::to_vec_pretty(&file).map_err(|e| e.to_string())?;
+        ops::put(&self.store, &key, json.into()).await?;
+
+        self.cache.write().await.insert(index_name.to_string(), file.clone());
+        tracing::info!(
+            "promoted '{}' to config {:?} (trial={})",
+            index_name, file.current.as_ref().unwrap().config, file.current.as_ref().unwrap().trial_id,
+        );
+        Ok(file)
+    }
+
+    /// Pop the latest promotion back onto the current slot (if any
+    /// history exists). If current is set but history is empty, the
+    /// current promotion is cleared — the index falls back to defaults.
+    pub async fn rollback(&self, index_name: &str) -> Result<PromotionFile, String> {
+        let mut file = self.load(index_name).await?;
+        match file.history.pop() {
+            Some(prev) => {
+                file.current = Some(prev);
+            }
+            None => {
+                if file.current.is_none() {
+                    return Err(format!("no promotion to rollback for '{index_name}'"));
+                }
+                file.current = None;
+            }
+        }
+        let key = Self::key(index_name);
+        let json = serde_json::to_vec_pretty(&file).map_err(|e| e.to_string())?;
+        ops::put(&self.store, &key, json.into()).await?;
+        self.cache.write().await.insert(index_name.to_string(), file.clone());
+        tracing::info!("rolled back promotion for '{}'", index_name);
+        Ok(file)
+    }
+
+    /// Get the currently-promoted config (if any). Callers use this to
+    /// pick the right HnswConfig at build time.
+    pub async fn get_current(&self, index_name: &str) -> Option<PromotionEntry> {
+        self.load(index_name).await.ok().and_then(|f| f.current)
+    }
+
+    /// Convenience: return the promoted config or the provided default.
+    pub async fn config_or(&self, index_name: &str, default: HnswConfig) -> HnswConfig {
+        match self.get_current(index_name).await {
+            Some(entry) => entry.config,
+            None => default,
+        }
+    }
+
+    /// List every index that has a promotion recorded (for operator UI).
+    pub async fn list_all(&self) -> Result<Vec<PromotionFile>, String> {
+        let keys = ops::list(&self.store, Some(&format!("{PROMOTION_PREFIX}/"))).await.unwrap_or_default();
+        let mut out = Vec::new();
+        for key in keys {
+            if !key.ends_with(".json") { continue; }
+            let bytes = match ops::get(&self.store, &key).await {
+                Ok(b) => b,
+                Err(_) => continue,
+            };
+            if let Ok(f) = serde_json::from_slice::<PromotionFile>(&bytes) {
+                out.push(f);
+            }
+        }
+        Ok(out)
+    }
+}