Dual-pipeline supervisor for embedding ingestion

- 4 parallel pipelines (tuned for i9 + A4000)
- Range-based work splitting (2500 chunks per range)
- Round-robin retry on failure (3 attempts before dead-letter)
- Checkpointing to disk every 1000 chunks (crash recovery)
- On restart, loads checkpoint and skips completed ranges
- Dead-letter queue for permanently failed ranges
- Vectors assembled in order after all pipelines finish
- Batch size 64 for GPU throughput

Architecture:
  Supervisor → splits 100K chunks into 40 ranges
    ├── Pipeline 0: grabs range, embeds, reports progress
    ├── Pipeline 1: grabs range, embeds, reports progress
    ├── Pipeline 2: grabs range, embeds, reports progress
    └── Pipeline 3: grabs range, embeds, reports progress
    Failed range → back to queue → next available pipeline retries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

crates/vectord/src/service.rs

@@ -10,7 +10,7 @@ use serde::{Deserialize, Serialize};
 use std::sync::Arc;
 
 use aibridge::client::{AiClient, EmbedRequest};
-use crate::{chunker, jobs, rag, search, store};
+use crate::{chunker, jobs, rag, search, store, supervisor};
 
 #[derive(Clone)]
 pub struct VectorState {
@@ -84,14 +84,17 @@ async fn create_index(
     let job_id = state.job_tracker.create(&index_name, n_chunks).await;
     tracing::info!("job {job_id}: indexing '{}' — {} docs → {} chunks (background)", index_name, n_docs, n_chunks);
 
-    // Spawn background embedding task
+    // Spawn supervised dual-pipeline embedding
     let tracker = state.job_tracker.clone();
     let ai_client = state.ai_client.clone();
     let obj_store = state.store.clone();
     let jid = job_id.clone();
 
     tokio::spawn(async move {
-        let result = run_embedding_job(&jid, &index_name, &chunks, &ai_client, &obj_store, &tracker).await;
+        let config = supervisor::SupervisorConfig::default();
+        let result = supervisor::run_supervised(
+            &jid, &index_name, chunks, &ai_client, &obj_store, &tracker, config,
+        ).await;
         match result {
             Ok(key) => {
                 tracker.complete(&jid, key).await;