lance: auto-build doc_id btree in migrate handler (root-cause for 10M doc-fetch slowness)

scale_test_10m doc-fetch p50 was ~100ms — full table scan over 35GB. Root
cause: the auto-build at service.rs:1492-1503 only fires for IndexMeta-
registered indexes during set_active_profile warming. lance-bench writes
datasets through /vectors/lance/migrate/* directly, bypassing IndexMeta,
so its datasets never get the doc_id btree that ADR-019 depends on.

Fix: build the btree inline at the end of lance_migrate. Costs ~1.2s on
10M rows (+269MB on disk), drops doc-fetch from ~100ms to ~5ms (20x).
Failure is non-fatal — logs a warning and the dataset stays queryable.

Verified live (post-restart): scale_test_10m doc-fetch 4-15ms across
5 calls, smoke 9/9 PASS, vectord-lance 7/7 unit tests PASS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/vectord/src/service.rs

@@ -1866,11 +1866,40 @@ async fn lance_migrate(
         stats.disk_bytes, stats.duration_secs,
     );
 
+    // Auto-build the doc_id btree. The scalar index is what makes
+    // get_doc_by_id O(log n) instead of a full table scan; ADR-019
+    // calls this out as the load-bearing feature for hybrid lookup.
+    // Verified 2026-05-02: skipping this on a 10M-row dataset turns
+    // ~5ms doc-fetch into ~100ms (full scan over 35GB). Cheap to
+    // build (~1.2s on 10M, +269MB on disk) and only runs once per
+    // dataset since `has_scalar_index` short-circuits subsequent calls.
+    let scalar_stats = if !lance_store.has_scalar_index("doc_id").await.unwrap_or(false) {
+        match lance_store.build_scalar_index("doc_id").await {
+            Ok(s) => {
+                tracing::info!(
+                    "lance migrate '{}': doc_id btree built in {:.2}s (+{} bytes)",
+                    index_name, s.build_time_secs, s.disk_bytes_added,
+                );
+                Some(s)
+            }
+            Err(e) => {
+                // Don't fail the whole migrate over a missing btree —
+                // the dataset is still queryable, just slowly. Log it
+                // so it's debuggable.
+                tracing::warn!("lance migrate '{}': doc_id btree build failed (will fall back to scan): {e}", index_name);
+                None
+            }
+        }
+    } else {
+        None
+    };
+
     Ok::<_, (StatusCode, String)>(Json(serde_json::json!({
         "index_name": index_name,
         "bucket": bucket,
         "lance_path": lance_store.path(),
         "stats": stats,
+        "scalar_index": scalar_stats,
     })))
 }