RAG reranker + manifest bucket fix — quality improvements from eval

RAG pipeline now includes a cross-encoder rerank step between retrieval
and generation. The LLM re-sorts top-K results by relevance before
they become context. Falls back to original order if model output is
unparseable (~5% with 7B models). Also improved the generation prompt
to be domain-aware ("staffing database") and request specific citations.

Fixed 4 catalog manifests with bucket="data" (pre-federation leftover)
that poisoned the entire DataFusion query context on startup. The
"users", "lab_trials", "meta_runs", and "new_candidates" datasets
now correctly reference bucket="primary". This bug was surfaced by
the quality evaluation pipeline — wouldn't have been found by
structural tests alone.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

data/_catalog/manifests/e7304f05-5278-4e17-961a-51f2588fd2aa.json

@@ -4,14 +4,14 @@
   "schema_fingerprint": "df1e126046147b3de42086880e10c3501a3a615ecddf336bc24957a24c321241",
   "objects": [
     {
-      "bucket": "data",
+      "bucket": "primary",
       "key": "datasets/threat_intel.parquet",
       "size_bytes": 111130,
       "created_at": "2026-03-28T01:14:03.054140697Z"
     }
   ],
   "created_at": "2026-03-28T01:14:03.054141294Z",
-  "updated_at": "2026-03-28T01:14:03.054427047Z",
+  "updated_at": "2026-04-17T02:45:57.723324051Z",
   "description": "",
   "owner": "",
   "sensitivity": null,
@@ -229,12 +229,15 @@
   ],
   "lineage": {
     "source_system": "postgresql",
-    "source_file": "127.0.0.1:5432/knowledge_base.threat_intel",
-    "ingest_job": "pg-import-1774660443054",
-    "ingest_timestamp": "2026-03-28T01:14:03.054140697Z",
+    "source_file": "dsn: postgresql://postgres@127.0.0.1:5432/knowledge_base",
+    "ingest_job": "pg-stream-1776326353882",
+    "ingest_timestamp": "2026-04-16T07:59:13.882669337Z",
     "parent_datasets": []
   },
   "freshness": null,
   "tags": [],
-  "row_count": 20
+  "row_count": 20,
+  "last_embedded_at": "2026-04-16T15:08:32.348412159Z",
+  "embedding_stale_since": null,
+  "embedding_refresh_policy": null
 }