Phase 19 wiring + Path 1/2 work + chain integrity fixes
Backend:
- crates/vectord/src/playbook_memory.rs (new): Phase 19 in-memory boost
store with seed/rebuild/snapshot, plus temporal decay (e^-age/30 per
playbook), persist_to_sql endpoint backing successful_playbooks_live,
and discover_patterns endpoint for meta-index pattern aggregation
(recurring certs/skills/archetype/reliability across similar past fills).
- DEFAULT_TOP_K_PLAYBOOKS bumped 5 → 25; old default silently missed
most boosts when memory had > 25 entries.
- service.rs: new routes /vectors/playbook_memory/{seed,rebuild,stats,
persist_sql,patterns}.
Bun staffing co-pilot (mcp-server/):
- /search, /match, /verify, /proof, /simulation/run, MCP tools all
forward use_playbook_memory:true and playbook_memory_k:25 to the
hybrid endpoint. Boost was previously dark across the entire app.
- /log no longer POSTs to /ingest/file — that endpoint REPLACES the
dataset's object list, so single-row CSV writes were wiping all prior
rows in successful_playbooks (sp_rows went 33→1 in one /log call).
/log now seeds playbook_memory with canonical short text and calls
/persist_sql to keep successful_playbooks_live in sync.
- /simulation/run cumulative end-of-week CSV write removed for the same
reason. Per-day per-contract /seed (added in this session) is the
accumulating feedback path now.
- search.html addWorkerInsight renders a green "Endorsed · N playbooks"
chip with playbook citations when boost > 0.
Internal Dioxus UI (crates/ui/):
- Dashboard phase list rewritten through Phase 19 (was stuck at "Phase
16: File Watcher" / "Phase 17: DB Connector" — both wrong).
- Removed fabricated "27ms" stat label.
- Ask tab examples + SQL default replaced with real staffing prompts
against candidates/clients/job_orders (was referencing nonexistent
employees/products/events).
- New Playbook tab exposes /vectors/playbook_memory/{stats,rebuild} and
side-by-side hybrid search (boost OFF vs ON) with citations.
Tests (tests/multi-agent/):
- run_e2e_rated.ts: parallel two-agent (mistral + qwen2.5) build phase
+ verifier rating (geo, auth, persist, boost, speed → /10).
- network_proving.ts: continuous build → verify → repeat with
staffing-recruiter profile hot-swap; geo-discrimination check.
- chain_of_custody.ts: single recruiter operation traced through every
layer (Bun /search, direct /vectors/hybrid parity, /log, SQL,
playbook_memory growth, profile activation, post-op boost lift).
This commit is contained in:
root
@@ -0,0 +1,825 @@
|
||||
//! Phase 19: Playbook memory — the feedback loop that makes the index
|
||||
//! learn from real outcomes instead of just logging them.
|
||||
//!
|
||||
//! When an agent (multi-agent orchestrator or human operator) seals a
|
||||
//! successful playbook, it lands in the `successful_playbooks` dataset.
|
||||
//! Historically that was a write-only log. This module turns it into a
|
||||
//! re-ranking signal:
|
||||
//!
|
||||
//! 1. `rebuild` reads every row of `successful_playbooks`, embeds the
|
||||
//! operation+approach+context as one vector per playbook, parses
|
||||
//! out the worker names from the `result` column, and stores both
|
||||
//! the vectors and the (playbook → names) endorsement map in memory.
|
||||
//!
|
||||
//! 2. At query time, `compute_boost_for` takes a new operation text
|
||||
//! (e.g. "fill: Welder x2 in Toledo, OH"), embeds it, brute-force
|
||||
//! ranks past playbooks by cosine similarity, and returns a boost
|
||||
//! map keyed by (city, state, worker_name) → `BoostEntry`. Each
|
||||
//! entry carries its similarity score and the citing playbook_ids,
|
||||
//! so explanations ("ranked higher because of 3 similar past fills
|
||||
//! in Toledo") are free.
|
||||
//!
|
||||
//! 3. The `use_playbook_memory` flag on `/vectors/hybrid` adds those
|
||||
//! boosts to matching search hits and re-sorts.
|
||||
//!
|
||||
//! Why brute force instead of another HNSW: `successful_playbooks` grows
|
||||
//! by operators, not automation. A few thousand rows is the realistic
|
||||
//! ceiling for years. Brute force at 10K × 768d is <10ms on this hardware
|
||||
//! — not worth the operational cost of another indexed surface.
|
||||
//!
|
||||
//! Persistence: the endorsements map round-trips through
|
||||
//! `_playbook_memory/state.json` in primary storage so the cache
|
||||
//! survives restarts without a full rebuild.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
use aibridge::client::{AiClient, EmbedRequest};
|
||||
use object_store::ObjectStore;
|
||||
use storaged::ops;
|
||||
|
||||
const STATE_KEY: &str = "_playbook_memory/state.json";
|
||||
|
||||
/// Maximum boost a single worker can accumulate across all similar past
|
||||
/// playbooks. Prevents one very popular worker from always winning.
|
||||
pub const MAX_BOOST_PER_WORKER: f32 = 0.25;
|
||||
|
||||
/// Default number of past playbooks to consider when ranking the current
|
||||
/// operation. Bumped 5 → 25 on 2026-04-20 because at >100 entries in
|
||||
/// memory the old default missed too many relevant playbooks — boost
|
||||
/// silently failed even when the seeded workers were ideal matches.
|
||||
/// 25 is brute-force-cheap (sub-ms) and covers most live operator memory.
|
||||
pub const DEFAULT_TOP_K_PLAYBOOKS: usize = 25;
|
||||
|
||||
/// Half-life of a playbook's contribution to boost, in days. A playbook
|
||||
/// 30 days old contributes half what a fresh one would; 60 days old, a
|
||||
/// quarter; etc. Per Path 1 (deepen statistical) — stale endorsements
|
||||
/// shouldn't dominate fresh signal. Recruiter trust depends on this.
|
||||
pub const BOOST_HALF_LIFE_DAYS: f32 = 30.0;
|
||||
|
||||
/// Shape of one playbook in memory. The embedding is optional so we can
|
||||
/// round-trip a cached state without re-embedding; the rebuild path
|
||||
/// populates it.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct PlaybookEntry {
|
||||
pub playbook_id: String,
|
||||
pub operation: String,
|
||||
pub approach: String,
|
||||
pub context: String,
|
||||
pub timestamp: String,
|
||||
/// Parsed out of `result` (e.g. "2/2 filled → Matthew Roberts, Amy Davis").
|
||||
/// Stored as raw names; matching against search results happens on
|
||||
/// (city, state, name) tuples at boost time.
|
||||
pub endorsed_names: Vec<String>,
|
||||
/// City + state parsed out of the operation string. Kept separately
|
||||
/// so boost matching doesn't re-parse on every query.
|
||||
pub city: Option<String>,
|
||||
pub state: Option<String>,
|
||||
/// Embedding of `operation + approach + context`. Option so persisted
|
||||
/// state can omit it on first load and have a later embed() fill in.
|
||||
#[serde(default)]
|
||||
pub embedding: Option<Vec<f32>>,
|
||||
}
|
||||
|
||||
/// Persisted / in-memory state.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
|
||||
struct PlaybookMemoryState {
|
||||
entries: Vec<PlaybookEntry>,
|
||||
/// Unix epoch millis when the last rebuild completed. Caller can
|
||||
/// use this to gate "stale > N hours → trigger rebuild" behavior.
|
||||
last_rebuilt_at: i64,
|
||||
}
|
||||
|
||||
/// Per-worker boost payload. `citations` lets the response layer show
|
||||
/// "boosted because of these past fills" without a second lookup.
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct BoostEntry {
|
||||
pub boost: f32,
|
||||
pub citations: Vec<String>, // playbook_ids that endorsed this worker
|
||||
}
|
||||
|
||||
/// Live handle passed around the service. Clone-cheap (all state is
|
||||
/// inside one Arc<RwLock>).
|
||||
#[derive(Clone)]
|
||||
pub struct PlaybookMemory {
|
||||
state: Arc<RwLock<PlaybookMemoryState>>,
|
||||
store: Arc<dyn ObjectStore>,
|
||||
}
|
||||
|
||||
impl PlaybookMemory {
|
||||
pub fn new(store: Arc<dyn ObjectStore>) -> Self {
|
||||
Self {
|
||||
state: Arc::new(RwLock::new(PlaybookMemoryState::default())),
|
||||
store,
|
||||
}
|
||||
}
|
||||
|
||||
/// Best-effort load from primary storage. Missing = empty memory; the
|
||||
/// first `/rebuild` call will hydrate it.
|
||||
pub async fn load_from_storage(&self) -> Result<usize, String> {
|
||||
let data = match ops::get(&self.store, STATE_KEY).await {
|
||||
Ok(d) => d,
|
||||
Err(_) => return Ok(0),
|
||||
};
|
||||
let persisted: PlaybookMemoryState = serde_json::from_slice(&data)
|
||||
.map_err(|e| format!("parse playbook_memory state: {e}"))?;
|
||||
let n = persisted.entries.len();
|
||||
*self.state.write().await = persisted;
|
||||
tracing::info!("playbook_memory: loaded {n} entries from {STATE_KEY}");
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
async fn persist(&self) -> Result<(), String> {
|
||||
let snapshot = self.state.read().await.clone();
|
||||
let bytes = serde_json::to_vec_pretty(&snapshot).map_err(|e| e.to_string())?;
|
||||
ops::put(&self.store, STATE_KEY, bytes.into()).await
|
||||
}
|
||||
|
||||
/// Replace the full in-memory state atomically and persist.
|
||||
pub async fn set_entries(&self, entries: Vec<PlaybookEntry>) -> Result<(), String> {
|
||||
let mut s = self.state.write().await;
|
||||
s.entries = entries;
|
||||
s.last_rebuilt_at = chrono::Utc::now().timestamp_millis();
|
||||
drop(s);
|
||||
self.persist().await
|
||||
}
|
||||
|
||||
pub async fn entry_count(&self) -> usize {
|
||||
self.state.read().await.entries.len()
|
||||
}
|
||||
|
||||
pub async fn snapshot(&self) -> Vec<PlaybookEntry> {
|
||||
self.state.read().await.entries.clone()
|
||||
}
|
||||
|
||||
/// Given an operation's embedding, find the top-K most similar past
|
||||
/// playbooks (by cosine similarity) and return a per-worker boost map
|
||||
/// keyed by (city, state, name). Worker is matched by the tuple so a
|
||||
/// shared name across cities doesn't cross-pollinate.
|
||||
///
|
||||
/// Boost formula: each qualifying playbook contributes
|
||||
/// `similarity * base_weight / n_workers` to each worker it endorsed,
|
||||
/// where `base_weight` is tuned to keep the cap realistic without
|
||||
/// forcing every result to saturate. Total per worker is capped at
|
||||
/// `MAX_BOOST_PER_WORKER`.
|
||||
pub async fn compute_boost_for(
|
||||
&self,
|
||||
query_embedding: &[f32],
|
||||
top_k_playbooks: usize,
|
||||
base_weight: f32,
|
||||
) -> HashMap<(String, String, String), BoostEntry> {
|
||||
let entries = self.state.read().await.entries.clone();
|
||||
|
||||
// Brute-force cosine. Empty / missing embeddings just skip.
|
||||
let mut scored: Vec<(f32, &PlaybookEntry)> = entries
|
||||
.iter()
|
||||
.filter_map(|e| e.embedding.as_ref().map(|v| (cosine(query_embedding, v), e)))
|
||||
.collect();
|
||||
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
|
||||
scored.truncate(top_k_playbooks.max(1));
|
||||
|
||||
let now = chrono::Utc::now();
|
||||
let mut boosts: HashMap<(String, String, String), BoostEntry> = HashMap::new();
|
||||
for (similarity, pb) in &scored {
|
||||
// Negative or near-zero similarity = not actually related;
|
||||
// skip so we don't inject noise when the memory is sparse.
|
||||
if *similarity <= 0.05 { continue; }
|
||||
let Some(city) = &pb.city else { continue; };
|
||||
let Some(state) = &pb.state else { continue; };
|
||||
let n_workers = pb.endorsed_names.len().max(1);
|
||||
// Path 1 — temporal decay. Older playbooks weight less. Failure
|
||||
// to parse the timestamp degrades to "no decay" (treat as fresh)
|
||||
// rather than dropping the entry entirely; keeps backward
|
||||
// compatibility with seed payloads that omitted timestamp.
|
||||
let decay = chrono::DateTime::parse_from_rfc3339(&pb.timestamp)
|
||||
.ok()
|
||||
.map(|t| {
|
||||
let age_days = (now.signed_duration_since(t.with_timezone(&chrono::Utc))
|
||||
.num_seconds() as f32) / 86400.0;
|
||||
if age_days <= 0.0 { 1.0 }
|
||||
else { (-age_days / BOOST_HALF_LIFE_DAYS).exp() }
|
||||
})
|
||||
.unwrap_or(1.0);
|
||||
let per_worker = similarity * base_weight * decay / (n_workers as f32);
|
||||
for name in &pb.endorsed_names {
|
||||
let key = (city.clone(), state.clone(), name.clone());
|
||||
let entry = boosts.entry(key).or_insert(BoostEntry {
|
||||
boost: 0.0,
|
||||
citations: Vec::new(),
|
||||
});
|
||||
entry.boost = (entry.boost + per_worker).min(MAX_BOOST_PER_WORKER);
|
||||
if !entry.citations.contains(&pb.playbook_id) {
|
||||
entry.citations.push(pb.playbook_id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
boosts
|
||||
}
|
||||
}
|
||||
|
||||
/// Cosine similarity — pulled out so rebuild/boost share one impl.
|
||||
fn cosine(a: &[f32], b: &[f32]) -> f32 {
|
||||
let (mut dot, mut na, mut nb) = (0.0_f32, 0.0_f32, 0.0_f32);
|
||||
let n = a.len().min(b.len());
|
||||
for i in 0..n {
|
||||
dot += a[i] * b[i];
|
||||
na += a[i] * a[i];
|
||||
nb += b[i] * b[i];
|
||||
}
|
||||
if na == 0.0 || nb == 0.0 { return 0.0; }
|
||||
dot / (na.sqrt() * nb.sqrt())
|
||||
}
|
||||
|
||||
// ---------------- Pattern discovery (Path 2 — meta-index) ----------------
|
||||
//
|
||||
// Phase 19's boost path answers "for THIS exact city + role, which workers
|
||||
// have we used before?" Pattern discovery answers a different question:
|
||||
// "for queries like this one, what TRAITS have past successful fills had
|
||||
// in common — even if no exact prior playbook covers this geo?"
|
||||
//
|
||||
// The discovered pattern surfaces signals the operator didn't query for:
|
||||
// e.g. "every successful Welder fill we've seen carried OSHA-10 + lockout
|
||||
// /tagout — you may want to filter on those." That's the meta-index
|
||||
// dimension of the original PRD: identify things we didn't know about.
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct PatternReport {
|
||||
pub query: String,
|
||||
pub matched_playbooks: usize,
|
||||
pub total_workers_examined: usize,
|
||||
pub common_certifications: Vec<TraitFreq>,
|
||||
pub common_skills: Vec<TraitFreq>,
|
||||
pub modal_archetype: Option<String>,
|
||||
pub reliability_p50: f64,
|
||||
pub reliability_min: f64,
|
||||
pub reliability_max: f64,
|
||||
pub matched_playbook_ids: Vec<String>,
|
||||
pub discovered_pattern: String,
|
||||
pub duration_secs: f32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct TraitFreq {
|
||||
pub name: String,
|
||||
pub count: usize,
|
||||
pub frequency: f32,
|
||||
}
|
||||
|
||||
pub async fn discover_patterns(
|
||||
memory: &PlaybookMemory,
|
||||
ai_client: &AiClient,
|
||||
catalog: &catalogd::registry::Registry,
|
||||
buckets: &Arc<storaged::registry::BucketRegistry>,
|
||||
query: &str,
|
||||
top_k_playbooks: usize,
|
||||
min_trait_frequency: f32,
|
||||
) -> Result<PatternReport, String> {
|
||||
let t0 = std::time::Instant::now();
|
||||
|
||||
// 1. Embed the query through the same nomic-embed-text model used
|
||||
// for playbook embeddings, so cosine is meaningful.
|
||||
let resp = ai_client
|
||||
.embed(EmbedRequest { texts: vec![query.into()], model: None })
|
||||
.await
|
||||
.map_err(|e| format!("embed query: {e}"))?;
|
||||
if resp.embeddings.is_empty() {
|
||||
return Err("embed returned no vectors".into());
|
||||
}
|
||||
let qv: Vec<f32> = resp.embeddings[0].iter().map(|x| *x as f32).collect();
|
||||
|
||||
// 2. Find top-K most similar past playbooks (cosine over embeddings).
|
||||
let entries = memory.snapshot().await;
|
||||
let mut scored: Vec<(f32, &PlaybookEntry)> = entries
|
||||
.iter()
|
||||
.filter_map(|e| e.embedding.as_ref().map(|v| (cosine(&qv, v), e)))
|
||||
.collect();
|
||||
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
|
||||
scored.truncate(top_k_playbooks);
|
||||
let matched: Vec<(f32, PlaybookEntry)> = scored
|
||||
.into_iter()
|
||||
.filter(|(s, _)| *s > 0.05)
|
||||
.map(|(s, e)| (s, e.clone()))
|
||||
.collect();
|
||||
|
||||
if matched.is_empty() {
|
||||
return Ok(PatternReport {
|
||||
query: query.into(),
|
||||
matched_playbooks: 0,
|
||||
total_workers_examined: 0,
|
||||
common_certifications: vec![],
|
||||
common_skills: vec![],
|
||||
modal_archetype: None,
|
||||
reliability_p50: 0.0, reliability_min: 0.0, reliability_max: 0.0,
|
||||
matched_playbook_ids: vec![],
|
||||
discovered_pattern: "No similar past playbooks found.".into(),
|
||||
duration_secs: t0.elapsed().as_secs_f32(),
|
||||
});
|
||||
}
|
||||
|
||||
// 3. Pull each endorsed worker's full profile from workers_500k.
|
||||
// Restrict by (name, city, state) tuple so cross-city homonyms
|
||||
// don't pollute the aggregate.
|
||||
let mut conditions: Vec<String> = Vec::new();
|
||||
let mut matched_ids: Vec<String> = Vec::new();
|
||||
for (_, pb) in &matched {
|
||||
matched_ids.push(pb.playbook_id.clone());
|
||||
let (Some(city), Some(state)) = (pb.city.as_ref(), pb.state.as_ref()) else { continue };
|
||||
for name in &pb.endorsed_names {
|
||||
let esc = |s: &str| s.replace('\'', "''");
|
||||
conditions.push(format!(
|
||||
"(name = '{}' AND city = '{}' AND state = '{}')",
|
||||
esc(name), esc(city), esc(state)
|
||||
));
|
||||
}
|
||||
}
|
||||
if conditions.is_empty() {
|
||||
return Ok(PatternReport {
|
||||
query: query.into(),
|
||||
matched_playbooks: matched.len(),
|
||||
total_workers_examined: 0,
|
||||
common_certifications: vec![], common_skills: vec![],
|
||||
modal_archetype: None, reliability_p50: 0.0,
|
||||
reliability_min: 0.0, reliability_max: 0.0,
|
||||
matched_playbook_ids: matched_ids,
|
||||
discovered_pattern: "Matched playbooks but no endorsed names with city/state to lookup.".into(),
|
||||
duration_secs: t0.elapsed().as_secs_f32(),
|
||||
});
|
||||
}
|
||||
|
||||
let sql = format!(
|
||||
"SELECT name, role, city, state, certifications, skills, archetype, \
|
||||
CAST(reliability AS DOUBLE) as reliability \
|
||||
FROM workers_500k WHERE {} LIMIT 500",
|
||||
conditions.join(" OR ")
|
||||
);
|
||||
let engine = queryd::context::QueryEngine::new(
|
||||
catalog.clone(), buckets.clone(), queryd::cache::MemCache::new(0),
|
||||
);
|
||||
let batches = engine.query(&sql).await.map_err(|e| format!("worker lookup: {e}"))?;
|
||||
|
||||
// 4. Aggregate. Pipe-separated cert/skill lists, single-string archetype,
|
||||
// numeric reliability. Frequencies are share-of-workers.
|
||||
use arrow::array::{Array, AsArray};
|
||||
let mut cert_counts: HashMap<String, usize> = HashMap::new();
|
||||
let mut skill_counts: HashMap<String, usize> = HashMap::new();
|
||||
let mut arch_counts: HashMap<String, usize> = HashMap::new();
|
||||
let mut reliabilities: Vec<f64> = Vec::new();
|
||||
let mut total = 0usize;
|
||||
|
||||
let get_string = |b: &arrow::record_batch::RecordBatch, col: &str, row: usize| -> String {
|
||||
let Some(c) = b.column_by_name(col) else { return String::new(); };
|
||||
if let Some(arr) = c.as_string_view_opt() {
|
||||
if arr.is_null(row) { return String::new(); }
|
||||
return arr.value(row).to_string();
|
||||
}
|
||||
if let Some(arr) = c.as_string_opt::<i32>() {
|
||||
if arr.is_null(row) { return String::new(); }
|
||||
return arr.value(row).to_string();
|
||||
}
|
||||
String::new()
|
||||
};
|
||||
let get_f64 = |b: &arrow::record_batch::RecordBatch, col: &str, row: usize| -> f64 {
|
||||
let Some(c) = b.column_by_name(col) else { return 0.0; };
|
||||
if let Some(arr) = c.as_primitive_opt::<arrow::datatypes::Float64Type>() {
|
||||
if arr.is_null(row) { return 0.0; }
|
||||
return arr.value(row);
|
||||
}
|
||||
0.0
|
||||
};
|
||||
|
||||
for b in &batches {
|
||||
for row in 0..b.num_rows() {
|
||||
total += 1;
|
||||
let certs = get_string(b, "certifications", row);
|
||||
for c in certs.split(['|', ',']).map(|s| s.trim()).filter(|s| !s.is_empty() && *s != "none") {
|
||||
*cert_counts.entry(c.to_string()).or_insert(0) += 1;
|
||||
}
|
||||
let skills = get_string(b, "skills", row);
|
||||
for s in skills.split(['|', ',']).map(|s| s.trim()).filter(|s| !s.is_empty()) {
|
||||
*skill_counts.entry(s.to_string()).or_insert(0) += 1;
|
||||
}
|
||||
let arch = get_string(b, "archetype", row);
|
||||
if !arch.is_empty() {
|
||||
*arch_counts.entry(arch).or_insert(0) += 1;
|
||||
}
|
||||
let rel = get_f64(b, "reliability", row);
|
||||
if rel > 0.0 { reliabilities.push(rel); }
|
||||
}
|
||||
}
|
||||
|
||||
let total_f = total.max(1) as f32;
|
||||
let to_freq = |m: HashMap<String, usize>, min: f32| -> Vec<TraitFreq> {
|
||||
let mut v: Vec<TraitFreq> = m.into_iter()
|
||||
.map(|(name, count)| TraitFreq { name, count, frequency: count as f32 / total_f })
|
||||
.filter(|t| t.frequency >= min)
|
||||
.collect();
|
||||
v.sort_by(|a, b| b.count.cmp(&a.count));
|
||||
v.truncate(8);
|
||||
v
|
||||
};
|
||||
let common_certifications = to_freq(cert_counts, min_trait_frequency);
|
||||
let common_skills = to_freq(skill_counts, min_trait_frequency);
|
||||
let modal_archetype = arch_counts.into_iter()
|
||||
.max_by_key(|(_, c)| *c)
|
||||
.map(|(name, _)| name);
|
||||
|
||||
reliabilities.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
|
||||
let p50 = if reliabilities.is_empty() { 0.0 } else { reliabilities[reliabilities.len() / 2] };
|
||||
let rmin = reliabilities.first().copied().unwrap_or(0.0);
|
||||
let rmax = reliabilities.last().copied().unwrap_or(0.0);
|
||||
|
||||
// Build a human-readable discovered-pattern summary
|
||||
let mut parts: Vec<String> = vec![
|
||||
format!("Across {} similar past playbooks ({} workers examined)", matched.len(), total),
|
||||
];
|
||||
if !common_certifications.is_empty() {
|
||||
let head: Vec<String> = common_certifications.iter().take(3)
|
||||
.map(|t| format!("{} ({:.0}%)", t.name, t.frequency * 100.0)).collect();
|
||||
parts.push(format!("recurring certifications: {}", head.join(", ")));
|
||||
}
|
||||
if !common_skills.is_empty() {
|
||||
let head: Vec<String> = common_skills.iter().take(3)
|
||||
.map(|t| format!("{} ({:.0}%)", t.name, t.frequency * 100.0)).collect();
|
||||
parts.push(format!("recurring skills: {}", head.join(", ")));
|
||||
}
|
||||
if let Some(a) = &modal_archetype { parts.push(format!("archetype mostly: {a}")); }
|
||||
if !reliabilities.is_empty() {
|
||||
parts.push(format!("reliability median {:.2} (range {:.2}–{:.2})", p50, rmin, rmax));
|
||||
}
|
||||
let discovered_pattern = parts.join(" · ");
|
||||
|
||||
Ok(PatternReport {
|
||||
query: query.into(),
|
||||
matched_playbooks: matched.len(),
|
||||
total_workers_examined: total,
|
||||
common_certifications, common_skills,
|
||||
modal_archetype, reliability_p50: p50,
|
||||
reliability_min: rmin, reliability_max: rmax,
|
||||
matched_playbook_ids: matched_ids,
|
||||
discovered_pattern,
|
||||
duration_secs: t0.elapsed().as_secs_f32(),
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------- Persist memory → SQL (Path 2 foundation) ----------------
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct PersistReport {
|
||||
pub rows_persisted: usize,
|
||||
pub dataset_name: String,
|
||||
pub fingerprint: String,
|
||||
pub duration_secs: f32,
|
||||
}
|
||||
|
||||
/// Dump current in-memory state to a queryable Parquet under
|
||||
/// `successful_playbooks_live`. Registers fresh objects each call — safe
|
||||
/// because in-memory state is the source of truth here, so REPLACING the
|
||||
/// objects list reflects the real state, not destroying it.
|
||||
///
|
||||
/// Distinct from the existing `successful_playbooks` dataset (which is
|
||||
/// read by `rebuild()`), so this never collides with operator imports of
|
||||
/// historical playbook data. Recruiter-facing SQL surfaces should query
|
||||
/// `successful_playbooks_live` for current operator activity.
|
||||
pub async fn persist_to_sql(
|
||||
memory: &PlaybookMemory,
|
||||
catalog: &catalogd::registry::Registry,
|
||||
) -> Result<PersistReport, String> {
|
||||
use arrow::array::StringArray;
|
||||
use arrow::datatypes::{DataType, Field, Schema};
|
||||
use arrow::record_batch::RecordBatch;
|
||||
|
||||
let t0 = std::time::Instant::now();
|
||||
let entries = memory.snapshot().await;
|
||||
|
||||
let schema = Arc::new(Schema::new(vec![
|
||||
Field::new("timestamp", DataType::Utf8, true),
|
||||
Field::new("operation", DataType::Utf8, true),
|
||||
Field::new("approach", DataType::Utf8, true),
|
||||
Field::new("result", DataType::Utf8, true),
|
||||
Field::new("context", DataType::Utf8, true),
|
||||
]));
|
||||
|
||||
let timestamps: Vec<&str> = entries.iter().map(|e| e.timestamp.as_str()).collect();
|
||||
let operations: Vec<&str> = entries.iter().map(|e| e.operation.as_str()).collect();
|
||||
let approaches: Vec<&str> = entries.iter().map(|e| e.approach.as_str()).collect();
|
||||
let contexts: Vec<&str> = entries.iter().map(|e| e.context.as_str()).collect();
|
||||
// Result column is reconstructed from endorsed_names so SQL queries
|
||||
// against successful_playbooks_live see the same shape as the original
|
||||
// CSV-fed successful_playbooks ("N/N filled → Name1, Name2").
|
||||
let results: Vec<String> = entries.iter().map(|e| {
|
||||
if e.endorsed_names.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
let n = e.endorsed_names.len();
|
||||
format!("{}/{} filled → {}", n, n, e.endorsed_names.join(", "))
|
||||
}
|
||||
}).collect();
|
||||
let result_refs: Vec<&str> = results.iter().map(|s| s.as_str()).collect();
|
||||
|
||||
let batch = RecordBatch::try_new(schema.clone(), vec![
|
||||
Arc::new(StringArray::from(timestamps)),
|
||||
Arc::new(StringArray::from(operations)),
|
||||
Arc::new(StringArray::from(approaches)),
|
||||
Arc::new(StringArray::from(result_refs)),
|
||||
Arc::new(StringArray::from(contexts)),
|
||||
]).map_err(|e| format!("build record batch: {e}"))?;
|
||||
|
||||
let parquet_bytes = shared::arrow_helpers::record_batch_to_parquet(&batch)?;
|
||||
let fp = shared::arrow_helpers::fingerprint_schema(&schema);
|
||||
|
||||
let key = "datasets/successful_playbooks_live.parquet";
|
||||
ops::put(&memory.store, key, parquet_bytes.clone()).await?;
|
||||
|
||||
let obj = shared::types::ObjectRef {
|
||||
bucket: "primary".into(),
|
||||
key: key.into(),
|
||||
size_bytes: parquet_bytes.len() as u64,
|
||||
created_at: chrono::Utc::now(),
|
||||
};
|
||||
|
||||
let manifest = catalog.register(
|
||||
"successful_playbooks_live".into(),
|
||||
fp.clone(),
|
||||
vec![obj],
|
||||
).await?;
|
||||
|
||||
Ok(PersistReport {
|
||||
rows_persisted: entries.len(),
|
||||
dataset_name: manifest.name,
|
||||
fingerprint: fp.0,
|
||||
duration_secs: t0.elapsed().as_secs_f32(),
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------- Rebuild (the core of Phase 19) ----------------
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct RebuildReport {
|
||||
pub rows_scanned: usize,
|
||||
pub entries_built: usize,
|
||||
pub total_names_endorsed: usize,
|
||||
pub duration_secs: f32,
|
||||
}
|
||||
|
||||
/// Full rebuild: scan `successful_playbooks`, extract endorsements, embed
|
||||
/// each row's operation+approach+context, replace the in-memory state.
|
||||
///
|
||||
/// Returns the report so callers can show operators what happened.
|
||||
pub async fn rebuild(
|
||||
memory: &PlaybookMemory,
|
||||
ai_client: &AiClient,
|
||||
catalog: &catalogd::registry::Registry,
|
||||
buckets: &Arc<storaged::registry::BucketRegistry>,
|
||||
) -> Result<RebuildReport, String> {
|
||||
let t0 = std::time::Instant::now();
|
||||
|
||||
// 1. Pull every row of successful_playbooks through the query engine.
|
||||
let sql = "SELECT timestamp, operation, approach, result, context \
|
||||
FROM successful_playbooks";
|
||||
let engine = queryd::context::QueryEngine::new(
|
||||
catalog.clone(),
|
||||
buckets.clone(),
|
||||
queryd::cache::MemCache::new(0),
|
||||
);
|
||||
let batches = engine
|
||||
.query(sql)
|
||||
.await
|
||||
.map_err(|e| format!("query successful_playbooks: {e}"))?;
|
||||
|
||||
let mut rows: Vec<(String, String, String, String, String)> = Vec::new();
|
||||
for b in &batches {
|
||||
let n = b.num_rows();
|
||||
let get = |col: &str, row: usize| -> String {
|
||||
use arrow::array::{Array, AsArray};
|
||||
let Some(c) = b.column_by_name(col) else { return String::new(); };
|
||||
if let Some(arr) = c.as_string_view_opt() {
|
||||
if arr.is_null(row) { return String::new(); }
|
||||
return arr.value(row).to_string();
|
||||
}
|
||||
if let Some(arr) = c.as_string_opt::<i32>() {
|
||||
if arr.is_null(row) { return String::new(); }
|
||||
return arr.value(row).to_string();
|
||||
}
|
||||
String::new()
|
||||
};
|
||||
for row in 0..n {
|
||||
rows.push((
|
||||
get("timestamp", row),
|
||||
get("operation", row),
|
||||
get("approach", row),
|
||||
get("result", row),
|
||||
get("context", row),
|
||||
));
|
||||
}
|
||||
}
|
||||
let rows_scanned = rows.len();
|
||||
|
||||
// 2. For each row, build a PlaybookEntry (no embedding yet). Parse
|
||||
// the operation for (city, state) and the result for names.
|
||||
let mut entries: Vec<PlaybookEntry> = rows
|
||||
.into_iter()
|
||||
.map(|(ts, op, approach, result, ctx)| {
|
||||
let (city, state) = parse_city_state(&op);
|
||||
let names = parse_names(&result);
|
||||
PlaybookEntry {
|
||||
playbook_id: stable_id(&ts, &op),
|
||||
operation: op,
|
||||
approach,
|
||||
context: ctx,
|
||||
timestamp: ts,
|
||||
endorsed_names: names,
|
||||
city,
|
||||
state,
|
||||
embedding: None,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
// 3. Embed in one batch. Sidecar's embed handles batching internally;
|
||||
// chunk here to ~64 per request to keep memory flat.
|
||||
const EMBED_BATCH: usize = 64;
|
||||
for chunk_start in (0..entries.len()).step_by(EMBED_BATCH) {
|
||||
let end = (chunk_start + EMBED_BATCH).min(entries.len());
|
||||
let texts: Vec<String> = entries[chunk_start..end]
|
||||
.iter()
|
||||
.map(embed_text)
|
||||
.collect();
|
||||
let req = EmbedRequest { texts, model: None };
|
||||
let resp = ai_client
|
||||
.embed(req)
|
||||
.await
|
||||
.map_err(|e| format!("embed batch [{chunk_start}..{end}]: {e}"))?;
|
||||
for (i, v) in resp.embeddings.iter().enumerate() {
|
||||
let f32v: Vec<f32> = v.iter().map(|&x| x as f32).collect();
|
||||
entries[chunk_start + i].embedding = Some(f32v);
|
||||
}
|
||||
}
|
||||
|
||||
let total_names_endorsed: usize = entries.iter().map(|e| e.endorsed_names.len()).sum();
|
||||
let entries_built = entries.len();
|
||||
|
||||
memory.set_entries(entries).await?;
|
||||
|
||||
Ok(RebuildReport {
|
||||
rows_scanned,
|
||||
entries_built,
|
||||
total_names_endorsed,
|
||||
duration_secs: t0.elapsed().as_secs_f32(),
|
||||
})
|
||||
}
|
||||
|
||||
fn embed_text(e: &PlaybookEntry) -> String {
|
||||
// Compact one-liner per playbook. Excludes timestamp (no semantic
|
||||
// signal) and includes the fills as words (they're occasionally
|
||||
// meaningful — "Luis Harris" might semantically correlate with
|
||||
// Spanish-speaker names in future queries).
|
||||
format!(
|
||||
"{} | {} | {} | fills: {}",
|
||||
e.operation,
|
||||
e.approach,
|
||||
e.context,
|
||||
e.endorsed_names.join(", "),
|
||||
)
|
||||
}
|
||||
|
||||
/// Derive a stable id from (timestamp, operation). Two playbooks with
|
||||
/// identical timestamp+operation collapse to one — benign dedup.
|
||||
fn stable_id(ts: &str, op: &str) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut h = Sha256::new();
|
||||
h.update(ts.as_bytes());
|
||||
h.update(b"|");
|
||||
h.update(op.as_bytes());
|
||||
let bytes = h.finalize();
|
||||
format!("pb-{}", hex_short(&bytes, 12))
|
||||
}
|
||||
|
||||
fn hex_short(b: &[u8], n: usize) -> String {
|
||||
let mut s = String::with_capacity(n * 2);
|
||||
for byte in &b[..b.len().min(n)] {
|
||||
s.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// Parse "fill: Welder x2 in Toledo, OH" → ("Toledo", "OH").
|
||||
/// Returns None for malformed operations.
|
||||
fn parse_city_state(op: &str) -> (Option<String>, Option<String>) {
|
||||
// Split on " in " then parse "City, ST"
|
||||
let after_in = match op.split(" in ").nth(1) {
|
||||
Some(s) => s,
|
||||
None => return (None, None),
|
||||
};
|
||||
let parts: Vec<&str> = after_in.splitn(2, ',').collect();
|
||||
if parts.len() != 2 {
|
||||
return (None, None);
|
||||
}
|
||||
let city = parts[0].trim().to_string();
|
||||
// state might be followed by more context; take leading alpha chars
|
||||
let state: String = parts[1].trim()
|
||||
.chars()
|
||||
.take_while(|c| c.is_ascii_alphabetic())
|
||||
.collect();
|
||||
if city.is_empty() || state.is_empty() {
|
||||
return (None, None);
|
||||
}
|
||||
(Some(city), Some(state))
|
||||
}
|
||||
|
||||
/// Parse "2/2 filled → Matthew Roberts, Amy Davis" → ["Matthew Roberts", "Amy Davis"].
|
||||
fn parse_names(result: &str) -> Vec<String> {
|
||||
// Everything after the arrow; split on ", ".
|
||||
let after_arrow = match result.split('→').nth(1) {
|
||||
Some(s) => s.trim(),
|
||||
None => return Vec::new(),
|
||||
};
|
||||
// Strip trailing noise like "(and N more)" that some emitters add.
|
||||
let cleaned = after_arrow.split(" (").next().unwrap_or(after_arrow);
|
||||
cleaned
|
||||
.split(',')
|
||||
.map(|n| n.trim().to_string())
|
||||
.filter(|n| !n.is_empty())
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parse_city_state_extracts_both() {
|
||||
let (c, s) = parse_city_state("fill: Welder x2 in Toledo, OH");
|
||||
assert_eq!(c.as_deref(), Some("Toledo"));
|
||||
assert_eq!(s.as_deref(), Some("OH"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_city_state_handles_multiword_city() {
|
||||
let (c, s) = parse_city_state("fill: Loader x1 in Grand Rapids, MI");
|
||||
assert_eq!(c.as_deref(), Some("Grand Rapids"));
|
||||
assert_eq!(s.as_deref(), Some("MI"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_city_state_malformed_returns_none() {
|
||||
let (c, s) = parse_city_state("fill: something weird");
|
||||
assert!(c.is_none());
|
||||
assert!(s.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_names_extracts_after_arrow() {
|
||||
let ns = parse_names("2/2 filled → Matthew Roberts, Amy Davis");
|
||||
assert_eq!(ns, vec!["Matthew Roberts".to_string(), "Amy Davis".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_names_handles_single_fill() {
|
||||
let ns = parse_names("1/1 filled → Jose Reed");
|
||||
assert_eq!(ns, vec!["Jose Reed".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_names_handles_no_arrow() {
|
||||
let ns = parse_names("0/2 filled");
|
||||
assert!(ns.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stable_id_is_deterministic() {
|
||||
let a = stable_id("2026-04-20T00:00:00Z", "fill: Welder x2 in Toledo, OH");
|
||||
let b = stable_id("2026-04-20T00:00:00Z", "fill: Welder x2 in Toledo, OH");
|
||||
assert_eq!(a, b);
|
||||
assert!(a.starts_with("pb-"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn boost_caps_per_worker() {
|
||||
// Even with 100 similar playbooks all endorsing the same name, the
|
||||
// boost never exceeds MAX_BOOST_PER_WORKER.
|
||||
let pm = PlaybookMemory::new(Arc::new(object_store::memory::InMemory::new()));
|
||||
let entries: Vec<PlaybookEntry> = (0..100)
|
||||
.map(|i| PlaybookEntry {
|
||||
playbook_id: format!("pb-{i}"),
|
||||
operation: "fill: Welder x1 in Toledo, OH".into(),
|
||||
approach: "transfer".into(),
|
||||
context: "".into(),
|
||||
timestamp: "2026-04-20".into(),
|
||||
endorsed_names: vec!["Deborah Powell".into()],
|
||||
city: Some("Toledo".into()),
|
||||
state: Some("OH".into()),
|
||||
embedding: Some(vec![1.0, 0.0, 0.0]),
|
||||
})
|
||||
.collect();
|
||||
tokio::runtime::Runtime::new().unwrap().block_on(async {
|
||||
pm.set_entries(entries).await.unwrap();
|
||||
let boosts = pm.compute_boost_for(&[1.0, 0.0, 0.0], 100, 0.5).await;
|
||||
let key = ("Toledo".into(), "OH".into(), "Deborah Powell".into());
|
||||
let entry = boosts.get(&key).expect("boost entry present");
|
||||
assert!(entry.boost <= MAX_BOOST_PER_WORKER + 1e-6,
|
||||
"boost {} exceeded cap {}", entry.boost, MAX_BOOST_PER_WORKER);
|
||||
});
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user