Compare commits

..
Author SHA1 Message Date
profit f0a3ed6832 Fix: UpsertOutcome newtype variants panicked serde from Phase 26
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "Verified live after gateway restart:"
playbook_memory.rs:257 — UpsertOutcome had two newtype variants
carrying a bare String:
  Added(String)
  Noop(String)
under #[serde(tag = "mode")]. serde cannot tag newtype variants of
primitive types, so every serialization threw:
  "cannot serialize tagged newtype variant UpsertOutcome::Added
   containing a string"
This caused gateway /vectors/playbook_memory/seed to panic the
tokio worker on EVERY call that reached Added or Noop, returning
an empty socket close to the client. The bug was silent from commit
640db8c (Phase 26, 2026-04-21) until 2026-04-22 when the auditor's
hybrid fixture (auditor/fixtures/hybrid_38_40_45.ts on the
auditor/scaffold branch) exercised the endpoint live and gateway
logs showed the panic.

Fix — convert both newtype variants to struct-like:
  Added { playbook_id: String }
  Noop { playbook_id: String }
Updated all 7 construction + pattern-match sites. Updated rustdoc
on the enum explaining why the shape is what it is.

JSON wire format is now uniform across all three variants:
  {"mode":"added","playbook_id":"pb-..."}
  {"mode":"updated","playbook_id":"pb-...","merged_names":[...]}
  {"mode":"noop","playbook_id":"pb-..."}

Verified live after gateway restart:
  curl /seed new payload               → mode=added, playbook 860231f5
  curl /seed new payload + doc_refs    → mode=added, playbook 11d348d9
  curl /seed identical re-submit       → mode=noop,  same id 860231f5,
                                         entries_after unchanged (Mem0
                                         contract intact)

Tests: 51/51 vectord lib tests green. Release build clean.

This is a follow-up bug fix landed in its own branch
(fix/upsert-outcome-serde) rather than commingled with other work.
The auditor's hybrid fixture on the auditor/scaffold branch will
now light up layer 3 (phase45_seed_with_doc_refs) as a pass once
this merges — previously it failed here with an empty socket close.
2026-04-22 03:48:05 -05:00
profitandClaude Opus 4.7 affab8ac83 Phase 45 slice 2: context7 HTTP bridge for doc drift detection
Bun bridge on :3900 that wraps context7's public API and exposes the
surface gateway consumes for Phase 45 drift checks. Own port so a
failure here never tips over mcp-server on :3700.

Endpoints:
  GET /health                    status + cache stats
  GET /docs/:tool                resolve tool → library_id → fetch
                                 docs → return descriptor
                                 {snippet_hash, last_updated,
                                 source_url, docs_preview, ...}
  GET /docs/:tool/diff?since=X   compare current snippet_hash to X;
                                 returns {drifted: bool, current,
                                 previous, preview if drifted}
  GET /cache                     debug dump of cached entries

Implementation notes:
- 5 minute in-memory cache (context7 rate-limits by IP; gateway
  drift-checks are the hot caller)
- 1500-token slices from context7 (enough for drift-meaningful
  hash, not so much we hammer their API)
- snippet_hash = SHA-256 prefix (16 hex chars) of fetched content
- Library resolution prefers "finalized" state; falls back to top
  result if none finalized

Verified live against context7.com:
- /health                                  → ok, 0 cache, 300s TTL
- /docs/docker                             → library_id /docker/docs,
                                             title "Docker", hash
                                             475a0396ca436bba, last
                                             updated 2026-04-20
- /docs/docker (again)                     → cache hit, 0.37ms
                                             (5400× speedup)
- /docs/docker/diff?since=stale-hash-0000  → drifted=true, preview
                                             included
- /docs/docker/diff?since=<current hash>   → drifted=false, preview
                                             omitted (honest: no
                                             drift to show)

Not yet wired:
- Gateway consumer (Phase 45 slice 3):
  /vectors/playbook_memory/doc_drift/check/{id} calls this bridge
  and updates DocRef.snippet_hash + doc_drift_flagged_at
- Systemd unit (bridge is manual-start for now, same as bot/)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 03:17:17 -05:00
profitandClaude Opus 4.7 2a4b81bf48 Phase 45 (first slice): DocRef + doc_refs field on PlaybookEntry
Phase J keeps asking for: playbooks know which external docs they
used, get flagged when those docs drift. This commit ships the data
model; context7 bridge + drift check endpoints land in follow-ups.

Added to crates/vectord/src/playbook_memory.rs:
- pub struct DocRef { tool, version_seen, snippet_hash, source_url,
  seen_at } — one external doc reference
- PlaybookEntry.doc_refs: Vec<DocRef> — empty on legacy entries,
  serde default ensures pre-Phase-45 persisted state loads cleanly
- PlaybookEntry.doc_drift_flagged_at: Option<String> — set by the
  (future) drift-check code when context7 reports newer version
- PlaybookEntry.doc_drift_reviewed_at: Option<String> — set by
  human via /resolve endpoint after reviewing the diagnosis
- impl Default for PlaybookEntry — collapses most test-helper
  constructors from 17 explicit fields to 6-9 fields +
  ..Default::default()

Updated SeedPlaybookRequest + RevisePlaybookRequest (service.rs) to
accept optional doc_refs: the seed/revise endpoints already take the
field, downstream drift detection (Phase 45.2) consumes it.

Docs: docs/CONTROL_PLANE_PRD.md gains full Phase 45 spec with gate
criteria, non-goals, and risk notes.

Tests: 51/51 vectord lib tests green (same count as before, field
additions are backward-compat).

Memory: project_doc_drift_vision.md written so this keeps coming
back to the front of mind.

Next slices (same phase): context7 HTTP bridge in mcp-server,
/vectors/playbook_memory/doc_drift/check/{id} endpoint, overview-
model drift synthesis writing to data/_kb/doc_drift_corrections.jsonl,
boost exclusion for flagged+unreviewed entries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 03:14:07 -05:00
4 changed files with 336 additions and 77 deletions
+104 -53
View File
@@ -134,10 +134,87 @@ pub struct PlaybookEntry {
/// full version chain.
#[serde(default)]
pub superseded_by: Option<String>,
/// Phase 45 — external documentation references captured at seal
/// time. One entry per tool/library the procedure consulted.
/// Drives drift detection: when context7 reports a newer version
/// for any entry here than what's in `version_seen`, the playbook
/// is `doc_drift_flagged_at` and excluded from boost until human
/// review clears it. Legacy entries (pre-Phase-45) load with an
/// empty vec — they simply never drift-flag, same as entries
/// without a `schema_fingerprint` in Phase 25.
#[serde(default)]
pub doc_refs: Vec<DocRef>,
/// Phase 45 — set by `flag_doc_drift()` when one or more
/// `doc_refs` entries have a newer version available than
/// `version_seen`. Flagged entries are excluded from boost until
/// `doc_drift_reviewed_at` is set via the /resolve endpoint.
#[serde(default)]
pub doc_drift_flagged_at: Option<String>,
/// Phase 45 — set by human operator via
/// `/vectors/playbook_memory/doc_drift/resolve/{id}` after
/// reviewing the drift diagnosis. Either re-admits the entry to
/// boost (if still applicable) or pairs with `retired_at` /
/// `superseded_by` if the procedure changed.
#[serde(default)]
pub doc_drift_reviewed_at: Option<String>,
}
fn default_version() -> u32 { 1 }
/// Phase 45 — one external doc reference. Recorded at seal time so
/// drift detection knows what version was consulted. `snippet_hash`
/// lets us detect "same version, different passage" when a library
/// patches docs without bumping the version number.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct DocRef {
/// Canonical tool/library name as context7 knows it, e.g.
/// "docker", "terraform", "react", "next.js". Case-insensitive
/// on compare.
pub tool: String,
/// Version string exactly as seen at seal time. Context7 typically
/// returns semver-like; we store raw string to avoid parsing
/// ambiguity ("latest", "next", "canary" are all valid).
pub version_seen: String,
/// Optional hash of the specific doc passage the procedure
/// referenced. Useful when version hasn't bumped but content
/// rewrote.
#[serde(default)]
pub snippet_hash: Option<String>,
/// Optional direct URL back to the doc (context7 can resolve
/// tool+version → URL, so this is cache not source-of-truth).
#[serde(default)]
pub source_url: Option<String>,
/// When this reference was captured. RFC3339.
pub seen_at: String,
}
impl Default for PlaybookEntry {
fn default() -> Self {
Self {
playbook_id: String::new(),
operation: String::new(),
approach: String::new(),
context: String::new(),
timestamp: String::new(),
endorsed_names: Vec::new(),
city: None,
state: None,
embedding: None,
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
doc_refs: Vec::new(),
doc_drift_flagged_at: None,
doc_drift_reviewed_at: None,
}
}
}
/// A recorded failure — worker who didn't deliver on a contract.
/// Tracked per (city, state, name) so a single worker's failures on
/// Toledo Welder contracts don't penalize the same name in Chicago.
@@ -175,11 +252,21 @@ pub struct BoostEntry {
/// Phase 26 — what happened during an upsert. The seed endpoint
/// returns this shape so the caller sees whether its write was a new
/// entry, a merge, or a dedup'd no-op.
///
/// SHAPE NOTE: `#[serde(tag = "mode")]` requires struct-like variants —
/// bare `Added(String)` and `Noop(String)` newtype variants would
/// panic serialization at runtime. That bug silently 500-ed every
/// /seed call from Phase 26 (commit 640db8c) until 2026-04-22 when the
/// auditor's hybrid fixture surfaced it. All three variants are now
/// struct-like, producing uniform JSON:
/// {"mode":"added","playbook_id":"pb-..."}
/// {"mode":"updated","playbook_id":"pb-...","merged_names":[...]}
/// {"mode":"noop","playbook_id":"pb-..."}
#[derive(Debug, Clone, Serialize)]
#[serde(tag = "mode", rename_all = "lowercase")]
pub enum UpsertOutcome {
/// New playbook appended. Carries the new playbook_id.
Added(String),
Added { playbook_id: String },
/// Existing same-day entry updated. Playbook_id unchanged; names
/// merged (union, original order preserved, new names appended).
Updated {
@@ -188,7 +275,7 @@ pub enum UpsertOutcome {
},
/// Identical same-day entry already exists; nothing changed.
/// Returns the stable playbook_id so caller still has a reference.
Noop(String),
Noop { playbook_id: String },
}
/// Phase 27 — shape returned from `revise_entry`. Reports both ends of
@@ -520,7 +607,7 @@ impl PlaybookMemory {
drop(state);
self.persist().await?;
self.rebuild_geo_index().await;
Ok(UpsertOutcome::Added(pid))
Ok(UpsertOutcome::Added { playbook_id: pid })
}
Some(i) => {
let mut existing_names_sorted = state.entries[i].endorsed_names.clone();
@@ -528,7 +615,7 @@ impl PlaybookMemory {
if existing_names_sorted == new_names_sorted {
// NOOP — identical data, just report the existing id
let pid = state.entries[i].playbook_id.clone();
Ok(UpsertOutcome::Noop(pid))
Ok(UpsertOutcome::Noop { playbook_id: pid })
} else {
// UPDATE — merge names (union, stable order).
let existing = state.entries.get_mut(i).ok_or("index invalidated")?;
@@ -1201,19 +1288,10 @@ pub async fn rebuild(
endorsed_names: names,
city,
state,
embedding: None,
// Rebuild doesn't know fingerprints; historical entries
// get no retirement signal until a seed with a
// fingerprint supersedes them or the operator calls
// /retire manually.
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
// Rebuild doesn't know fingerprints or doc_refs;
// historical entries get no drift signal until a seed
// supersedes them or /retire is called manually.
..Default::default()
}
})
.collect();
@@ -1386,20 +1464,12 @@ mod tests {
playbook_id: format!("pb-{i}"),
operation: "fill: Welder x1 in Toledo, OH".into(),
approach: "transfer".into(),
context: "".into(),
timestamp: "2026-04-20".into(),
endorsed_names: vec!["Deborah Powell".into()],
city: Some("Toledo".into()),
state: Some("OH".into()),
embedding: Some(vec![1.0, 0.0, 0.0]),
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
..Default::default()
})
.collect();
tokio::runtime::Runtime::new().unwrap().block_on(async {
@@ -1431,12 +1501,7 @@ mod validity_window_tests {
embedding: Some(vec![1.0, 0.0, 0.0]),
schema_fingerprint: fingerprint,
valid_until,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
..Default::default()
}
}
@@ -1536,14 +1601,7 @@ mod upsert_tests {
city: Some("Nashville".into()),
state: Some("TN".into()),
embedding: Some(vec![1.0, 0.0, 0.0]),
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
..Default::default()
}
}
@@ -1552,7 +1610,7 @@ mod upsert_tests {
let pm = PlaybookMemory::new(Arc::new(InMemory::new()));
let e = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith"]);
match pm.upsert_entry(e).await.unwrap() {
UpsertOutcome::Added(_) => {}
UpsertOutcome::Added { .. } => {}
other => panic!("expected Added, got {:?}", other),
}
assert_eq!(pm.entry_count().await, 1);
@@ -1565,7 +1623,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]);
pm.upsert_entry(e1).await.unwrap();
let outcome = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(outcome, UpsertOutcome::Noop(_)));
assert!(matches!(outcome, UpsertOutcome::Noop { .. }));
// Still exactly one entry, no duplicate from the re-seed.
assert_eq!(pm.entry_count().await, 1);
}
@@ -1577,7 +1635,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Alice Smith", "Bob Jones"]);
let o1 = pm.upsert_entry(e1).await.unwrap();
let pid = match o1 {
UpsertOutcome::Added(p) => p,
UpsertOutcome::Added { playbook_id: p } => p,
other => panic!("expected Added, got {:?}", other),
};
let o2 = pm.upsert_entry(e2).await.unwrap();
@@ -1598,7 +1656,7 @@ mod upsert_tests {
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-22", &["Alice Smith"]);
pm.upsert_entry(e1).await.unwrap();
let o2 = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o2, UpsertOutcome::Added(_)), "different day → fresh ADD");
assert!(matches!(o2, UpsertOutcome::Added { .. }), "different day → fresh ADD");
assert_eq!(pm.entry_count().await, 2);
}
@@ -1611,7 +1669,7 @@ mod upsert_tests {
// A new seed on same day should ADD, not merge into the retired one.
let e2 = mk("fill: Welder x2 in Nashville, TN", "2026-04-21", &["Carol Davis"]);
let o = pm.upsert_entry(e2).await.unwrap();
assert!(matches!(o, UpsertOutcome::Added(_)));
assert!(matches!(o, UpsertOutcome::Added { .. }));
assert_eq!(pm.entry_count().await, 2);
}
}
@@ -1632,14 +1690,7 @@ mod version_tests {
city: Some(city.into()),
state: Some(state.into()),
embedding: Some(vec![1.0, 0.0, 0.0]),
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
..Default::default()
}
}
+22 -24
View File
@@ -2211,6 +2211,12 @@ struct SeedPlaybookRequest {
/// retired, just inactive). Useful for seasonal/temp contracts.
#[serde(default)]
valid_until: Option<String>,
/// Phase 45 — optional external doc references captured at seal
/// time. Each entry names a tool + version_seen; context7-driven
/// drift check compares against current versions later. None or
/// empty = no drift signal (never flagged).
#[serde(default)]
doc_refs: Option<Vec<playbook_memory::DocRef>>,
}
/// Bootstrap / test-only: inject a playbook entry directly into
@@ -2232,21 +2238,12 @@ async fn seed_playbook_memory(
// Embed the entry through the same text shape `rebuild` uses so
// similarity math is comparable across seed + real entries.
let tmp_entry = playbook_memory::PlaybookEntry {
playbook_id: String::new(),
operation: req.operation.clone(),
approach: req.approach.clone(),
context: req.context.clone(),
timestamp: chrono::Utc::now().to_rfc3339(),
endorsed_names: req.endorsed_names.clone(),
city: None, state: None, embedding: None,
schema_fingerprint: None,
valid_until: None,
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
..Default::default()
};
let text = format!(
"{} | {} | {} | fills: {}",
@@ -2304,12 +2301,11 @@ async fn seed_playbook_memory(
// works). valid_until + retired_at start None.
schema_fingerprint: req.schema_fingerprint.clone(),
valid_until: req.valid_until.clone(),
retired_at: None,
retirement_reason: None,
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
// Phase 45 — seed request may also carry doc_refs; defaults
// empty so pre-Phase-45 callers still work and the entry
// degrades to "no drift signal" (never flagged).
doc_refs: req.doc_refs.clone().unwrap_or_default(),
..Default::default()
};
// Phase 26 — when append=true (default), route through upsert so
@@ -2521,6 +2517,11 @@ struct RevisePlaybookRequest {
schema_fingerprint: Option<String>,
#[serde(default)]
valid_until: Option<String>,
/// Phase 45 — updated doc references. Typically a revise happens
/// BECAUSE docs drifted; pass the new versions seen so the revised
/// entry starts with fresh drift signal.
#[serde(default)]
doc_refs: Option<Vec<playbook_memory::DocRef>>,
}
/// Phase 27 — create a new version of an existing playbook. The parent
@@ -2613,14 +2614,11 @@ async fn revise_playbook_memory(
embedding: Some(emb),
schema_fingerprint: req.schema_fingerprint,
valid_until: req.valid_until,
retired_at: None,
retirement_reason: None,
// revise_entry overwrites these from the parent — values here
// are just placeholders so the struct is well-formed.
version: 1,
parent_id: None,
superseded_at: None,
superseded_by: None,
// Phase 45 — doc_refs may be provided on revise too.
doc_refs: req.doc_refs.clone().unwrap_or_default(),
// revise_entry overwrites version / parent_id / supersession
// from the parent; other fields keep defaults.
..Default::default()
};
let outcome = state.playbook_memory.revise_entry(&req.parent_id, new_entry)
+32
View File
@@ -217,6 +217,38 @@ Ship each phase before starting the next. Each ends with green tests + docs upda
---
## Phase 45 — Doc-drift detection + context7 integration
**Goal:** Playbooks know which external docs they were written against. When those docs change (Docker adds a feature, npm lib goes major, Terraform renames a resource), the playbook is automatically flagged. Small models never run confidently-outdated procedures — the drift signal reaches them before the next execution does.
**Why this phase exists at all:** The 0→85% thesis depends on the hyperfocus lane staying valid. External doc drift invalidates the lane silently — popular playbooks can compound the wrong way, accumulating boost while growing more wrong. Phase 25 already retires playbooks on *internal* schema drift; Phase 45 is the same mechanism against *external* doc drift. This is the completion of the learning loop, not an optional add-on.
**Ships:**
- `shared::types::DocRef` — `{ tool: String, version_seen: String, snippet_hash: Option<String>, source_url: Option<String>, seen_at: DateTime<Utc> }`
- `PlaybookEntry.doc_refs: Vec<DocRef>` — `#[serde(default)]` so pre-Phase-45 entries load as empty vec
- `/vectors/playbook_memory/seed` + `/revise` accept `doc_refs` in the request body
- `/vectors/playbook_memory/doc_drift/check/{id}` — manual drift check: looks up each `doc_refs[]` entry via the context7 bridge, returns per-tool `{version_seen, version_current, drifted: bool}` plus overall verdict
- `/vectors/playbook_memory/doc_drift/scan` — batch scan across all active playbooks (scheduled path for Phase 45.2)
- `mcp-server/context7_bridge.ts` — Bun HTTP bridge. Exposes `GET /docs/:tool/version` + `GET /docs/:tool/:version/diff?since=X` against the installed context7 MCP plugin. Gateway calls this over localhost.
- `PlaybookMemory::compute_boost_for_filtered_with_role` — excludes entries where `doc_drift_flagged_at.is_some() && doc_drift_review.is_none()` (same rule as retired + superseded)
- Overview model synthesis writes `data/_kb/doc_drift_corrections.jsonl` per detected drift: `{playbook_id, tool, version_seen, version_current, diff_summary, recommended_action, generated_at}`
- Human-in-the-loop re-seal path: `/vectors/playbook_memory/doc_drift/resolve/{id}` — marks reviewed, optionally triggers `revise_entry` if procedure changed
**Gate:**
- Seal a playbook referencing Docker 24.x → doc_refs captured. Bump Docker version behind the scenes → `/doc_drift/check/{id}` returns `drifted: true, from: 24.0.7, to: 25.0.1, summary: "..."`. The boosted playbook count on next `/vectors/hybrid` query drops by 1 (drift-flagged skipped).
- `doc_drift_corrections.jsonl` contains the overview model's synthesis for the drift with at least: summary of change, recommended action, cost/impact estimate.
- Human calls `/doc_drift/resolve/{id}` after reviewing → playbook returns to active boost pool (or supersedes via Phase 27 if procedure materially changed).
- Unit tests: DocRef serde default (legacy entries load as empty), drift check against mocked context7 bridge, boost exclusion when drifted+unreviewed.
**Non-goals (explicit):**
- Automatic re-seal without human review. Drift-detection → flag, not silent rewrite.
- Cross-playbook propagation of one drift diagnosis. Each playbook reviewed individually (aggregation later if warranted).
- Generating the updated procedure. T3 *suggests*; human or separate bot (see `bot/`) *writes*.
**Risk:** Medium. The context7 bridge is new infrastructure (Bun ↔ context7 MCP plugin ↔ HTTP shape for gateway consumption). Mitigation: context7 plugin is already installed; its MCP tools return structured JSON; the bridge is thin adapter code. Start with single-tool drift check (Docker) before broadening.
---
## Long-horizon domains (not in current phase sequence)
The architecture was drafted with DevOps execution (Terraform, Ansible) as the eventual target. **That remains aspirational, not current scope** — we don't start wiring `terraform validate` / `ansible-lint` until the staffing domain proves the six-layer architecture at scale.
+178
View File
@@ -0,0 +1,178 @@
// Phase 45 slice 2 — context7 HTTP bridge.
//
// Exposes an HTTP surface the Rust gateway consumes to check external
// doc drift on playbooks. Wraps context7's public API:
//
// https://context7.com/api/v1/search?query=<name> → resolve
// https://context7.com/api/v1/<lib-id>/cli?type=txt&tokens=N → docs
//
// Own port so a failure here never tips over the mcp-server on :3700.
// Cache is in-memory (5 min TTL) — context7 rate-limits by IP, and
// gateway drift-checks are the hot caller.
//
// Endpoints:
// GET /health health + cache stats
// GET /docs/:tool resolve + fetch + return descriptor
// GET /docs/:tool/diff?since=X return drift vs content hash X
// GET /cache dump current cache (debugging)
import { createHash } from "node:crypto";
const PORT = Number(process.env.CONTEXT7_BRIDGE_PORT ?? 3900);
const CONTEXT7_BASE = (process.env.CONTEXT7_BASE_URL ?? "https://context7.com/api/v1").replace(/\/+$/, "");
const TOKENS_PER_FETCH = 1500; // enough for a drift-meaningful slice, not so much we hammer
const CACHE_TTL_MS = 5 * 60 * 1000;
interface CachedDoc {
tool: string;
library_id: string; // context7 canonical, e.g. "/docker/docs"
title: string;
last_updated: string | null;
snippet_hash: string; // SHA-256 of first TOKENS_PER_FETCH of fetched docs
docs_preview: string; // first 400 chars, returned in responses for display
retrieved_at: string; // ISO when WE last fetched
source_url: string; // human-facing context7 URL
}
const cache = new Map<string, { entry: CachedDoc; at: number }>();
function normalizeTool(s: string): string {
return s.trim().toLowerCase();
}
async function resolveLibraryId(tool: string): Promise<{ id: string; title: string; last_updated: string | null } | null> {
const r = await fetch(`${CONTEXT7_BASE}/search?query=${encodeURIComponent(tool)}`, {
signal: AbortSignal.timeout(10000),
});
if (!r.ok) throw new Error(`context7 search ${r.status}: ${await r.text().catch(() => "?")}`);
const j = await r.json() as any;
const results = Array.isArray(j.results) ? j.results : [];
// First result is top-ranked; context7 sorts by its internal score.
// Skip "state != finalized" to avoid unstable previews.
const pick = results.find((x: any) => x?.state === "finalized") ?? results[0];
if (!pick?.id) return null;
return {
id: String(pick.id),
title: String(pick.title ?? tool),
last_updated: typeof pick.lastUpdateDate === "string" ? pick.lastUpdateDate : null,
};
}
async function fetchDocsText(libraryId: string): Promise<string> {
// Strip the leading slash on library_id so the URL doesn't have a
// double slash. context7 returns ids like "/docker/docs" but the
// fetch path wants "docker/docs".
const cleanId = libraryId.replace(/^\/+/, "");
const url = `${CONTEXT7_BASE}/${cleanId}?type=txt&tokens=${TOKENS_PER_FETCH}`;
const r = await fetch(url, { signal: AbortSignal.timeout(20000) });
if (!r.ok) throw new Error(`context7 fetch ${r.status} on ${cleanId}: ${await r.text().catch(() => "?")}`);
return await r.text();
}
function hashContent(s: string): string {
return createHash("sha256").update(s).digest("hex").slice(0, 16);
}
async function getCurrent(tool: string): Promise<CachedDoc> {
const key = normalizeTool(tool);
const cached = cache.get(key);
if (cached && Date.now() - cached.at < CACHE_TTL_MS) {
return cached.entry;
}
const resolved = await resolveLibraryId(key);
if (!resolved) {
throw new Error(`no context7 library found for '${tool}'`);
}
const text = await fetchDocsText(resolved.id);
const entry: CachedDoc = {
tool: key,
library_id: resolved.id,
title: resolved.title,
last_updated: resolved.last_updated,
snippet_hash: hashContent(text),
docs_preview: text.slice(0, 400),
retrieved_at: new Date().toISOString(),
source_url: `https://context7.com${resolved.id.startsWith("/") ? resolved.id : "/" + resolved.id}`,
};
cache.set(key, { entry, at: Date.now() });
return entry;
}
function jsonResponse(body: unknown, status: number = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
Bun.serve({
port: PORT,
hostname: "0.0.0.0",
async fetch(req) {
const url = new URL(req.url);
if (url.pathname === "/health") {
return jsonResponse({
status: "ok",
cache_size: cache.size,
context7_base: CONTEXT7_BASE,
ttl_ms: CACHE_TTL_MS,
});
}
if (url.pathname === "/cache") {
const dump = Array.from(cache.entries()).map(([k, v]) => ({
tool: k,
library_id: v.entry.library_id,
snippet_hash: v.entry.snippet_hash,
retrieved_at: v.entry.retrieved_at,
age_ms: Date.now() - v.at,
}));
return jsonResponse({ entries: dump });
}
// GET /docs/:tool — return current descriptor (fetches + caches)
const m1 = url.pathname.match(/^\/docs\/([^/]+)$/);
if (m1 && req.method === "GET") {
const tool = decodeURIComponent(m1[1]);
try {
const entry = await getCurrent(tool);
return jsonResponse(entry);
} catch (e) {
return jsonResponse({ error: (e as Error).message }, 404);
}
}
// GET /docs/:tool/diff?since=hash — compare current vs recorded hash
const m2 = url.pathname.match(/^\/docs\/([^/]+)\/diff$/);
if (m2 && req.method === "GET") {
const tool = decodeURIComponent(m2[1]);
const since = url.searchParams.get("since");
if (!since) {
return jsonResponse({ error: "query param 'since' (snippet_hash) required" }, 400);
}
try {
const entry = await getCurrent(tool);
const drifted = entry.snippet_hash !== since;
return jsonResponse({
tool: entry.tool,
drifted,
previous_snippet_hash: since,
current_snippet_hash: entry.snippet_hash,
library_id: entry.library_id,
title: entry.title,
last_updated_upstream: entry.last_updated,
retrieved_at: entry.retrieved_at,
source_url: entry.source_url,
docs_preview: drifted ? entry.docs_preview : null,
});
} catch (e) {
return jsonResponse({ error: (e as Error).message }, 404);
}
}
return jsonResponse({ error: "not found", paths: ["/health", "/docs/:tool", "/docs/:tool/diff?since=HASH", "/cache"] }, 404);
},
});
console.log(`[context7-bridge] listening on :${PORT} (context7 base: ${CONTEXT7_BASE})`);