profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: sync ADR-019 + PRD + DECISIONS with 2026-05-02 substrate changes

Browse Source 
ADR-019: closed the "re-bench when 10M corpus exists" follow-up. Added
"Follow-up: 10M re-bench (2026-05-02)" section with the post-fix numbers
(search ~20ms warm / ~46ms cold, doc-fetch ~5ms post-btree). Documented
the lance-bench-bypassing-IndexMeta bug + 2-layer fix + gauntlet
(7 unit + 12 sanitize + 10 smoke probes). Reframes the strategic
question as "Lance vs Parquet+HNSW-with-spilling" since HNSW doesn't
fit RAM at 10M.

DECISIONS: added ADR-022 — drop Python sidecar from Rust hot path.
Captures the rationale (236× embed perf gap was pure overhead),
co-shipped LRU cache, dev-only Python that survives, cross-runtime
parity verification, and the operator runbook signal (ps -ef ABSENT
post-deploy).

PRD: updated AI Boundary table line + aibridge crate description to
reflect direct Ollama path (was: Python FastAPI sidecar → Ollama).
Both lines reference ADR-022 for the full rationale.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-05-03 00:44:57 -05:00
co-authored by Claude Opus 4.7
parent e9d17f7d5a
commit 5368aca4d4
3 changed files with 31 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PRD.md
+2 -2
View File
@@ -77,7 +77,7 @@ A modular Rust service mesh over S3-compatible object storage, with a local AI l
| Data Format | Parquet + Arrow | Yes |
| RPC (internal) | tonic (gRPC) | Yes |
| AI Runtime | Ollama (local models) | Yes |
| AI Boundary | Python FastAPI sidecar → Ollama HTTP API | Yes |
| AI Boundary | Direct Ollama HTTP API (gateway → Ollama, no sidecar). Updated 2026-05-02 per ADR-022 — was "Python FastAPI sidecar → Ollama HTTP API". Sidecar's lab_ui/pipeline_lab Python remain as dev-only tools (not on hot path). | Yes |
| Vector Index | TBD — evaluate `hora`, `qdrant` crate, or HNSW from scratch | **Open** |
No new frameworks without documented ADR.
@@ -97,7 +97,7 @@ No new frameworks without documented ADR.
| **ingestd** | Ingest pipeline: CSV / JSON / PDF / Postgres-stream → normalize → Parquet → catalog |
| **vectord** | Embedding store + vector indexes + HNSW trial system (EmbeddingCache, trial journal, eval harness) |
| **journald** | Append-only mutation event log (ADR-012) — distinct from storaged error journal |
| **aibridge** | Rust↔Python boundary — HTTP client to FastAPI sidecar |
| **aibridge** | AI client — direct Ollama HTTP (per ADR-022, 2026-05-02; was Rust↔Python sidecar boundary). Owns LRU embed cache. |
| **ui** | Dioxus frontend — Ask, Explore, SQL, System tabs |
| **shared** | Types, errors, Arrow helpers, config, protobuf definitions, **secrets provider trait**, **PII detection** |
| **mcp-server** | Agent gateway (Bun) — MCP tools, intelligence endpoints, scenario observer (:3700) |