profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Update PRD + PHASES.md — reflect 8-commit 2026-04-17 push

Browse Source 
PRD status line: "Phases 0-18 shipped; hybrid operational; scheduled
ingest live; PDF OCR live; entering horizon items."

PHASES.md: federation L2 items marked complete, Phase 16.2 (autotune
agent), Phase 17 VRAM gate, MySQL connector, Phase 18 (hybrid Lance),
scheduled ingest, PDF OCR all documented with dates and measurements.

Stats updated: 52+ unit tests, 13 crates, 19 ADRs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-16 20:54:05 -05:00
co-authored by Claude Opus 4.6
parent fd4b6836ae
commit 3bc82833ac
2 changed files with 48 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PRD.md
+6 -5
View File
@@ -1,8 +1,8 @@
# PRD: Lakehouse — Rust-First Substrate for Versioned Knowledge Stores
**Status:** Active — Phase 0-14 complete; federation foundation + HNSW trial system shipped 2026-04-16; entering Phase 16 (hot-swap + model profiles)
**Status:** Active — Phases 0-18 shipped; hybrid Parquet+HNSW ⊕ Lance operational; scheduled ingest live; PDF OCR live; entering horizon items
**Created:** 2026-03-27
**Last reframed:** 2026-04-16 — from "staffing analytics platform" to "dual-use knowledge substrate" (see §Problem below)
**Last reframed:** 2026-04-17 — hybrid architecture proven end-to-end on 100K vectors (see §Phase 18 + ADR-019)
**Owner:** J
---
@@ -297,9 +297,10 @@ Per-contract overlays with daily/weekly/monthly tiers and instant handoff.
- [x] HNSW vector index with trial system (shipped 2026-04-16)
- [x] Federation foundation — ADR-017 (shipped 2026-04-16)
- [x] Database connector ingest — Postgres batch with streaming (shipped 2026-04-16)
- [ ] Federation layer 2 — X-Lakehouse-Bucket middleware, catalog migration, cross-bucket SQL in queryd
- [ ] PDF OCR for scanned documents (Tesseract integration)
- [ ] Scheduled ingest (cron-based file watching, S3 event triggers)
- [x] Federation layer 2 — runtime bucket lifecycle, per-index bucket scoping, profile bucket auto-provisioning (shipped 2026-04-17)
- [x] MySQL streaming connector — mirrors Postgres path, verified on live MariaDB (shipped 2026-04-17)
- [x] PDF OCR for scanned documents — Tesseract 5.5 fallback when lopdf yields no text (shipped 2026-04-17)
- [x] Scheduled ingest — interval-based per-source schedules with CRUD + run-now + auto-trigger agent (shipped 2026-04-17)
- [ ] Multi-node query distribution (DataFusion supports this architecturally)
### Phase 16: Hot-Swap Index Generations