Phase 2: DataFusion query engine over Parquet
- queryd: SessionContext with custom URL scheme to avoid path doubling with LocalFileSystem
- queryd: ListingTable registration from catalog ObjectRefs with schema inference
- queryd: POST /query/sql returns JSON {columns, rows, row_count}
- queryd→catalogd wiring: reads all datasets, registers as named tables
- gateway: wires QueryEngine with shared store + registry
- e2e verified: SELECT *, WHERE/ORDER BY, COUNT/AVG all correct
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
+6
-6
@@ -22,13 +22,13 @@
|
||||
**Gate: PASSED** — PUT object → register dataset → list → get by name. All via gateway HTTP.
|
||||
|
||||
## Phase 2: Query Engine
|
||||
- [ ] 2.1 — queryd: SessionContext + object_store config
|
||||
- [ ] 2.2 — queryd: ListingTable from catalog ObjectRefs
|
||||
- [ ] 2.3 — queryd service: POST /query → Arrow IPC or JSON
|
||||
- [ ] 2.4 — queryd → catalogd wiring
|
||||
- [ ] 2.5 — gateway routes /query
|
||||
- [x] 2.1 — queryd: SessionContext + object_store config (custom scheme to avoid path doubling)
|
||||
- [x] 2.2 — queryd: ListingTable from catalog ObjectRefs with schema inference
|
||||
- [x] 2.3 — queryd service: POST /query/sql → JSON (columns + rows + row_count)
|
||||
- [x] 2.4 — queryd → catalogd wiring (reads dataset list, registers as tables)
|
||||
- [x] 2.5 — gateway routes /query with QueryEngine state
|
||||
|
||||
**Gate:** SQL over Parquet returns correct results via catalog resolution.
|
||||
**Gate: PASSED** — SELECT *, WHERE/ORDER BY, COUNT/AVG all return correct results via catalog.
|
||||
|
||||
## Phase 3: AI Integration
|
||||
- [ ] 3.1 — Python sidecar: FastAPI + Ollama (embed/generate/rerank)
|
||||
|
||||
Reference in New Issue
Block a user