Server-side pagination for large result sets

Browse Source

- ResultStore: execute query, store batches server-side, serve pages on demand
- POST /query/paged → returns query_id + total_rows + page count (no rows)
- GET /query/page/{id}/{page}?size=100 → returns one page of rows
- RecordBatch slicing for efficient page extraction from Arrow batches
- LRU eviction: keeps 50 most recent query results in memory
- Tested: 100K rows → 1,000 pages of 100, any page fetchable by number
- Supervisor pattern: chunk results, serve on demand, retry-safe (idempotent GET)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

...

This commit is contained in:

root

2026-03-27 20:54:44 -05:00

co-authored by Claude Opus 4.6

parent ed17216005

commit 238cb84d26

18 changed files with 441 additions and 15 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

crates/ui/data/datasets/call_log.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/candidates.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/clients.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/email_log.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/job_orders.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/placements.parquet

BIN

Binary file not shown.

crates/ui/data/datasets/timesheets.parquet

BIN

Binary file not shown.