Scheduled ingest: file watcher auto-ingests from ./inbox

- Drop CSV/JSON/PDF/text into ./inbox → auto-detected → Parquet → queryable
- Polls every 10 seconds (configurable)
- Processed files moved to ./inbox/processed/
- Failed files moved to ./inbox/failed/
- Dedup: same file dropped twice = no-op
- Watcher starts automatically on gateway boot
- Tested: CSV dropped → queryable in <15s

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This commit is contained in:

root

2026-03-27 20:04:40 -05:00

co-authored by

Claude Opus 4.6

parent 04770c97eb

commit 294f3f6a49

6 changed files with 213 additions and 0 deletions

									
										crates/ingestd/src/lib.rs
									
		+1
		
												View File
												
				@@ -5,3 +5,4 @@ pub mod pdf_ingest;

				pub mod pipeline;

				pub mod schema_evolution;

				pub mod service;

				pub mod watcher;