Phase 14: Schema evolution with AI migration rules

- Schema diff detection: compare old vs new schema, identify changes
  (added, removed, type changed, renamed columns)
- Fuzzy rename detection: "first_name" → "full_name" detected by shared word parts
- Auto-generated migration rules: direct map, cast, concat, split, drop
  Each rule has confidence score (0.0-1.0)
- AI migration prompt builder: generates LLM prompt for complex schema changes
  LLM suggests JSON migration rules when heuristics aren't enough
- 5 new unit tests (detect added, removed, type change, rename, rule generation)
- 30 total unit tests passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This commit is contained in:

root

2026-03-27 19:31:19 -05:00

co-authored by

Claude Opus 4.6

parent d61096e26f

commit 35f0559d78

2 changed files with 280 additions and 0 deletions

									
										crates/ingestd/src/lib.rs
									
		+1
		
												View File
												
				@@ -3,4 +3,5 @@ pub mod csv_ingest;

				pub mod json_ingest;

				pub mod pdf_ingest;

				pub mod pipeline;

				pub mod schema_evolution;

				pub mod service;