Fix: job tracker field name mismatch — the overnight killer

ROOT CAUSE: Python scripts polled status.get("processed", 0) but the
Rust Job struct serialized as "embedded_chunks". Scripts always saw 0,
looped forever printing "unknown: 0/50000" for 8+ hours.

Fix (both sides):
- Rust: added "processed" alias field + "total" field to Job struct,
  kept in sync on every update_progress() and complete() call
- Python: fixed autonomous_agent.py and overnight_proof.sh to read
  "embedded_chunks" as primary key

The actual embedding pipeline was working the whole time — 673K real
chunks embedded overnight. Only the monitoring was blind.

One-word bug, 8 hours of zombie output. This is why you test the
monitoring, not just the pipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-17 10:41:32 -05:00
co-authored by Claude Opus 4.6
parent 0bd48771ff
commit 1565f536eb
3 changed files with 11 additions and 2 deletions
+1 -1
View File
@@ -202,7 +202,7 @@ def phase_embed(runner: PlaybookRunner, errors: ErrorPipeline):
time.sleep(5)
status = get(f"/vectors/jobs/{job_id}")
state = status.get("status", "unknown")
progress = status.get("processed", 0)
progress = status.get("embedded_chunks", 0)
if state == "completed":
runner.record("embedding complete", True, f"{progress} chunks embedded")
break