Fix: job tracker field name mismatch — the overnight killer
ROOT CAUSE: Python scripts polled status.get("processed", 0) but the
Rust Job struct serialized as "embedded_chunks". Scripts always saw 0,
looped forever printing "unknown: 0/50000" for 8+ hours.
Fix (both sides):
- Rust: added "processed" alias field + "total" field to Job struct,
kept in sync on every update_progress() and complete() call
- Python: fixed autonomous_agent.py and overnight_proof.sh to read
"embedded_chunks" as primary key
The actual embedding pipeline was working the whole time — 673K real
chunks embedded overnight. Only the monitoring was blind.
One-word bug, 8 hours of zombie output. This is why you test the
monitoring, not just the pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -202,7 +202,7 @@ def phase_embed(runner: PlaybookRunner, errors: ErrorPipeline):
|
||||
time.sleep(5)
|
||||
status = get(f"/vectors/jobs/{job_id}")
|
||||
state = status.get("status", "unknown")
|
||||
progress = status.get("processed", 0)
|
||||
progress = status.get("embedded_chunks", 0)
|
||||
if state == "completed":
|
||||
runner.record("embedding complete", True, f"{progress} chunks embedded")
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user