profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

staffing: face pool fetch preserves prior tags + --shrink gate + atomic manifest write

Browse Source 
fetch_face_pool was wiping 952 hand-classified rows when re-run from
a Python without deepface installed (it reset every gender to None).
Now:

- Loads existing manifest by id and overlays only fetch-owned fields,
  so gender/race/age/excluded survive a refetch.
- deepface pass tags only records that don't already have a gender;
  deepface unavailable means "leave existing tags alone" not "reset".
- New --shrink flag required to drop ids >= --count. Default refuses
  to shrink the pool silently.
- Atomic write via tmp + os.replace so an interrupted run can't
  corrupt the manifest.
- Dedupes duplicate id lines (root cause of the 2497-row manifest
  backing a 1000-face pool).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-28 06:01:04 -05:00
co-authored by Claude Opus 4.7
parent a05174d2fa
commit 1745881426
2 changed files with 80 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/headshots/manifest.jsonl
+2 -2
View File
@@ -175,7 +175,7 @@
{"id": 174, "file": "face_0174.jpg", "gender": "woman", "race": "east_asian", "age": 26}
{"id": 175, "file": "face_0175.jpg", "gender": "man", "race": "caucasian", "age": 30}
{"id": 176, "file": "face_0176.jpg", "gender": "man", "race": "middle_eastern", "age": 37}
{"id": 177, "file": "face_0177.jpg", "gender": "man", "race": "middle_eastern", "age": 37}
{"id": 177, "file": "face_0177.jpg", "gender": "woman", "race": "caucasian", "age": 25}
{"id": 178, "file": "face_0178.jpg", "gender": "woman", "race": "caucasian", "age": 28}
{"id": 179, "file": "face_0179.jpg", "gender": "man", "race": "hispanic", "age": 28}
{"id": 180, "file": "face_0180.jpg", "gender": "woman", "race": "caucasian", "age": 38}
@@ -588,7 +588,7 @@
{"id": 587, "file": "face_0587.jpg", "gender": "man", "race": "caucasian", "age": 34}
{"id": 588, "file": "face_0588.jpg", "gender": "man", "race": "caucasian", "age": 21, "excluded": "minor"}
{"id": 589, "file": "face_0589.jpg", "gender": "man", "race": "caucasian", "age": 21, "excluded": "minor"}
{"id": 590, "file": "face_0590.jpg", "gender": "man", "race": "caucasian", "age": 26}
{"id": 590, "file": "face_0590.jpg", "gender": "woman", "race": "hispanic", "age": 31}
{"id": 591, "file": "face_0591.jpg", "gender": "woman", "race": "hispanic", "age": 31}
{"id": 592, "file": "face_0592.jpg", "gender": "woman", "race": "caucasian", "age": 28}
{"id": 593, "file": "face_0593.jpg", "gender": "woman", "race": "caucasian", "age": 28}