8dd0d46c3c5d05d93a51ad6aded3dbeff8a9aacd
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rootandClaude Opus 4.7
|
f1fa6e4e61 |
phase 1.6 Gate 3a: photo upload endpoint with consent gate
Per docs/PHASE_1_6_BIPA_GATES.md §1 Gate 3 (consent-gate substrate).
Deepface classification (Gate 3b) deferred to its own session — needs
Python subprocess design conversation after the 2026-05-02 sidecar drop.
What ships:
shared/types.rs:
- new BiometricCollection sub-struct: data_path, template_hash,
collected_at, consent_version_hash, classifications (Option<JSON>)
- SubjectManifest gains biometric_collection: Option<BiometricCollection>
with #[serde(default)] so existing on-disk manifests parse and
re-emit without drift
catalogd/biometric_endpoint.rs (NEW, ~600 LOC):
POST /subject/{candidate_id}/photo
- Auth: X-Lakehouse-Legal-Token, constant-time-eq compared against
same legal token file as /audit. Same 32-byte minimum.
- Content-Type: must be image/jpeg or image/png (415 otherwise)
- Body: raw image bytes, max 10MB
- 401: missing or wrong token
- 404: subject not registered
- 403: consent.biometric.status != "given" (returns current status)
- 403: subject status in {Withdrawn, Erased, RetentionExpired}
- 200: writes photo to data/biometric/uploads/<sanitized_id>/<ts>.<ext>
with mode 0700 dir + 0600 file, updates SubjectManifest with
BiometricCollection record, appends audit row
(kind="biometric_collection", purpose="photo_upload"), returns
UploadResponse with template_hash + audit_row_hmac.
Logic split: pure async fn process_upload() takes the headers-as-args
so unit tests exercise every branch without HTTP machinery; the
axum handler is just glue. 10 tests covering all 4 reject paths +
happy path + repeated uploads chaining + structural assertion that
the quarantine path is NOT under data/headshots/ (synthetic faces).
gateway/main.rs:
Mounts /biometric on the same condition as /audit — only when the
SubjectAuditWriter is present AND the legal token loads. Storage
root configurable via LH_BIOMETRIC_STORAGE_ROOT (default
./data/biometric/uploads).
Live verification on the running gateway (post-restart):
- GET /biometric/health → "biometric endpoint ready"
- POST without token → 401 auth_failed
- POST with token, no consent → 403 consent_required (status=NeverCollected)
- Flipped WORKER-2 to consent=given, POST → 200 with hash + path
- File at data/biometric/uploads/WORKER-2/<ts>.jpg, mode 0600
- Manifest biometric_collection field reflects the upload
- Audit row chain links cleanly off the prior validator_lookup row
- GET /audit/subject/WORKER-2 returns chain_verified=true, 2 rows
- Cross-runtime parity probe still 6/6 byte-identical post-change
Phase 1.6 status table updated: Gate 3a DONE, Gate 3b (deepface)
deferred. Calendar bottleneck remains counsel review of items 1/2/5/6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
2a4b316a15 |
subjects: 2nd scrum fix wave (token min, chain_tip, tampering, rebuild collision warn)
Second cross-lineage scrum on Steps 5+6 returned 13 distinct findings, 0 convergent.
Three BLOCK-class claims verified as false positives (cache IS written, per-subject
Mutex IS in place, spawn IS safe under writer's lock). Five real fixes shipped:
1. audit_endpoint: legal token min length 16->32 (HMAC-SHA256 best practice, kimi)
2. subject_audit: new chain_tip() returns last hash from full log; audit_endpoint
now reports chain_root from full chain instead of windowed slice (opus)
3. registry: rebuild loader now warns on sanitize collision (symmetric with
put_subject's collision guard - opus)
4. audit_endpoint: tampering detection - if manifest expects non-empty chain_root
but log returns 0 rows, flag chain_verified=false with explicit message (opus)
5. execution_loop::audit_result_state: tightened heuristic - error/denied/not_found
only classify when no rows/data/results sibling (opus INFO)
Tests: 17 catalogd subject + 6 gateway audit_result_state, all green.
New: audit_result_state_does_not_classify_error_when_data_sibling_present,
audit_result_state_status_is_authoritative_even_with_data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
15cfd76c04 |
catalogd + gateway: Step 6 — /audit/subject/{id} legal-tier HTTP endpoint
Implementation of docs/specs/SUBJECT_MANIFESTS_ON_CATALOGD.md §5 Step 6
+ §4 (response shape) + §6 (auth model). The defense-against-EEOC-
discovery surface is live: legal counsel hits one URL with one token,
gets back a signed-by-HMAC-chain audit response naming every PII access
for a subject in a time window.
New module: crates/catalogd/src/audit_endpoint.rs (~340 LOC)
- AuditEndpointState { registry, writer, legal_token }
- router() exposes:
GET /subject/{candidate_id}?from=ISO&to=ISO (full audit response)
GET /health (liveness + token check)
- require_legal_auth() — constant-time-eq compare against the
X-Lakehouse-Legal-Token header. Avoids timing leaks on the token
check without pulling in `subtle` for one comparison.
- Token loaded from /etc/lakehouse/legal_audit.token (env-overridable
via LH_LEGAL_AUDIT_TOKEN_FILE). Empty file or <16 chars = endpoint
serves 503 with a clear reason. Token value NEVER logged.
- Response schema: subject_audit_response.v1 with manifest +
audit_log (rows + chain verification) + datasets_referenced +
safe_views_available + completeness_attestation.
New helper on SubjectAuditWriter:
- read_rows_in_range(candidate_id, from, to) — returns rows in window,
used by the endpoint to assemble the response without re-reading
the entire chain.
- verify_chain() now returns Ok(0) when the audit log file doesn't
exist (empty = trivially valid). Prevents legitimate "no PII access
yet for this subject" from showing as integrity=BROKEN in the
audit response. Caller can detect "log was deleted" via comparison
to SubjectManifest.audit_log_chain_root (when that mirror lands).
main.rs:
- Audit endpoint mounted at /audit ONLY when both subject_audit
writer AND legal token are present. Disabled-by-default keeps the
surface from accidentally serving in dev/bring-up environments
without proper credentials.
Tests (9/9 passing):
- constant_time_eq (correctness on equal/diff/empty/length-mismatch)
- missing_legal_token_returns_503
- missing_header_returns_401
- wrong_token_returns_401
- correct_token_passes_auth
- audit_response_assembly_full_path (manifest + 3 rows + chain verify)
- audit_response_window_filters_rows (time-bounded window)
- empty_token_file_results_in_disabled_endpoint
- short_token_file_rejected_at_load (<16 char min)
LIVE end-to-end verification:
1. Plant signing key + legal token in /tmp/lakehouse_audit/
2. Restart gateway with LH_SUBJECT_AUDIT_KEY + LH_LEGAL_AUDIT_TOKEN_FILE
pointing at the test files
3. /audit/health → 200 "audit endpoint ready"
4. /audit/subject/WORKER-1 (no token) → 401 "missing X-Lakehouse-Legal-Token"
5. /audit/subject/WORKER-1 (wrong token) → 401 "X-Lakehouse-Legal-Token mismatch"
6. /audit/subject/WORKER-1 (correct token) → 200 + full manifest + 0 rows
+ chain_verified=true (empty log path)
7. POST /v1/validate with candidate_id=WORKER-1 → triggers WorkerLookup.find()
via the AuditingWorkerLookup wrapper from Step 5
8. data/_catalog/subjects/WORKER-1.audit.jsonl now exists with 1 row
(accessor.purpose=validator_worker_lookup, result=not_found,
prev_chain_hash=GENESIS, valid HMAC)
9. /audit/subject/WORKER-1 (correct token) → 200 + manifest + 1 row +
chain_verified=true + chain_rows_total=1 + completeness attestation
The full audit-trail loop (PII access → audit row → chain → audit response)
works end-to-end on the live gateway.
NOT in this commit (future steps):
- Step 7: Daily retention sweep
- Step 8: Cross-runtime parity (Go side reads the same shapes)
- Mirror chain root to SubjectManifest.audit_log_chain_root after
each append (so tampering detection can use the manifest's
cached root as ground truth)
- Live row projection from datasets (currently caller follows up
via /query/sql against the safe_views named in the response)
- Ed25519 signature on the response (chain verification IS the v1
attestation; signing is future hardening per spec §10)
cargo build --release clean. cargo test -p catalogd audit_endpoint
9/9 PASS. Live verification successful.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|