phase 1.6: lock Gate 3b = C, reconcile docs to shipped state, fix double-upload file leak

Four threads landing together — all driven by the audit J asked for before
production cutover.

(1) Gate 3b DECIDED: Option C (defer classifications). `BiometricCollection.classifications`
    stays `Option<JSON> = None` in v1. `docs/specs/GATE_3B_DEEPFACE_DESIGN.md` status
    flipped from "draft / awaits product" to DECIDED. Consent template + retention
    schedule revised to remove all "automated facial-classification" / "deepface"
    language so disclosed scope matches implemented scope.

(2) Endpoint-path drift reconciled across 3 docs. `PHASE_1_6_BIPA_GATES.md`,
    `BIPA_DESTRUCTION_RUNBOOK.md`, and `biometric_retention_schedule_v1.md` had
    references to legacy `/v1/identity/subjects/*` paths (proposed under a separate
    identityd daemon, never shipped) — corrected to actual shipped routes
    `/biometric/subject/*` (catalogd-local). Schema block in PHASE_1_6_BIPA_GATES
    rewritten to reflect JSON `SubjectManifest.biometric_collection` substrate
    (not the proposed Postgres `subjects` table).

(3) New operational artifacts:
    - `scripts/staffing/verify_biometric_erasure.sh` — checks 4 things post-erasure
      (manifest cleared, uploads dir empty, audit row matches, chain verified).
      Smoke-tested live against WORKER-2.
    - `scripts/staffing/biometric_destruction_report.sh` — monthly anonymized
      destruction-event aggregation. Smoke-tested clean.
    - `scripts/staffing/bundle_counsel_packet.sh` — tarballs the counsel-review
      packet with per-file SHA-256 manifest.
    - `docs/runbooks/LEGAL_AUDIT_KEY_ROTATION.md` — formal rotation procedure
      operationalized after the 2026-05-05 /tmp wipe incident.
    - `docs/counsel/COUNSEL_REVIEW_PACKET_2026-05-05.md` — cover note bundling
      all eng-staged BIPA docs for counsel review with per-doc questions, sign-off
      checklist, recommended review sequence.

(4) Double-upload file leak fixed in `crates/catalogd/src/biometric_endpoint.rs`.
    `verify_biometric_erasure.sh` smoked WORKER-2 and surfaced a stranded photo
    file. Investigation showed the file was 13-byte test-fixture bytes (zero PII,
    no biometric content); audit timeline showed two consecutive uploads followed
    by one erasure — the second upload had silently overwritten manifest.data_path,
    orphaning the first file. Patched `process_upload` to refuse a second upload
    with HTTP 409 + `error: "biometric_already_collected"` when
    `biometric_collection.is_some()` on the manifest. Operator must explicitly
    POST `/biometric/subject/{id}/erase` first.

    Tests: new `second_upload_without_erase_returns_409` (asserts 409 + manifest
    pointer unchanged + first file untouched on disk). Replaced
    `repeated_uploads_grow_the_chain` with `upload_erase_upload_grows_the_chain_cleanly`
    (covers the legitimate re-collection cycle: chain grows to 3 rows). Updated
    `content_type_with_parameters_accepted` to use 2 distinct subjects (was
    using 1 subject with 2 uploads to test ct parsing — would now 409).

    22/22 biometric_endpoint tests + 59/59 catalogd lib tests green post-patch.

Production posture: gateway needs `cargo build --release -p gateway` +
`systemctl restart lakehouse.service` to pick up the new 409 in live traffic.

Counsel calendar is now the only remaining blocker for first real-photo intake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-05-05 06:19:40 -05:00
co-authored by Claude Opus 4.7
parent 03e8a91d97
commit b2c34b80b3
13 changed files with 1451 additions and 65 deletions
+27 -11
View File
@@ -66,10 +66,11 @@ Before initiating destruction, the operator MUST:
Invoke the legal-tier erasure endpoint:
```bash
curl -sf -X POST "http://localhost:3100/v1/identity/subjects/${CANDIDATE_ID}/erase" \
curl -sf -X POST "http://localhost:3100/biometric/subject/${CANDIDATE_ID}/erase" \
-H "Authorization: Bearer $(cat /etc/lakehouse/legal_audit.token)" \
-H "Content-Type: application/json" \
-d '{
"scope": "biometric_only|full",
"trigger": "retention_expiry|consent_withdrawal|rtbf|court_order",
"trigger_evidence_path": "<path to signed artifact>",
"operator_of_record": "<operator name>",
@@ -77,17 +78,25 @@ curl -sf -X POST "http://localhost:3100/v1/identity/subjects/${CANDIDATE_ID}/era
}'
```
⚖ ENGINEERING — `POST /v1/identity/subjects/{id}/erase` is Phase 1.6
Gate 3 dependent. Until it ships, the manual procedure is:
The endpoint is **shipped** (commit `848a458`, 21 unit tests). It is
served from catalogd-local at `/biometric/subject/{id}/erase` (the
original v1 spec proposed `/v1/identity/subjects/{id}/erase` under a
separate identityd daemon — that daemon was collapsed into catalogd
per the architecture pivot).
a. Set `SubjectManifest.consent.biometric.status = "withdrawn"` and
`SubjectManifest.status = "erased"` via direct registry write
(operator-of-record only).
b. Securely overwrite + unlink the quarantined photo path:
`shred -uvz data/biometric/uploads/${CANDIDATE_ID}/*.jpg`
(or equivalent for the configured backend).
c. NULL the deepface classification fields on the subject row.
d. Append the destruction-event audit row (Step 2 below).
The endpoint exposes two scopes:
- **`scope: "biometric_only"`** — clears `BiometricCollection` from
the SubjectManifest (drops `data_path`, `template_hash`, and
`classifications` together) + securely unlinks the quarantined
photo file. Subject manifest itself remains. Use for retention
expiry / consent withdrawal where only biometric data must go.
- **`scope: "full"`** — full subject erasure (manifest + biometric
files). Use for court-ordered erasure or full RTBF requests.
In both scopes, the audit row is appended BEFORE photo unlink so
the chain has legal proof of intent even if the file delete fails
(transactional rollback on audit failure).
### Step 2 — Append the destruction-event audit row
@@ -224,5 +233,12 @@ training program.
## 8. Change log
- 2026-05-05 — Endpoint path reconciled with shipped state:
`/v1/identity/subjects/{id}/erase` (legacy proposal under a
separate identityd daemon) → `/biometric/subject/{id}/erase`
(catalogd-local, shipped `848a458`). Step 1 manual-fallback
block removed (the endpoint is no longer "TODO"). Two-scope
body shape (`biometric_only` / `full`) documented to match
the implementation.
- 2026-05-03 — Initial scaffold. ⚖ COUNSEL review required before
adoption.