phase 1.6: lock Gate 3b = C, reconcile docs to shipped state, fix double-upload file leak

Four threads landing together — all driven by the audit J asked for before
production cutover.

(1) Gate 3b DECIDED: Option C (defer classifications). `BiometricCollection.classifications`
    stays `Option<JSON> = None` in v1. `docs/specs/GATE_3B_DEEPFACE_DESIGN.md` status
    flipped from "draft / awaits product" to DECIDED. Consent template + retention
    schedule revised to remove all "automated facial-classification" / "deepface"
    language so disclosed scope matches implemented scope.

(2) Endpoint-path drift reconciled across 3 docs. `PHASE_1_6_BIPA_GATES.md`,
    `BIPA_DESTRUCTION_RUNBOOK.md`, and `biometric_retention_schedule_v1.md` had
    references to legacy `/v1/identity/subjects/*` paths (proposed under a separate
    identityd daemon, never shipped) — corrected to actual shipped routes
    `/biometric/subject/*` (catalogd-local). Schema block in PHASE_1_6_BIPA_GATES
    rewritten to reflect JSON `SubjectManifest.biometric_collection` substrate
    (not the proposed Postgres `subjects` table).

(3) New operational artifacts:
    - `scripts/staffing/verify_biometric_erasure.sh` — checks 4 things post-erasure
      (manifest cleared, uploads dir empty, audit row matches, chain verified).
      Smoke-tested live against WORKER-2.
    - `scripts/staffing/biometric_destruction_report.sh` — monthly anonymized
      destruction-event aggregation. Smoke-tested clean.
    - `scripts/staffing/bundle_counsel_packet.sh` — tarballs the counsel-review
      packet with per-file SHA-256 manifest.
    - `docs/runbooks/LEGAL_AUDIT_KEY_ROTATION.md` — formal rotation procedure
      operationalized after the 2026-05-05 /tmp wipe incident.
    - `docs/counsel/COUNSEL_REVIEW_PACKET_2026-05-05.md` — cover note bundling
      all eng-staged BIPA docs for counsel review with per-doc questions, sign-off
      checklist, recommended review sequence.

(4) Double-upload file leak fixed in `crates/catalogd/src/biometric_endpoint.rs`.
    `verify_biometric_erasure.sh` smoked WORKER-2 and surfaced a stranded photo
    file. Investigation showed the file was 13-byte test-fixture bytes (zero PII,
    no biometric content); audit timeline showed two consecutive uploads followed
    by one erasure — the second upload had silently overwritten manifest.data_path,
    orphaning the first file. Patched `process_upload` to refuse a second upload
    with HTTP 409 + `error: "biometric_already_collected"` when
    `biometric_collection.is_some()` on the manifest. Operator must explicitly
    POST `/biometric/subject/{id}/erase` first.

    Tests: new `second_upload_without_erase_returns_409` (asserts 409 + manifest
    pointer unchanged + first file untouched on disk). Replaced
    `repeated_uploads_grow_the_chain` with `upload_erase_upload_grows_the_chain_cleanly`
    (covers the legitimate re-collection cycle: chain grows to 3 rows). Updated
    `content_type_with_parameters_accepted` to use 2 distinct subjects (was
    using 1 subject with 2 uploads to test ct parsing — would now 409).

    22/22 biometric_endpoint tests + 59/59 catalogd lib tests green post-patch.

Production posture: gateway needs `cargo build --release -p gateway` +
`systemctl restart lakehouse.service` to pick up the new 409 in live traffic.

Counsel calendar is now the only remaining blocker for first real-photo intake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-05-05 06:19:40 -05:00
co-authored by Claude Opus 4.7
parent 03e8a91d97
commit b2c34b80b3
13 changed files with 1451 additions and 65 deletions
@@ -3,6 +3,7 @@
**Spec:** docs/PHASE_1_6_BIPA_GATES.md §1 Gate 2 (BIPA §15(b)(1)-(3))
**Status:** Engineering scaffold — ⚖ COUNSEL must author the binding text before deployment
**Version:** v1 (initial; supersession requires a new version + new hash)
**Updated 2026-05-05:** Disclosure 1 + plain-language summary revised to match the Gate 3b deferral recommendation in `docs/specs/GATE_3B_DEEPFACE_DESIGN.md` (Option C — defer classifications). Pending J's product confirmation of Gate 3b; if Gate 3b chooses Option A or B, this template needs counsel re-authoring.
> This is the consent template a candidate signs (electronically or
> on paper) BEFORE Lakehouse collects, stores, or processes any
@@ -25,11 +26,12 @@ content; counsel provides the legally-sufficient wording.
### Disclosure 1 — Notice of collection (§15(b)(1))
Lakehouse will collect, store, and use my **biometric identifier**
(facial geometry derived from a photograph of me) and **biometric
information** (gender, race, and age classifications derived from
that photograph by an automated facial-classification model called
deepface).
Lakehouse will collect and store my **biometric identifier** (a
photograph of me from which facial geometry is implicit). The
photograph itself is the data we keep — we do NOT run automated
facial-classification (gender / race / age inference) against it
in v1. If at a later date we add automated classification, we will
re-collect consent under a superseding template before doing so.
### Disclosure 2 — Specific purpose and length of term (§15(b)(2))
@@ -66,9 +68,9 @@ is appropriate to include or whether a different plain-language
section is preferred.
> **What you're agreeing to:** if you upload a photo of yourself,
> we'll keep that photo and a few descriptive labels about the photo
> (estimated age, perceived gender, perceived race) to help your
> staffing coordinator recognize you when you arrive at job sites.
> we'll keep that photo so your staffing coordinator can recognize
> you when you arrive at job sites. We don't run automated guesses
> about your age, gender, or race against the photo.
>
> **How long we keep it:** at most 18 months after your last
> placement or interaction with us, then it's permanently destroyed.
@@ -3,6 +3,7 @@
**Spec:** docs/PHASE_1_6_BIPA_GATES.md §1 Gate 1 (BIPA §15(a))
**Status:** Engineering scaffold — ⚖ COUNSEL must author the binding text before public publication
**Version:** v1 (initial; supersession requires a new version + new hash)
**Updated 2026-05-05:** §1 + §2 revised to match the Gate 3b deferral recommendation in `docs/specs/GATE_3B_DEEPFACE_DESIGN.md` (Option C — defer classifications). §5 destruction-trigger endpoint corrected to the shipped catalogd-local route. Pending J's product confirmation of Gate 3b.
> This is a publicly-available retention schedule for biometric identifiers
> and biometric information collected by the Lakehouse staffing platform.
@@ -15,12 +16,15 @@
This schedule applies to:
- **Biometric identifiers** as defined in 740 ILCS 14/10: facial geometry
derived from candidate photographs.
- **Biometric identifiers** as defined in 740 ILCS 14/10: candidate
photographs from which facial geometry is implicit.
- **Biometric information** as defined in 740 ILCS 14/10: any information
derived from a biometric identifier, including but not limited to
the gender, race, and age classifications produced by the deepface
model when applied to a candidate photograph.
derived from a biometric identifier. **In v1 of this schedule, no
derived information is collected** — automated facial-classification
(gender, race, age inference) is deferred per
`docs/specs/GATE_3B_DEEPFACE_DESIGN.md` Option C. If a future version
of this schedule introduces classification, that is a superseding
v2 schedule with re-consent under the matching v2 consent template.
**Out of scope** (explicitly NOT biometric data under this schedule):
@@ -39,14 +43,15 @@ This schedule applies to:
| Category | Source | Storage location |
|---|---|---|
| Photograph (raw bytes) | Candidate upload via the consent-gated photo endpoint | Quarantined under `data/biometric/uploads/<candidate_id>/<ts>.<ext>`; encrypted at rest |
| Facial geometry classifications | deepface inference run against the photograph | `subjects.biometric_classifications` (JSONB on the identityd `subjects` row) |
| Photograph integrity hash | SHA-256 of the original bytes | `subjects.biometric_template_hash` |
| Photograph (raw bytes) | Candidate upload via the consent-gated photo endpoint | Quarantined under `data/biometric/uploads/<candidate_id>/<ts>_<uuid>.<ext>`; mode 0700 dir / 0600 file |
| Photograph integrity hash | SHA-256 of the original bytes | `SubjectManifest.biometric_collection.template_hash` (catalogd JSON manifest at `data/_catalog/subjects/<id>.json`) |
We do NOT collect raw biometric template vectors that could be used
to re-derive a face from the encoded form. The deepface output is
stored as discrete classification labels (e.g. `{"age_estimate": 32,
"gender": "...", "race": "..."}`), not as a re-identifiable embedding.
to re-derive a face from the encoded form. We do NOT run automated
facial-classification (gender, race, age inference) in v1 — see
`docs/specs/GATE_3B_DEEPFACE_DESIGN.md` for the deferral rationale.
The `BiometricCollection.classifications` field on the subject
manifest exists in the schema but is `None` for every subject.
---
@@ -104,8 +109,8 @@ Runbook** (`docs/runbooks/BIPA_DESTRUCTION_RUNBOOK.md`) when:
- Retention period under §4 expires
- Candidate withdraws biometric consent under the consent template (Gate 2)
- Candidate exercises a right-to-be-forgotten request
- An identityd `POST /v1/identity/subjects/{id}/erase` is invoked under
legal-tier authentication
- A catalogd-local `POST /biometric/subject/{id}/erase` is invoked
under legal-tier authentication (shipped `848a458`)
Every destruction event is recorded as an append-only audit row in
the affected subject's per-subject HMAC-chained audit log (see