phase 1.6: BIPA withdrawal endpoint + UI + retention sweep timer

Closes the four production gaps that were live after the consent endpoint shipped (76cb5ac): (1) Withdrawal endpoint POST /biometric/subject/{id}/withdraw backs the BIPA right of withdrawal that consent template v1 §2 explicitly promises. Without it, the only way to honor a candidate's withdrawal request was the heavier /erase, which destroys immediately rather than starting the 30-day SLA clock that the consent template commits to. Side-effects: - manifest.consent.biometric.status: Given → Withdrawn - manifest.consent.biometric.retention_until: 18mo → 30d - audit row kind=biometric_consent_withdrawal, captures reason + operator_of_record + evidence_path - DOES NOT touch general_pii or subject.status — biometric is independently revocable State machine: Given→Withdrawn (happy), NeverCollected/Pending→ 409 nothing_to_withdraw, Withdrawn→409 already_withdrawn (won't advance the destruction clock), Expired→409 already_expired, subject Erased/RetentionExpired→403 subject_inactive. 12 new unit tests covering happy path + all guards + a full grant→withdraw cycle that asserts retention_until is correctly accelerated and the audit chain has 2 rows in correct order. (2) Withdraw UI at /biometric/withdraw (mcp-server-served HTML). 3-screen flow: operator auth (token + name in sessionStorage), withdrawal form (candidate_id + free-text reason ≥10 chars + optional evidence path), confirmation showing the audit row HMAC + the 30-day retention_until clock + a curl recipe for /audit/subject/{id} verification. Same neo-brutalist styling as biometric_intake.html. Mounted at http://localhost:3700/biometric/withdraw and externally at https://devop.live/lakehouse/biometric/withdraw. (3) Retention sweep systemd timer. crates/catalogd/bin/retention_sweep binary already existed; this commit schedules it. Daily 03:00 UTC, Persistent=true so a missed boot triggers on next start. Service runs as oneshot with --apply (writes a date-stamped JSONL to data/_catalog/subjects/_retention_sweep_<date>.jsonl ONLY when overdue subjects exist, per the binary's existing semantics). install.sh updated to handle .timer + paired .service correctly: enables the timer, skips direct start of the oneshot service (the timer pulls it in). One-shot manual test confirmed clean: 100 subjects scanned, 0 overdue (all backfill subjects within their 4-year general retention window). (4) operator_of_record bug fix in intake UI. Previously the page hardcoded the literal string 'intake_ui_operator' as the operator_of_record sent to /consent — meaning every audit row captured the same useless placeholder, defeating the whole point of operator traceability. Fixed by adding an operator name field to the token-paste step (sessionStorage-backed), passed through to consent + photo POSTs as the actual operator. Verified live post-restart: - gateway /audit/health + /biometric/health both 200 - mcp-server /biometric/intake + /biometric/withdraw both 200 - Live withdraw probes: 401 (no token), 400 (empty body), 404 (ghost subject), 409 nothing_to_withdraw on WORKER-1 (which is NeverCollected per backfill default) — all expected - Binary strings contain: process_withdraw, withdraw_consent, biometric_consent_withdrawal, biometric_withdraw_response.v1, nothing_to_withdraw, already_withdrawn, already_expired, /subject/{candidate_id}/withdraw route - systemd: lakehouse-retention-sweep.timer active+enabled, next fire Tue 2026-05-05 22:00 CDT (= 03:00 UTC May 6) - Manual one-shot of retention sweep service: exit 0/SUCCESS, 100 subjects loaded, 0 overdue 83/83 catalogd lib tests + 46/46 biometric_endpoint tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 15:09:32 -05:00
parent 7f0f500050
commit 68d226c314
7 changed files with 844 additions and 8 deletions
@@ -87,10 +87,12 @@ canvas{display:none}
  <!-- Step 1: operator auth -->
  <div id="screen-token" class="screen active">
    <h2>Operator authentication</h2>
-    <p class="lede">Paste the legal-tier audit token. Stored in this tab's session only; cleared on close. Never persists to disk.</p>
+    <p class="lede">Paste the legal-tier audit token + your name. Both stored in this tab's session only; cleared on close. Never persists to disk. Your name is recorded as <code>operator_of_record</code> in the audit row for legal traceability.</p>
    <div class="card">
      <label for="token-input">Legal audit token</label>
      <input type="password" id="token-input" placeholder="44-char token from /etc/lakehouse/legal_audit.token" autocomplete="off">
+      <label for="op-name" style="margin-top:14px">Your name (operator of record)</label>
+      <input type="text" id="op-name" placeholder="First Last" autocomplete="off">
      <div style="margin-top:14px">
        <button id="token-submit">Continue →</button>
      </div>
@@ -241,6 +243,7 @@ const CANDIDATE_ID = (function(){
 // ── State ─────────────────────────────────────────────────────────
 const state = {
  token: null,                 // sessionStorage-backed
+  operator: null,              // operator_of_record — captured at token paste
  consentVersionHash: null,    // SHA-256 of the rendered consent template
  consentResp: null,           // server response from /consent
  photoBlob: null,             // captured/selected photo
@@ -277,12 +280,17 @@ async function sha256Hex(text) {
  }
  document.getElementById('cid-display').textContent = CANDIDATE_ID;

-  // Restore token from sessionStorage if previously set in this tab.
+  // Restore token + operator from sessionStorage if previously set in this tab.
  const saved = sessionStorage.getItem('lh_legal_token');
+  const savedOp = sessionStorage.getItem('lh_operator_name');
  if (saved) {
    state.token = saved;
    document.getElementById('token-input').value = '••••••••';
  }
+  if (savedOp) {
+    state.operator = savedOp;
+    document.getElementById('op-name').value = savedOp;
+  }

  // Compute consent template hash (SHA-256 of the rendered consent block).
  // This is what we'll send as consent_version_hash. Counsel later
@@ -293,18 +301,25 @@ async function sha256Hex(text) {
  document.getElementById('version-hash-display').textContent = state.consentVersionHash.substring(0,16) + '…';
 })();

-// ── Step 1: token ──────────────────────────────────────────────────
+// ── Step 1: token + operator ───────────────────────────────────────
 document.getElementById('token-submit').addEventListener('click', () => {
  clearErr('token-error');
  const v = document.getElementById('token-input').value.trim();
  // If user kept the masked placeholder, use the saved token.
  const tok = v === '••••••••' ? state.token : v;
+  const op = document.getElementById('op-name').value.trim();
  if (!tok || tok.length < 32) {
    err('token-error', 'Token must be ≥32 characters.');
    return;
  }
+  if (!op || op.length < 2) {
+    err('token-error', 'Operator name is required (recorded as operator_of_record in audit row).');
+    return;
+  }
  state.token = tok;
+  state.operator = op;
  sessionStorage.setItem('lh_legal_token', tok);
+  sessionStorage.setItem('lh_operator_name', op);
  show('screen-consent');
 });

@@ -336,7 +351,7 @@ document.getElementById('grant-btn').addEventListener('click', async () => {
      consent_version_hash: state.consentVersionHash,
      consent_collection_method: 'click_acceptance',
      consent_collection_evidence_path: 'inline:sha256=' + evidenceHash,
-      operator_of_record: 'intake_ui_operator',
+      operator_of_record: state.operator,
    };
    const r = await fetch(`${GATEWAY}/biometric/subject/${encodeURIComponent(CANDIDATE_ID)}/consent`, {
      method: 'POST',
@@ -0,0 +1,253 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width,initial-scale=1">
+<title>Lakehouse — Biometric Consent Withdrawal</title>
+<style>
+*{margin:0;padding:0;box-sizing:border-box}
+body{font-family:'Inter',-apple-system,system-ui,sans-serif;background:#090c10;color:#b0b8c4;font-size:14px;line-height:1.55;-webkit-font-smoothing:antialiased}
+a{color:#58a6ff;text-decoration:none}
+a:hover{color:#79c0ff}
+
+.bar{background:#0d1117;padding:0 24px;height:56px;border-bottom:1px solid #171d27;display:flex;justify-content:space-between;align-items:center;position:sticky;top:0;z-index:10}
+.bar h1{font-size:14px;font-weight:600;color:#e6edf3;letter-spacing:-0.2px}
+.bar .step{font-size:11px;color:#7d8590;text-transform:uppercase;letter-spacing:1px}
+
+.wrap{max-width:680px;margin:0 auto;padding:28px 20px 60px}
+.screen{display:none}
+.screen.active{display:block}
+
+h2{font-size:20px;color:#e6edf3;font-weight:600;margin-bottom:8px;letter-spacing:-0.3px}
+h3{font-size:14px;color:#e6edf3;font-weight:600;margin:16px 0 8px}
+.lede{color:#7d8590;font-size:13px;margin-bottom:24px}
+
+.card{background:#0d1117;border:1px solid #171d27;padding:20px;margin-bottom:16px}
+.card.warn{border-left:3px solid #d29922}
+.card.bad{border-left:3px solid #f85149}
+.card.good{border-left:3px solid #3fb950}
+
+label{display:block;margin-bottom:12px;color:#7d8590;font-size:12px;text-transform:uppercase;letter-spacing:0.5px}
+input[type=text],input[type=password],textarea{width:100%;background:#0d1117;border:1px solid #2d333b;color:#e6edf3;padding:10px 12px;font-family:inherit;font-size:14px;border-radius:0;margin-top:6px;transition:border-color .15s;resize:vertical}
+input[type=text]:focus,input[type=password]:focus,textarea:focus{outline:none;border-color:#58a6ff}
+textarea{font-family:inherit;line-height:1.5;min-height:80px}
+
+button{background:#1f6feb;color:#fff;border:none;padding:10px 20px;font-family:inherit;font-size:14px;font-weight:500;cursor:pointer;border-radius:0;transition:background .15s;margin-right:8px}
+button:hover:not(:disabled){background:#388bfd}
+button:disabled{background:#21262d;color:#545d68;cursor:not-allowed}
+button.secondary{background:#21262d;color:#c9d1d9}
+button.secondary:hover:not(:disabled){background:#30363d}
+button.danger{background:#da3633}
+button.danger:hover:not(:disabled){background:#f85149}
+
+.notice{background:#0a0c10;border:1px solid #1f242c;padding:14px 18px;margin-bottom:16px;font-size:13px;color:#b0b8c4;line-height:1.7}
+.notice strong{color:#e6edf3}
+
+.kv{display:grid;grid-template-columns:140px 1fr;gap:8px;margin:6px 0;font-size:13px}
+.kv .k{color:#7d8590;text-transform:uppercase;font-size:11px;letter-spacing:0.5px;padding-top:2px}
+.kv .v{color:#e6edf3;font-family:ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;font-size:12px}
+
+.hash{font-family:ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;font-size:11px;color:#7ee787;word-break:break-all;background:#0a0c10;padding:8px 12px;border:1px solid #1f242c;margin:6px 0}
+
+.error{background:#3a0f0f;border:1px solid #f85149;color:#ffa198;padding:12px 16px;margin:12px 0;font-size:13px;display:none}
+.error.show{display:block}
+.error code{color:#fff}
+
+.foot{margin-top:32px;padding-top:16px;border-top:1px solid #171d27;font-size:11px;color:#545d68;text-transform:uppercase;letter-spacing:0.5px}
+.foot a{color:#58a6ff}
+</style>
+</head>
+<body>
+
+<div class="bar">
+  <h1>⚡ Biometric Consent Withdrawal</h1>
+  <span class="step" id="step-label">step 1 of 3</span>
+</div>
+
+<div class="wrap">
+
+  <!-- Step 1: operator auth -->
+  <div id="screen-token" class="screen active">
+    <h2>Operator authentication</h2>
+    <p class="lede">Withdrawal is operator-recorded on behalf of the candidate. Paste the legal-tier audit token + your name.</p>
+    <div class="card">
+      <label for="token-input">Legal audit token</label>
+      <input type="password" id="token-input" placeholder="44-char token from /etc/lakehouse/legal_audit.token" autocomplete="off">
+      <label for="op-name" style="margin-top:14px">Your name (operator of record)</label>
+      <input type="text" id="op-name" placeholder="First Last" autocomplete="off">
+      <div style="margin-top:14px">
+        <button id="token-submit">Continue →</button>
+      </div>
+    </div>
+    <div class="error" id="token-error"></div>
+  </div>
+
+  <!-- Step 2: withdrawal form -->
+  <div id="screen-form" class="screen">
+    <h2>Record withdrawal</h2>
+    <p class="lede">The candidate has requested withdrawal of biometric consent. This action sets a <strong>30-day SLA clock</strong> for destruction (per consent template v1 §2). The retention sweep + erase runbook handle actual destruction; this endpoint records intent + starts the clock.</p>
+
+    <div class="notice warn">
+      <strong>What withdrawal does:</strong> sets <code>consent.biometric.status = Withdrawn</code>, accelerates <code>retention_until</code> from the 18-month default to <strong>30 days from now</strong>. Future photo uploads will be refused (403). General-PII consent is NOT touched — the candidate can keep their non-biometric data on the platform.
+    </div>
+
+    <div class="card">
+      <label for="cand-id">Candidate ID</label>
+      <input type="text" id="cand-id" placeholder="WORKER-XXX" autocomplete="off">
+
+      <label for="reason" style="margin-top:14px">Reason for withdrawal</label>
+      <textarea id="reason" placeholder="Free-text. Captured to the audit row for legal traceability. e.g., 'candidate emailed 2026-05-05 asking to remove their photo, no specific reason given'"></textarea>
+
+      <label for="evidence" style="margin-top:14px">Evidence path (optional)</label>
+      <input type="text" id="evidence" placeholder="/path/to/email_thread.eml or scanned signed paper" autocomplete="off">
+
+      <div style="margin-top:16px">
+        <button id="submit-btn" class="danger" disabled>Record withdrawal</button>
+        <button id="back-btn" class="secondary">Back</button>
+      </div>
+    </div>
+    <div class="error" id="form-error"></div>
+  </div>
+
+  <!-- Step 3: confirmation -->
+  <div id="screen-done" class="screen">
+    <div class="card good">
+      <h2>✓ Withdrawal recorded</h2>
+      <p class="lede" style="margin:8px 0 0">Audit chain row appended; retention sweep will pick it up at the SLA.</p>
+    </div>
+
+    <div class="card">
+      <h3>State change</h3>
+      <div class="kv">
+        <span class="k">Candidate</span><span class="v" id="done-cid">…</span>
+        <span class="k">Status</span><span class="v" id="done-status">Withdrawn</span>
+        <span class="k">Withdrawn at</span><span class="v" id="done-withdrawn-at">…</span>
+        <span class="k">Retention until</span><span class="v" id="done-retention">…</span>
+      </div>
+      <h3>Audit row hmac</h3>
+      <div class="hash" id="done-hmac">…</div>
+    </div>
+
+    <div class="card warn">
+      <h3>What happens next</h3>
+      <p>The retention sweep flags this subject as overdue once <code>retention_until</code> passes. An operator with legal-tier credentials runs the destruction runbook (<code>POST /biometric/subject/&lt;id&gt;/erase</code>) within the 30-day SLA.</p>
+      <p style="margin-top:8px">To verify the withdrawal landed cleanly:</p>
+      <div class="hash" id="verify-cmd">curl -H "X-Lakehouse-Legal-Token: $TOKEN" http://localhost:3100/audit/subject/&lt;id&gt;</div>
+    </div>
+
+    <div style="margin-top:16px">
+      <button onclick="location.reload()" class="secondary">Record another withdrawal</button>
+    </div>
+  </div>
+
+</div>
+
+<div class="foot">
+  <a href="https://git.agentview.dev/profit/lakehouse/src/branch/main/docs/policies/consent/biometric_consent_template_v1.md">Consent template v1</a>
+  · <a href="https://git.agentview.dev/profit/lakehouse/src/branch/main/docs/runbooks/BIPA_DESTRUCTION_RUNBOOK.md">Destruction runbook</a>
+  · <a href="https://git.agentview.dev/profit/lakehouse/src/branch/main/docs/PHASE_1_6_BIPA_GATES.md">Phase 1.6 BIPA Gates</a>
+</div>
+
+<script>
+const GATEWAY = (function(){
+  const p = new URLSearchParams(location.search);
+  return p.get('gw') || 'http://localhost:3100';
+})();
+
+const state = { token: null, operator: null, response: null };
+
+function show(id) {
+  document.querySelectorAll('.screen').forEach(s => s.classList.remove('active'));
+  document.getElementById(id).classList.add('active');
+  const map = {'screen-token':'step 1 of 3', 'screen-form':'step 2 of 3', 'screen-done':'step 3 of 3'};
+  document.getElementById('step-label').textContent = map[id] || '';
+}
+function err(id, msg){ const e=document.getElementById(id); e.textContent=msg; e.classList.add('show'); }
+function clearErr(id){ document.getElementById(id).classList.remove('show'); }
+
+// Pre-populate candidate_id from URL if present.
+(function init(){
+  const p = new URLSearchParams(location.search);
+  const cid = p.get('candidate_id') || p.get('cid');
+  if (cid) document.getElementById('cand-id').value = cid;
+  // Restore token from sessionStorage if previously set in this tab.
+  const saved = sessionStorage.getItem('lh_legal_token');
+  const savedOp = sessionStorage.getItem('lh_operator_name');
+  if (saved) {
+    state.token = saved;
+    document.getElementById('token-input').value = '••••••••';
+  }
+  if (savedOp) {
+    state.operator = savedOp;
+    document.getElementById('op-name').value = savedOp;
+  }
+})();
+
+// Step 1.
+document.getElementById('token-submit').addEventListener('click', () => {
+  clearErr('token-error');
+  const v = document.getElementById('token-input').value.trim();
+  const tok = v === '••••••••' ? state.token : v;
+  const op = document.getElementById('op-name').value.trim();
+  if (!tok || tok.length < 32) { err('token-error', 'Token must be ≥32 characters.'); return; }
+  if (!op || op.length < 2) { err('token-error', 'Operator name is required.'); return; }
+  state.token = tok;
+  state.operator = op;
+  sessionStorage.setItem('lh_legal_token', tok);
+  sessionStorage.setItem('lh_operator_name', op);
+  show('screen-form');
+  refreshSubmit();
+});
+
+// Step 2.
+function refreshSubmit() {
+  const cid = document.getElementById('cand-id').value.trim();
+  const reason = document.getElementById('reason').value.trim();
+  document.getElementById('submit-btn').disabled = !(cid.length > 0 && reason.length >= 10);
+}
+['cand-id','reason'].forEach(id => document.getElementById(id).addEventListener('input', refreshSubmit));
+
+document.getElementById('back-btn').addEventListener('click', () => show('screen-token'));
+
+document.getElementById('submit-btn').addEventListener('click', async () => {
+  clearErr('form-error');
+  document.getElementById('submit-btn').disabled = true;
+  try {
+    const cid = document.getElementById('cand-id').value.trim();
+    const body = {
+      reason: document.getElementById('reason').value.trim(),
+      operator_of_record: state.operator,
+      evidence_path: document.getElementById('evidence').value.trim(),
+    };
+    const r = await fetch(`${GATEWAY}/biometric/subject/${encodeURIComponent(cid)}/withdraw`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+        'X-Lakehouse-Legal-Token': state.token,
+        'X-Lakehouse-Trace-Id': 'withdraw-' + Date.now(),
+      },
+      body: JSON.stringify(body),
+    });
+    const respBody = await r.json();
+    if (!r.ok) {
+      err('form-error', `${r.status} ${respBody.error || 'unknown'} — ${respBody.detail || ''}`);
+      refreshSubmit();
+      return;
+    }
+    state.response = respBody;
+    document.getElementById('done-cid').textContent = respBody.candidate_id;
+    document.getElementById('done-status').textContent = respBody.status_after;
+    document.getElementById('done-withdrawn-at').textContent = respBody.withdrawn_at;
+    document.getElementById('done-retention').textContent = respBody.retention_until;
+    document.getElementById('done-hmac').textContent = respBody.audit_row_hmac;
+    document.getElementById('verify-cmd').textContent =
+      `curl -H "X-Lakehouse-Legal-Token: $TOKEN" ${GATEWAY}/audit/subject/${respBody.candidate_id}`;
+    show('screen-done');
+  } catch (e) {
+    err('form-error', `Network error: ${e.message}`);
+    refreshSubmit();
+  }
+});
+</script>
+</body>
+</html>
@@ -778,6 +778,17 @@ async function main() {
          });
        }

+        // Biometric withdrawal — BIPA right of withdrawal frontend.
+        // Operator records candidate's withdrawal request. POSTs to
+        // gateway's /biometric/subject/{id}/withdraw. Sets a 30-day
+        // SLA clock for destruction. Optional ?candidate_id=WORKER-XXX
+        // pre-populates the form.
+        if (url.pathname === "/biometric/withdraw") {
+          return new Response(Bun.file(import.meta.dir + "/biometric_withdraw.html"), {
+            headers: { ...cors, "Content-Type": "text/html" },
+          });
+        }
+
        // Workspaces — per-contract state (Phase 8.5). UI layer over the
        // gateway's /workspaces/* routes: list, create, detail, handoff,
        // save-search, shortlist, log-activity. All persisted on the