catalogd: Step 2 — SubjectAuditWriter with HMAC chain
Implementation of docs/specs/SUBJECT_MANIFESTS_ON_CATALOGD.md Step 2.
Per-subject append-only audit JSONL with HMAC-SHA256 chain. Local-first
— no Vault, no external anchor (those are v2 if SOC2 Type II becomes
contract-required; v1 deliberately stays small).
shared/types.rs additions:
- AuditAccessor — kind, daemon, purpose, trace_id
- SubjectAuditRow — schema/ts/candidate_id/accessor/fields_accessed/
result/prev_chain_hash/row_hmac
crates/catalogd/src/subject_audit.rs (NEW):
- SubjectAuditWriter — holds signing key + per-subject latest-hash cache
- from_key_file() — loads key from sealed file, requires ≥32 bytes
- with_inline_key() — for tests + bring-up
- append() — computes HMAC chain link, persists JSONL row, returns new
chain root (caller mirrors to SubjectManifest.audit_log_chain_root)
- verify_chain() — full re-verification of a subject's audit log,
catches both prev_hash drift AND row-level HMAC tampering
- scan_latest_hash() — cold-start path, finds prev_hash from JSONL tail
- append_line() — read-modify-write pattern (object stores have no
native append; same shape as the rest of catalogd's persistence)
Crypto: HMAC-SHA256 via the standard `hmac` crate (added to workspace
+ catalogd deps; not implementing crypto by hand). Output is lowercase
hex matching the rest of the codebase's SHA-256 conventions.
Security choices:
- NO Debug impl on SubjectAuditWriter — auto-deriving Debug would risk
leaking the signing key into log lines. Tests work around this by
matching on Result instead of using .unwrap_err().
- Key min length 32 bytes (HMAC-SHA256 block size guidance).
- Failures are NOT swallowed — Result returned, caller decides whether
to log + continue (per spec §3.2 the gateway tool registry SHOULD
log + continue rather than block reads).
Tests (7/7 passing):
- first_append_uses_genesis_prev_hash
- chain_links_each_append (3-row chain verifies)
- separate_subjects_have_independent_chains (per-subject isolation)
- tamper_detected_on_verify (mutation in middle of chain breaks verify)
- cold_writer_picks_up_existing_chain (process restart preserves chain)
- empty_candidate_id_rejected
- key_too_short_rejected_via_file
NOT in this commit (future steps):
- Step 3: Backfill ETL from workers_500k.parquet (next per J)
- Step 4: Wire gateway tool registry to call append() on every
candidate_id returned by search_candidates / get_candidate
- Step 5: Wire validator WorkerLookup similarly
- Step 6: /audit/subject/{id} HTTP endpoint
- Step 7: Daily retention sweep
- Mirroring chain root to SubjectManifest.audit_log_chain_root
(separate concern; do at the call site)
cargo check --workspace clean. cargo test -p catalogd subject_audit
7/7 PASS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -510,3 +510,47 @@ pub struct SubjectManifest {
|
||||
}
|
||||
|
||||
fn default_subject_manifest_schema() -> String { "subject_manifest.v1".into() }
|
||||
|
||||
/// Audit accessor — who/why touched a subject's PII.
|
||||
/// Persisted in subject_audit.v1 rows; never logged in plaintext outside
|
||||
/// the per-subject audit file.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct AuditAccessor {
|
||||
/// One of: "gateway_lookup", "audit_response", "legal_request",
|
||||
/// "system_resolve". Caller-defined; SubjectAuditWriter does not
|
||||
/// validate the kind string.
|
||||
pub kind: String,
|
||||
/// Daemon name (e.g. "gateway", "validator", "catalogd").
|
||||
pub daemon: String,
|
||||
/// Purpose token (e.g. "fill_validation", "audit_subject_response").
|
||||
pub purpose: String,
|
||||
/// X-Lakehouse-Trace-Id for cross-daemon trace correlation. Empty
|
||||
/// when the request didn't carry one.
|
||||
#[serde(default)]
|
||||
pub trace_id: String,
|
||||
}
|
||||
|
||||
/// One audit row in a subject's per-subject audit JSONL.
|
||||
/// HMAC chain: row_hmac = HMAC-SHA256(key, prev_chain_hash || canon(row - row_hmac))
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SubjectAuditRow {
|
||||
/// Always "subject_audit.v1" for v1.
|
||||
#[serde(default = "default_subject_audit_schema")]
|
||||
pub schema: String,
|
||||
pub ts: chrono::DateTime<chrono::Utc>,
|
||||
pub candidate_id: String,
|
||||
pub accessor: AuditAccessor,
|
||||
/// Field names accessed (e.g. ["name"], ["name", "phone"]).
|
||||
pub fields_accessed: Vec<String>,
|
||||
/// "success" | "denied" | "not_found" | "error".
|
||||
pub result: String,
|
||||
/// Hash of the previous row in this subject's audit log. "GENESIS"
|
||||
/// for the first row.
|
||||
pub prev_chain_hash: String,
|
||||
/// HMAC-SHA256 over (prev_chain_hash || canonical_json(row - row_hmac)).
|
||||
/// Computed by SubjectAuditWriter; callers leave empty on input.
|
||||
#[serde(default)]
|
||||
pub row_hmac: String,
|
||||
}
|
||||
|
||||
fn default_subject_audit_schema() -> String { "subject_audit.v1".into() }
|
||||
|
||||
Reference in New Issue
Block a user