distillation: audit-FULL pipeline port (phases 0/3/4) — cross-runtime metric parity verified
Ports the metric-collection passes from scripts/distillation/audit_full.ts.
The substrate that PRODUCES audit_baselines.jsonl entries — the
half OPEN #2 left as "deferred to next wave" after the read/write
substrate landed in ca142b9.
Phase coverage:
Phase 0 (file presence) ported
Phase 1 (schema validators) skipped (Go's `go test` covers it)
Phase 2 (materializer dry-run) deferred (Go materializer not yet ported)
Phase 3 (scored-runs distribution) ported
Phase 4 (contamination firewall) ported
Phase 5 (receipts validation) deferred (Go run-summary JSON not yet emitted)
Phase 6 (replay sanity) deferred (Go replay tool not ported)
Phase 7 (run summary lineage) deferred (same)
Cross-runtime parity verified end-to-end:
Go-side audit-full against /home/profit/lakehouse produced
metrics IDENTICAL to the last Rust-emitted audit_baselines.jsonl
entry. All 8 ported metrics match byte-for-byte:
p3_accepted=386, p3_partial=132, p3_rejected=57, p3_human=480,
p4_sft_rows=353, p4_rag_rows=448, p4_pref_pairs=83, p4_total_quarantined=1325
6/6 required checks pass on live data.
Components:
- internal/distillation/audit_full.go: PhaseCheck struct (mirrors
Rust shape), PhaseCheckReport aggregation, RunAuditFull
orchestrator, auditPhase0/3/4 implementations, FormatAuditFullReport
Markdown writer.
- cmd/audit_full/main.go: CLI binary with -root, -out, -json,
-append-baseline flags. Operators run "./bin/audit_full
-append-baseline" to grow the longitudinal log alongside the
Rust pipeline (entries are interchangeable — same envelope shape).
- 6 new tests: empty-root failure handling, full-fixture clean PASS
(locks all 8 metrics + all 6 required checks), SFT firewall
contamination detection, preference self-pair detection, sig_hash
regex correctness (rejects wrong-length + uppercase), Markdown
formatter smoke.
Live-data probe captured at reports/cutover/audit_full_go_vs_rust.md
(linked from reports/cutover/SUMMARY.md). Same shape as the
audit_baselines round-trip evidence — both Go-side ports of the
distillation surface are now validated against real Rust data, not
just fixtures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,7 @@ what's safe to flip. Append a row when a new endpoint clears parity.
|
||||
| `embed` (forced v1) | 2026-04-30 | `/ai/embed` | `/v1/embed` | ✅ PASS 5/5 cos=1.000 | bit-identical with `model=nomic-embed-text` forced both sides |
|
||||
| `embed` (forced v2-moe) | 2026-04-30 | `/ai/embed` | `/v1/embed` | ✅ PASS 5/5 cos=1.000 | bit-identical with `model=nomic-embed-text-v2-moe` forced both sides — both Ollamas have the model |
|
||||
| `audit_baselines.jsonl` | 2026-05-01 | `data/_kb/audit_baselines.jsonl` | `internal/distillation` `LoadLastBaseline` / `AppendBaseline` / `BuildAuditDriftTable` | ✅ PASS round-trip | Live Rust file (7 entries) parses + round-trips byte-equal; lineage drift table fires correctly on zero-baseline metrics. See `audit_baselines_roundtrip.md`. |
|
||||
| `audit-FULL` (phases 0/3/4) | 2026-05-01 | `scripts/distillation/audit_full.ts` | `cmd/audit_full` + `internal/distillation` `RunAuditFull` | ✅ PASS metric-equal | Go-side run against live Rust root: all 8 ported metrics (p3_*, p4_*) byte-equal to the last Rust-emitted `audit_baselines.jsonl` entry. 6/6 required checks pass. 4 phases (1, 2, 5, 6, 7) deferred — depend on broader Rust-side pieces (materializer / replay / run-summaries) not yet ported. See `audit_full_go_vs_rust.md`. |
|
||||
|
||||
## Wire-format drift catalog
|
||||
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# Audit-FULL report (Go)
|
||||
|
||||
**git HEAD:** `eb0dfdff047e34439896552d483abbee673d5a47`
|
||||
|
||||
**Verdict:** PASS — 6/6 required checks passed; 4 phase(s) deferred.
|
||||
|
||||
## Checks
|
||||
|
||||
| phase | name | expected | actual | required | passed |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | recon doc exists | docs/recon/local-distillation-recon.md present | true | no | ✓ |
|
||||
| 0 | tier-1 source streams present | all 4 tier-1 jsonls on disk | all present | no | ✓ |
|
||||
| 3 | on-disk scored-runs distribution non-empty | >=1 accepted | acc=386 part=132 rej=57 hum=480 | **yes** | ✓ |
|
||||
| 3 | scored-runs distribution sums positive | >0 total | 1055 total | no | ✓ |
|
||||
| 4 | SFT contamination firewall: 0 forbidden quality_scores | 0 | 0 | **yes** | ✓ |
|
||||
| | _note_ | this is the spec non-negotiable — rejected/needs_human_review must NEVER appear in SFT | | | |
|
||||
| 4 | RAG firewall: 0 rejected leaks | 0 | 0 | **yes** | ✓ |
|
||||
| 4 | Preference: 0 self-pairs (chosen_run_id != rejected_run_id) | 0 | 0 | **yes** | ✓ |
|
||||
| 4 | Preference: 0 identical-text pairs | 0 | 0 | **yes** | ✓ |
|
||||
| 4 | every export row carries valid sha256 provenance.sig_hash | 0 missing | 0 missing | **yes** | ✓ |
|
||||
|
||||
## Metrics
|
||||
|
||||
| metric | value |
|
||||
|---|---:|
|
||||
| p3_accepted | 386 |
|
||||
| p3_human | 480 |
|
||||
| p3_partial | 132 |
|
||||
| p3_rejected | 57 |
|
||||
| p4_pref_pairs | 83 |
|
||||
| p4_rag_rows | 448 |
|
||||
| p4_sft_rows | 353 |
|
||||
| p4_total_quarantined | 1325 |
|
||||
Reference in New Issue
Block a user