parity: session_log probe + Rust observability parity recorded

Companion to lakehouse commit 57bde63 (Rust gateway gains
trace-id propagation + coordinator session JSONL). The
cross-runtime parity probe is the regression gate that prevents
silent schema drift between the two runtimes.

scripts/cutover/parity/session_log_parity.sh:
  - 4 fixtures (accepted_grounded, max_iter_exhausted, infra_error,
    unicode_in_prompt) feed identical input to both helpers
  - jq -e validity gate + non-trivial-equal guard prevents the
    "both sides fail identically → spurious match" failure mode
    (caught one IFS='||' bug during initial authoring — recorded
    in the script comment)
  - normalize() strips timestamp + daemon (legitimate per-producer
    differences); everything else must be byte-equal
  - Result: 4/4 fixtures match, including unicode

scripts/cutover/parity/session_log_helper/main.go:
  - Tiny stdin/stdout Go helper that round-trips a fixture
    through validator.SessionRecord serde
  - Counterpart to crates/gateway/src/bin/parity_session_log.rs

docs/ARCHITECTURE_COMPARISON.md decisions tracker:
  - "Rust observability parity" row added (DONE 2026-05-02)
  - Cross-runtime probe documented as reusable gate

STATE_OF_PLAY refreshed.

Both observability pieces (trace-id propagation, session JSONL)
now exist on both runtimes. Operators who point Rust gateway and
Go validatord at the same session-log path get a unified
longitudinal stream queryable via DuckDB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/ARCHITECTURE_COMPARISON.md

@@ -49,6 +49,7 @@ Don't:
 | 2026-05-01 | **coder/hnsw v0.6.1 panic — REAL FIX landed** | Lifted source-of-truth out of coder/hnsw via `i.vectors map[string][]float32` side store + `safeGraphAdd`/`safeGraphDelete` recover wrappers + warm-path rebuild fallback. Re-run: 0 failures across 19,622 scenarios (was 96-98% on 2/6). **DONE.** Architecture invariant in STATE_OF_PLAY "DO NOT RELITIGATE". |
 | 2026-05-02 | **Substrate fix verified at original failure scale** | Re-ran multitier 5min @ conc=50 (the footprint that originally surfaced the bug at 96-98% fail). Result: 132,211 scenarios at 438/sec, **6/6 classes at 0% failure**. Throughput dropped 1,115/sec → 438/sec because broken scenarios now do real HNSW Add work. Tails healthy: surge_fill_validate p99=1.53s, playbook_record_replay p99=2.32s. **Fix scales — closing the open thread.** |
 | 2026-05-02 | **Drop Python sidecar from Rust aibridge — DONE** | `crates/aibridge/src/client.rs` rewrite (commit `ba928b1` in lakehouse). AiClient now talks Ollama directly: per-text `/api/embed` loop, `/api/generate` for chat + rerank-loop + admin (unload/preload), `/api/ps` + `std::process::Command nvidia-smi` for vram_snapshot. Public API unchanged — 0 callers updated. Verification: cargo test -p aibridge 32/32 PASS, live smoke `/ai/embed` returns 768-dim vector + `/v1/chat` returns "OK". Sidecar's `lab_ui.py` + `pipeline_lab.py` (~888 LOC dev-only UIs) keep running; only the hot-path embed/generate/rerank/admin routes are retired. Process count drops from "mega-binary + sidecar" to "mega-binary alone". |
+| 2026-05-02 | **Rust observability parity (trace-id + session JSONL) — DONE** | Mirrors the Go-side wave (commits `d6d2fdf` + `1a3a82a` in golangLAKEHOUSE). `crates/gateway/src/v1/iterate.rs` now reads `X-Lakehouse-Trace-Id` header, forwards to `/v1/chat` + `/v1/validate` hops, emits per-attempt Langfuse spans (`iterate.attempt[N]`), and writes one `SessionRecord` JSONL row per session via the new `crates/gateway/src/v1/session_log.rs` writer. New `crates/gateway/src/bin/parity_session_log` helper enables the cross-runtime parity probe at `scripts/cutover/parity/session_log_parity.sh` — **4/4 fixtures byte-equal** (including unicode prompts) after normalizing the 2 fields that legitimately differ (timestamp, daemon name). Both runtimes can write to the same path and DuckDB queries see them as one stream. 90/90 Rust unit tests PASS, 33 Go packages PASS. |
 | 2026-05-02 | **Port Rust materializer to Go (transforms.ts) — DONE** | `internal/materializer` + `cmd/materializer` + `materializer_smoke.sh`. Ports `transforms.ts` (12 transforms) + `build_evidence_index.ts`. Idempotency, day-partition, receipt. 14 tests green; on-wire JSON matches TS so both runtimes interoperate. |
 | 2026-05-02 | **Port Rust replay tool to Go — DONE** | `internal/replay` + `cmd/replay` + `replay_smoke.sh`. Ports `replay.ts` retrieve → bundle → /v1/chat → validate → log. Closes audit-FULL phase 7 live invocation on Go side. 14 tests green; same `data/_kb/replay_runs.jsonl` shape (schema=replay_run.v1) as TS. |
 | 2026-05-02 | **`/v1/validate` + `/v1/iterate` HTTP surface — DONE** | `cmd/validatord` (port 3221) hosts both endpoints. `internal/validator` gains `PlaybookValidator` (3rd kind), JSONL roster loader, and the `Iterate` orchestrator + `ExtractJSON` helper. Gateway proxies `/v1/validate` + `/v1/iterate` to validatord. Closes the last "Go-primary" backlog item (architecture_comparison.md item #7). 30+ tests + `validatord_smoke.sh` 5/5 PASS. |