G0 D2: storaged S3 GET/PUT/LIST/DELETE · 3-lineage scrum · 4 fixes applied

Phase G0 Day 2 ships storaged: aws-sdk-go-v2 wrapper + chi routes
binding 127.0.0.1:3211 with 256 MiB MaxBytesReader, Content-Length
up-front 413, and a 4-slot non-blocking semaphore returning 503 +
Retry-After:5 when full. Acceptance smoke (6/6 probes) PASSES against
the dedicated MinIO bucket lakehouse-go-primary, isolated from the
Rust system's lakehouse bucket during coexistence.

Cross-lineage scrum on the shipped code:
  - Opus 4.7 (opencode): 1 BLOCK + 3 WARN + 3 INFO
  - Qwen3-coder (openrouter): 2 BLOCK + 1 WARN + 1 INFO (3 false positives)
  - Kimi K2-0905 (openrouter, after route-shopping past opencode's 4k
    cap and the direct adapter's empty-content reasoning bug):
    1 BLOCK + 2 WARN + 1 INFO

Fixed:
  C1 buildRegistry ctx cancel footgun → context.Background()
     (Opus + Kimi convergent; future credential refresh chains)
  C2 MaxBytesReader unwrap through manager.Uploader multipart
     goroutines → Content-Length up-front 413 + string-suffix fallback
     (Opus + Kimi convergent; latent 500-instead-of-413 in 5-256 MiB range)
  C3 Bucket.List unbounded accumulation → MaxListResults=10_000 cap
     (Opus + Kimi convergent; OOM guard)
  S1 PUT response Content-Type: application/json (Opus single-reviewer)

Strict validateKey policy (J approved): rejects empty, >1024B, NUL,
leading "/", ".." path components, CR/LF/tab control characters.
DELETE exposed at HTTP layer (J approved option A) for symmetry +
smoke ergonomics.

Build clean, vet clean, all unit tests pass, smoke 6/6 PASS after
every fix round. go.mod 1.23 → 1.24 (required by aws-sdk-go-v2).

Process finding worth recording: opencode caps non-streaming Kimi at
max_tokens=4096; the direct kimi.com adapter consumed 8192 tokens of
reasoning but surfaced empty content; openrouter/moonshotai/kimi-k2-0905
delivered structured output in ~33s. Future Kimi scrums should default
to that route.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/PHASE_G0_KICKOFF.md

@@ -435,3 +435,125 @@ D1 ships harder than it would have without the scrum: bind-error
 handling is now race-free, smoke is deterministic, log volume on
 /health is acknowledged, secrets handling has a flagged path
 forward.
+
+---
+
+## D2 — actual run results (2026-04-28 evening)
+
+Phase G0 Day 2 executed end-to-end. Output of `scripts/d2_smoke.sh`:
+
+```
+[d2-smoke] PUT round-trip:
+  ✓ PUT d2-smoke/<ts>.bin → 200
+[d2-smoke] GET echoes bytes:
+  ✓ GET d2-smoke/<ts>.bin → bytes match
+[d2-smoke] LIST includes key:
+  ✓ LIST prefix=d2-smoke/ → contains d2-smoke/<ts>.bin
+[d2-smoke] DELETE then GET → 404:
+  ✓ DELETE then GET → 404
+[d2-smoke] 256 MiB cap → 413:
+  ✓ PUT 257 MiB → 413
+[d2-smoke] semaphore: 5th concurrent PUT → 503 + Retry-After:5
+  ✓ 5th concurrent PUT → 503 + Retry-After: 5
+[d2-smoke] D2 acceptance gate: PASSED
+```
+
+What landed:
+- `internal/secrets/provider.go` — `Provider` interface, `FileProvider`
+  reading `/etc/lakehouse/secrets-go.toml` with inline-fallback for
+  G0 dev convenience, `StaticProvider` test helper
+- `internal/storaged/bucket.go` — `aws-sdk-go-v2/service/s3` wrapper:
+  Get/Put/List/Delete; `manager.Uploader` for multipart-streaming PUT;
+  `MaxListResults=10_000` cap with `...truncated...` sentinel
+- `internal/storaged/registry.go` — `BucketRegistry` (single bucket
+  in G0; G2 multi-bucket federation extends this)
+- `cmd/storaged/main.go` — verb-paths `GET/PUT/LIST/DELETE`, strict
+  `validateKey` (rejects empty, >1024B, NUL, leading-`/`, `..`,
+  CR/LF/tab), Content-Length up-front 413, `MaxBytesReader` 256 MiB
+  body cap, 4-slot non-blocking semaphore (503+Retry-After:5)
+- `scripts/d2_smoke.sh` — 6 acceptance probes; uses `curl -T --limit-rate`
+  for true streaming uploads (`--data-binary @-` first attempt
+  buffered client-side, semaphore never engaged)
+- Per-package unit tests for provider + registry + validateKey
+- New bucket `lakehouse-go-primary` on the existing MinIO at `:9000`,
+  isolated from the Rust `lakehouse` bucket during coexistence
+
+Out of spec but added: DELETE was exposed at the HTTP layer (the
+kickoff D2.4 listed only GET/PUT/LIST in routes; `bucket.Delete` was
+in the wrapper). J approved option **A** (DELETE exposed) for symmetry
++ smoke ergonomics.
+
+`validateKey` policy: J approved the strict stance — leading `/`,
+`..` components, and CR/LF/tab control characters all rejected at the
+HTTP boundary. Costs ~5 lines, propagates safety to every downstream
+consumer.
+
+Next: D3 — catalogd Parquet manifests with idempotent register.
+
+---
+
+## D2 — code scrum review (3-lineage parallel pass)
+
+After D2 shipped, the actual code went through the same 3-lineage
+parallel scrum as D1.
+
+| Pass | Reviewer | Latency | Findings |
+|---|---|---|---|
+| 1 | `opencode/claude-opus-4-7` | ~30s | 1 BLOCK + 3 WARN + 3 INFO = 7 |
+| 2 | `openrouter/qwen/qwen3-coder` | ~32s | 2 BLOCK + 1 WARN + 1 INFO = 4 |
+| 3a | `opencode/kimi-k2.6` (max_tokens=4096) | 18s | discursive — finish_reason=length, no structured output (model spent budget thinking, never reached BLOCK/WARN format). opencode rejected `max_tokens>4096` without `stream=true` |
+| 3b | `kimi/kimi-k2-turbo` (direct adapter) | 124s | empty content, finish_reason=length (8192 reasoning tokens, no surfaced output) |
+| 3c | `openrouter/moonshotai/kimi-k2-0905` | 33s | 1 BLOCK + 2 WARN + 1 INFO = 4 (used as the K-lineage representative) |
+
+The Kimi route shopping (3a → 3c) was a process finding worth
+recording: opencode caps non-streaming Kimi calls at 4096 output
+tokens, the direct kimi.com adapter consumed 8192 tokens of
+reasoning but surfaced empty `content`, and openrouter's
+`moonshotai/kimi-k2-0905` route delivered structured output in
+~33s. For future scrums on Kimi, default to
+`openrouter/moonshotai/kimi-k2-0905`.
+
+### Convergent findings (≥2 reviewers — high confidence)
+
+| # | Severity | Finding | Reviewers | Disposition |
+|---|---|---|---|---|
+| C1 | BLOCK | `buildRegistry` `defer cancel()` cancels the ctx that the AWS SDK was loaded with — fine today (static creds, sync call) but breaks future credential refresh chains (EC2 IMDS, SSO, AssumeRole) | Opus + Kimi | **Fixed** — switched to `context.Background()` for SDK construction. Per-request lifetimes flow through `r.Context()` |
+| C2 | WARN→BLOCK | `*http.MaxBytesError` may not unwrap through `manager.Uploader`'s multipart goroutines for bodies >5 MiB; smoke covers 257 MiB single-PutObject path only — bodies in the multipart range that exceed 256 MiB could surface as 500 instead of 413 | Opus (WARN) + Kimi (BLOCK) | **Fixed** — added Content-Length up-front 413 (deterministic for honest clients) + string-suffix fallback (`"http: request body too large"`) on the Put error path for chunked / lying-CL cases |
+| C3 | INFO→WARN | `Bucket.List` accumulates unbounded results into a slice; stray `prefix=""` against a large bucket OOMs the daemon | Opus (INFO) + Kimi (WARN) | **Fixed** — `MaxListResults=10_000` cap; truncated responses append a sentinel `ObjectInfo{Key: "...truncated..."}` so callers see they didn't get everything |
+
+### Single-reviewer findings (lineage-specific catches)
+
+| # | Severity | Finding | Reviewer | Disposition |
+|---|---|---|---|---|
+| S1 | WARN | PUT 200 response missing `Content-Type: application/json` — Go's content-sniffing won't catch the JSON prefix, clients may see `text/plain` | Opus | **Fixed** — explicit header set before `WriteHeader` |
+| S2 | WARN | `Bucket.Get` body close ordering fragile — currently correct but the contract is implicit | Opus | **Accepted** — current order is correct (early returns on errors where body is nil); no fix required |
+| S3 | INFO | `extractKey` uses chi `*` wildcard — works correctly, just noting empty-key path is covered by `validateKey` | Opus | **Accepted** — already covered |
+| S4 | INFO | `FileProvider.mu sync.RWMutex` is unused given the immutable-after-construction design | Opus | **Accepted** — drop the mutex when SIGHUP reload lands in G1 (S6 below); keep it for now as a placeholder |
+| S5 | INFO | `Bucket.Delete` doesn't translate not-found to `ErrKeyNotFound` | (Kimi noted, not flagged) | **Accepted** — S3 DeleteObject is idempotent by spec; non-error on missing key is the correct behavior |
+| S6 | INFO | `FileProvider` never reloads (no SIGHUP handler) | Kimi | **Deferred to G1** — reload-on-SIGHUP is on the G1 list per `internal/shared/server.go` opening comment |
+| S7 | INFO | Error messages on no-creds-found could specify which source (file vs fallback) was missing | Qwen | **Deferred** — minor debug ergonomics, no production impact |
+| F1 | BLOCK | "Body close happens after semaphore release" | Qwen | **Dismissed** — false. Defer order is LIFO; `r.Body.Close()` was registered AFTER `<-h.putSem`, so it fires FIRST. Body closes before slot frees |
+| F2 | BLOCK | "ObjectInfo fields can be nil-dereferenced in List" | Qwen | **Dismissed** — false. `bucket.go:147-160` checks `if o.Key != nil` etc. before dereferencing every field |
+| F3 | WARN | "MaxBytesReader not applied to semaphore-protected path" | Qwen | **Dismissed** — false. Semaphore is non-blocking try-acquire (`select { default }`); there is no waiting state where pre-cap MaxBytesReader matters |
+
+### Cumulative D2 disposition
+
+- 3-lineage parallel pass: **15 distinct findings** (3 convergent + 7 single-reviewer real + 3 false + 2 absorbed by other findings)
+- **Fixed: 5** (3 convergent + 2 single-reviewer Opus)
+- **Accepted-with-rationale: 5**
+- **Deferred to G1+: 2**
+- **Dismissed (false positives): 3** (all from Qwen — its strength on D1 doesn't reproduce on D2 code; the lineage caught real C1/C2 issues there but misread defer order + nil-checks here)
+- Build clean, vet clean, all tests pass, smoke 6/6 PASS after every fix round.
+
+The Kimi/Opus convergence on **C1 (ctx cancel)** is the highest-signal
+finding of the round: two completely different lineages flagged the
+same architectural footgun. C2 (MaxBytesReader unwrap) was the most
+*consequential* — the smoke test would have stayed green while
+production multi-MiB uploads silently returned 500 on oversize. C3
+was a latent OOM that's a 5-line fix.
+
+The Qwen lineage delivered three false BLOCKs on D2 — different from
+its D1 contribution where it caught two real BLOCKs that Opus missed.
+Lineage rotation is real; on a given PR, one lineage may be the only
+one finding bugs and another may be confidently wrong. The convergence
+filter (≥2 reviewers) is the right gate.