playbook_lift v4 metric: warm-top-1 re-judge — quality lift +24%/-14%

The rank-based "lift" metric (warm-top-1 == cold-judge-best) doesn't
distinguish "Shape B surfaced a strictly-better answer" from "Shape B
shuffled ranks but quality is unchanged" from "Shape B replaced a good
answer with a wrong one." This commit adds Pass 4: judge warm top-1
with the same prompt as cold ratings, then bucket the comparison.

Implementation:
- New --with-rejudge driver flag (default off).
- New WITH_REJUDGE harness env (default 1, on for prod runs).
- queryRun gains WarmTop1Metadata (cached during Pass 2 for the
  rejudge call) + WarmTop1Rating *int (nil-distinguishable; nil = no
  rejudge, 0..5 = rating).
- summary gains RejudgeAttempted, QualityLifted, QualityNeutral,
  QualityRegressed (counts of warm-rating > / == / < cold-rating).
- Markdown headline gains a Quality block when rejudge ran.
- ~21 extra judge calls (~30s on qwen2.5).

Run #005 result (split inject threshold 0.20 + paraphrase + rejudge):

  Quality lifted     5 / 21  (24%)  — 3× +2 rating, 2× +1 rating
  Quality neutral   13 / 21  (62%)  — includes OOD queries holding 1
  Quality regressed  3 / 21  (14%)
  Net rating delta  +3 across 21 queries (+0.14 average)

The 5 lifts were all rating-2 cold replaced with rating-3 or rating-4
warm — Shape B took mediocre matches and substituted substantively
better ones. The 3 regressions were small (-1, -1, -3).

Q11 is the cautionary tale: cold top-1 "production line worker"
(rating 4) got replaced by Q1's recorded "forklift OSHA-30 operator"
e-5729 (rating 1). Adjacent-domain cross-pollination — production
worker and forklift operator embed within 0.20 cosine because both
are warehouse-adjacent staffing queries, even though the judge
correctly distinguishes them. The split-threshold defense (0.5 boost
/ 0.20 inject) catches OOD cross-pollination (Q19/Q20/Q21 all stayed
neutral at rating 1) but not adjacent-domain cross-pollination.

Net product verdict: working, net-positive on quality, but the worst
case (Q11 4→1) is customer-visible and warrants a tighter inject
threshold OR an additional gate beyond cosine distance. Filed in
STATE_OF_PLAY OPEN as a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

scripts/playbook_lift.sh

@@ -52,6 +52,11 @@ CONFIG_PATH="${CONFIG_PATH:-lakehouse.toml}"
 # actual learning-property test (does cosine on paraphrase find the
 # recorded entry?). Set WITH_PARAPHRASE=0 for a faster verbatim-only run.
 WITH_PARAPHRASE="${WITH_PARAPHRASE:-1}"
+# WITH_REJUDGE=1 (default) adds a Pass 4 — judge warm top-1 to measure
+# quality lift (warm rating vs cold rating). Catches cases where Shape B
+# surfaces a different-but-equally-good answer (which the rank-based
+# lift metric misses). +21 judge calls (~30s on qwen2.5).
+WITH_REJUDGE="${WITH_REJUDGE:-1}"
 
 OUT_JSON="reports/reality-tests/playbook_lift_${RUN_ID}.json"
 OUT_MD="reports/reality-tests/playbook_lift_${RUN_ID}.md"
@@ -271,9 +276,12 @@ echo "[lift] running driver — judge=$EFFECTIVE_JUDGE · queries=$QUERIES_FILE
 # and runs its own resolution chain (env → config → fallback). When
 # JUDGE_MODEL IS set, the explicit -judge wins inside the Go driver
 # regardless of what its env-lookup would find — flag wins by design.
-PARAPHRASE_FLAG=""
+EXTRA_FLAGS=""
 if [ "$WITH_PARAPHRASE" = "1" ]; then
-  PARAPHRASE_FLAG="-with-paraphrase"
+  EXTRA_FLAGS="$EXTRA_FLAGS -with-paraphrase"
+fi
+if [ "$WITH_REJUDGE" = "1" ]; then
+  EXTRA_FLAGS="$EXTRA_FLAGS -with-rejudge"
 fi
 ./bin/playbook_lift \
   -config  "$CONFIG_PATH" \
@@ -284,7 +292,7 @@ fi
   -judge   "$JUDGE_MODEL" \
   -k       "$K" \
   -out     "$OUT_JSON" \
-  $PARAPHRASE_FLAG
+  $EXTRA_FLAGS
 
 echo
 echo "[lift] generating markdown report → $OUT_MD"
@@ -302,6 +310,10 @@ generate_md() {
   p_attempted=$(jq -r '.summary.paraphrase_attempted // 0' "$json")
   p_top1=$(jq -r '.summary.paraphrase_top1_lifts // 0' "$json")
   p_anyrank=$(jq -r '.summary.paraphrase_any_rank_hits // 0' "$json")
+  rj_attempted=$(jq -r '.summary.rejudge_attempted // 0' "$json")
+  q_lifted=$(jq -r '.summary.quality_lifted // 0' "$json")
+  q_neutral=$(jq -r '.summary.quality_neutral // 0' "$json")
+  q_regressed=$(jq -r '.summary.quality_regressed // 0' "$json")
 
   # Only emit the paraphrase block when --with-paraphrase actually ran
   # (i.e. .summary.paraphrase_attempted > 0). For verbatim-only runs we
@@ -312,6 +324,13 @@ generate_md() {
 | Paraphrase pass — recorded answer at any rank in top-K | ${p_anyrank} / ${p_attempted} |"
   fi
 
+  rj_block=""
+  if [ "$rj_attempted" != "0" ] && [ "$rj_attempted" != "null" ]; then
+    rj_block="| **Quality lift** (warm top-1 rating > cold top-1 rating) | **${q_lifted} / ${rj_attempted}** |
+| Quality neutral (warm top-1 rating = cold top-1 rating) | ${q_neutral} / ${rj_attempted} |
+| Quality regressed (warm top-1 rating < cold top-1 rating) | ${q_regressed} / ${rj_attempted} |"
+  fi
+
   cat > "$md" <<MDEOF
 # Playbook-Lift Reality Test — Run ${RUN_ID}
 
@@ -322,6 +341,7 @@ generate_md() {
 **Queries:** \`${QUERIES_FILE}\` (${total} executed)
 **K per pass:** ${K}
 **Paraphrase pass:** $([ "$WITH_PARAPHRASE" = "1" ] && echo "ENABLED" || echo "disabled")
+**Re-judge pass:** $([ "$WITH_REJUDGE" = "1" ] && echo "ENABLED" || echo "disabled")
 **Evidence:** \`${OUT_JSON}\`
 
 ---
@@ -337,6 +357,7 @@ generate_md() {
 | Playbook boosts triggered (warm pass) | ${boosted} |
 | Mean Δ top-1 distance (warm − cold) | ${mean_delta} |
 ${p_block}
+${rj_block}
 
 **Verbatim lift rate:** ${lift} of ${discovery} discoveries became top-1 after warm pass.