27b1d276053716bc534d9e098d868b3834a49f5f
100
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rootandClaude Opus 4.7
|
27b1d27605 |
distillation: Phase 0 recon + Phase 1 schemas + Phase 2 transforms scaffold
lakehouse/auditor 9 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Phase 0 — docs/recon/local-distillation-recon.md
Inventories the 23 KB JSONL streams + 20 vector corpora + auditor's
kb_index.ts as substrate for the now.md distillation pipeline. Maps
spec modules to existing producers, identifies real gaps, lists 9
schemas to formalize. ZERO implementation in recon — gating doc only.
Phase 1 — auditor/schemas/distillation/
9 schemas + foundation types + 48 tests passing in 502ms:
types.ts shared validators + canonicalSha256
evidence_record.ts EVIDENCE_SCHEMA_VERSION=1, ModelRole enum
scored_run.ts 4 categories pinned, anchor_grounding ∈ [0,1]
receipt.ts git_sha 40-char, sha256 file refs, validation_pass:bool
playbook.ts non-empty source_run_ids + acceptance_criteria
scratchpad_summary.ts validation_status enum, hash sha256
model_ledger.ts success_rate ∈ [0,1], sample_count ≥ 1
rag_sample.ts success_score ∈ {accepted, partially_accepted}
sft_sample.ts quality_score MUST be 'accepted' (no leak)
preference_sample.ts chosen != rejected, source_run_ids must differ
evidence_record.test.ts 10 tests, JSON-fixture round-trip
schemas.test.ts 30 tests, inline fixtures
realdata.test.ts 8 tests, real-JSONL probe
Real-data validation probe (one of the 3 notables from recon):
46 rows across 7 sources, 100% pass. distilled_facts/procedures alive.
Report at data/_kb/realdata_validation_report.md (also written by the
test). Confirms schema fits existing producers without migration.
Phase 2 scaffold — scripts/distillation/transforms.ts
Promoted PROBES from realdata.test.ts into a real TRANSFORMS array
covering 12 source streams (8 Tier 1 validated + 4 Tier 2 from
recon's untested-streams list). Pure functions: no I/O, no model
calls, no clock reads. Caller supplies recorded_at + sig_hash so
materializer is deterministic by construction.
Spec non-negotiables enforced at schema layer (defense in depth):
- provenance{source_file, sig_hash, recorded_at} required everywhere
- schema_version mismatch hard-rejects (forward-compat gate)
- SFT no-leak: validateSftSample REJECTS partially_accepted, rejected,
needs_human_review — three explicit tests
- Every score has WHY (reasons non-empty)
- Every playbook traces to source (source_run_ids non-empty)
- Every preference has WHY (reason non-empty)
- Receipts substantive (git_sha 40-char, sha256 64-char, validation_pass:bool)
Branch carries uncommitted auditor rebuild work (mode.rs + modes.toml
+ inference.ts + static.ts) blocked on upstream Ollama Cloud kimi-k2
500 ISE; held pending recon-driven design decisions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
f753e11157 |
docs: SCRUM_MASTER_SPEC timeline — productization wave + verified live state
lakehouse/auditor 9 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Splits the existing 04-25/26 section into two waves:
- experiment wave (mode-runner build-out, pre-productization)
- productization wave (OpenAI-compat, Archon, answers corpus,
staffing native runner, multi-corpus + downgrade gate, observer
paid escalation, /v1/chat → observer event wiring)
Adds verified-live block at the end with the numbers a fresh session
needs to anchor on: pathway memory 88 traces / 11 successful replays
at 100% (probation gate crossed), strong-model auto-downgrade firing
on grok-4.1-fast, and the auditor blind spot at static.ts:117 (now
fixed in
|
||
|
rootandClaude Opus 4.7
|
107a68224d |
auditor: skip serde-derived structs in unread-field check
Fields on structs that derive Serialize or Deserialize ARE read — by
the macro, on every JSON round-trip — but the static check only
looked for explicit `.field` references in the diff. Result: every
new response/request struct shipped through `/v1/*` was flagged as
"placeholder state without a consumer."
PR #11 head
|
||
|
rootandClaude Opus 4.7
|
0844206660 |
observer + scrum: gold-standard answer corpus for compounding context
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
The compose-don't-add discipline applied to the original ask: when big
models produce good results (scrum reviews + observer escalations),
save them into the matrix indexer so future small-model handlers can
retrieve them as scaffolding. Local model gets near-paid quality from
a fraction of the cost.
New: scripts/build_answers_corpus.ts indexes lakehouse_answers_v1
from data/_kb/scrum_reviews.jsonl + data/_kb/observer_escalations.jsonl.
doc_id prefixes ('review:' vs 'escalation:') let consumers same-file-
gate the prior-reviews case while keeping escalations broad.
observer.ts: buildKbPreamble adds lakehouse_answers_v1 as a third
retrieval source alongside pathway/bug_fingerprints + lakehouse_arch_v1.
qwen3.5:latest synthesis now compresses three lenses into a single
briefing for the cloud reviewer.
scrum_master_pipeline.ts: epilogue dispatches a fire-and-forget rebuild
of lakehouse_answers_v1 after each run so this run's accepted reviews
are retrievable within ~30s. LH_SCRUM_SKIP_ANSWERS_REBUILD=1 disables.
Verified live: kb_preamble grew 416 → 727 chars after wiring third
source; qwen3.5:latest synthesis (702 → 128 tokens) compresses
correctly; deepseek-v3.1-terminus diagnosis (301 → 148 tokens) is
sharper, citing architectural patterns (circuit breaker, adapter
files) instead of generic timeouts. Total cost per escalation
unchanged at ~$0.0002.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
340fca2427 |
observer: route escalation to paid OpenRouter (deepseek-v3.1-terminus)
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
ollama_cloud/qwen3-coder:480b was hitting weekly 429 quota; observer escalations were silently failing 502 with no audit row. Switched escalation cloud-call to deepseek-v3.1-terminus on paid OpenRouter: 671B reasoning specialist, $0.21 in / $0.79 out per M tokens (under the $0.85/M ceiling J set), 164K ctx. End-to-end verified: kb_preamble_chars=416, prompt 245 tokens, completion 155 tokens, ~$0.00018 per escalation. Diagnosis output is specific (cites adapter + route file), not generic. Two-stage chain holds: qwen3.5:latest compresses raw KB hits into a tight briefing, deepseek-v3.1-terminus reasons over the briefing for diagnosis. Audit `mode` field updated to direct_chat_deepseek_v3_1_terminus so downstream consumers can attribute analyses to the correct rung. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
d9bd4c9bdf |
observer: KB enrichment preamble before failure-cluster escalation
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
escalateFailureClusterToLLMTeam now calls a new buildKbPreamble() that mirrors what scrum_master_pipeline does on every per-file review: queries /vectors/pathway/bug_fingerprints + /vectors/search against the lakehouse_arch_v1 corpus, then asks local qwen3.5:latest (provider=ollama) to synthesize a tight briefing. The synthesized preamble prepends the existing escalation prompt so the cloud reviewer sees historical context the same way scrum reviewers do. Reuses existing KB primitives — no new corpora, no new endpoints, no new abstractions. Same code path scrum already exercises 3+ times per review; observer joins the same compounding loop. Audit row gains kb_preamble_chars so we can later track enrichment yield per escalation. Empty preamble (both fingerprints + matrix return nothing) → empty string, prompt unchanged. Verified: qwen3.5:latest synthesis fires for every escalation with non-empty matrix hits (gateway log: 445→72 tokens, 3.1s). Matrix retrieval correctly surfaces PRD Phase 40/44 chunks for chat_completion clusters. Pathway memory stays consistent with scrum (84→87 traces); chat_completion task_class doesn't have fingerprints yet — graceful. Local-model synthesis was J's explicit ask: compress the raw bundle before the cloud call so the briefing is actionable, not a dump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
69919d9d57 |
.archon: add lakehouse-architect-review workflow
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Demonstrates Archon → Pi → our gateway → OpenRouter end-to-end on Lakehouse code. Three Pi nodes (shape, weakness, improvement) — each fires a /v1/chat/completions call that lands a Langfuse trace AND an observer event for KB-consolidation parity with scrum. Run: ARCHON_SUPPRESS_NESTED_CLAUDE_WARNING=1 \ PI_OPENROUTER_BASE_URL=http://localhost:3100/v1 \ OPENROUTER_API_KEY=sk-anything \ archon workflow run lakehouse-architect-review --no-worktree Verified 2026-04-26 — 3 grok-4.1-fast calls, 14.6s total, observer ring delta=3, three Langfuse traces. Read-only (allowed_tools: [read]) so running it doesn't mutate the repo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
d1d97a045b |
v1: fire observer /event from /v1/chat alongside Langfuse trace
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Observer at :3800 already collects scrum + scenario events into a ring
buffer that pathway-memory + KB consolidation read from. /v1/chat now
posts a lightweight {endpoint, source:"v1.chat", input_summary,
output_summary, success, duration_ms} event there too — fire-and-forget
tokio::spawn, observer-down doesn't block the chat response.
Now any tool routed through our gateway (Pi CLI, Archon, openai SDK
clients, langchain-js) shows up in the same ring buffer the scrum loop
reads, ready for the same KB-consolidation analysis. Independent of the
existing langfuse-bridge that polls Langfuse — this path is immediate.
Verified: GET /stats shows {by_source: {v1.chat: N}} grows by 1 per
chat call, both for direct curl and for Pi CLI invocations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
540a9a27ee |
v1: accept OpenAI multimodal content shape (array-of-parts)
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Modern OpenAI clients (pi-ai, openai SDK 6.x, langchain-js, the official
agents) send `messages[].content` as an array of content parts:
`[{type:"text", text:"..."}, {type:"image_url", ...}]`. Our gateway
typed `content` as plain `String` and 422'd those calls.
Fix: `Message.content` is now `serde_json::Value` so requests
deserialize regardless of shape. `Message::text()` flattens
content-parts arrays (concat'd `text` fields, non-text parts skipped)
for places that need a plain string — Ollama prompt assembly, char
counts, the assistant's own response synthesis. `Message::new_text()`
constructs string-content messages without writing the wrapper at
each call site. Forwarders (openrouter) clone content through
verbatim so providers see exactly what the client sent.
Verified end-to-end: Pi CLI (`pi --print --provider openrouter`)
landed a clean 1902-token request through `/v1/chat/completions`,
routed to OpenRouter as `openai/gpt-oss-120b:free`, response in
1.62s, Langfuse trace `v1.chat:openrouter` recorded with provider
tag. Same path that any tool using the official openai SDK takes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
3a0b37ed93 |
v1: OpenAI-compat alias + smart provider routing — gateway is now drop-in middleware
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
/v1/chat/completions route alias (same handler as /chat) lets any tool
using the official `openai` SDK adopt the gateway via OPENAI_BASE_URL
alone — no custom provider field needed.
resolve_provider() extended:
- bare `vendor/model` (slash) → openrouter (catches x-ai/grok-4.1-fast,
moonshotai/kimi-k2, deepseek/deepseek-v4-flash, openai/gpt-oss-120b:free)
- bare vendor model names (no slash, no colon) get auto-prefixed:
gpt-* / o1-* / o3-* / o4-* → openai/<name> (OpenRouter form)
claude-* → anthropic/<name>
grok-* → x-ai/<name>
Then routed to openrouter. Ollama models (with colon, no slash) keep
default routing. Tools like pi-ai validate against an OpenAI-style
catalog and send bare names — this lets them flow through cleanly.
Verified end-to-end:
- curl POST /v1/chat/completions {model: "gpt-4o-mini", ...} → 200,
routed to openrouter as openai/gpt-4o-mini
- openai SDK with baseURL=http://localhost:3100/v1 → 3 model variants all
succeed (openai/gpt-4o-mini, gpt-4o-mini, x-ai/grok-4.1-fast)
- Langfuse traces fire automatically on every call
(v1.chat:openrouter, provider tagged in metadata)
scripts/mode_pass5_variance_paid.ts gains LH_CONDITIONS env so subset
runs (e.g. just isolation vs composed) take half the latency.
Archon-on-Lakehouse integration: gateway side is done. Pi-ai's
openai-responses backend uses /v1/responses (not /chat/completions) and
its openrouter backend appears to bail in client-side validation before
sending. Patching Pi locally to override baseUrl works for arch but the
harness still rejects — needs more work in a follow-up. Direct openai
SDK path (langchain-js / agents / patched Pi) works today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
2dbc8dbc83 |
v1/mode: model-aware enrichment downgrade + 3 corpora + variance harness
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Pass 5 (5 reps × 4 conditions × 1 file on grok-4.1-fast) showed composing matrix corpora is anti-additive on strong models — composed lakehouse_arch + symbols LOST 5/5 head-to-head vs codereview_isolation (Δ −1.8 grounded findings, p=0.031). Default flips to isolation; matrix path now auto- downgrades when the resolved model is strong. Mode runner: - matrix_corpus is Vec<String> (string OR array via deserialize_string_or_vec) - top_k=6 from each corpus, merge by score, take top 8 globally - chunk tag prefers doc_id over source so reviewer sees [adr:009] vs [lakehouse_arch] - is_weak_model() gate auto-downgrades codereview_lakehouse → codereview_isolation for strong models (default-strong; weak = :free suffix or local last-resort) - LH_FORCE_FULL_ENRICHMENT=1 bypasses for diagnostic runs - EnrichmentSources.downgraded_from records when the gate fires Three corpora indexed via /vectors/index (5849 chunks total): - lakehouse_arch_v1 — ADRs + phases + PRD + scrum spec (93 docs, 2119 chunks) - scrum_findings_v1 — past scrum_reviews.jsonl (168 docs, 1260 chunks; EXCLUDED from defaults — 24% out-of-bounds line citations from cross-file drift) - lakehouse_symbols_v1 — regex-extracted pub items + /// docs (656 docs, 2470 chunks) Experiment infra: - scripts/build_*_corpus.ts — re-runnable when source content changes - scripts/mode_pass5_variance_paid.ts — N reps × M conditions on one file - scripts/mode_pass5_summarize.ts — mean ± σ + head-to-head, parser handles numbered + path-with-line + path-with-symbol finding tables - scripts/mode_compare.ts — groups by mode|corpus when sweeps span corpora - scripts/mode_experiment.ts — default model bumped to x-ai/grok-4.1-fast, --corpus flag for per-call override Decisions + open follow-ups: docs/MODE_RUNNER_TUNING_PLAN.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
56bf30cfd8 |
v1/mode: override knobs + staffing native runner + pass 2/3/4 harnesses
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Setup for the corpus-tightening experiment sweep (J 2026-04-26 — "now
is the only cheap window before the corpus gets large and refactoring
costs go up").
Override params on /v1/mode/execute (additive — old callers unaffected):
force_matrix_corpus — Pass 2: try alternate corpora per call
force_relevance_threshold — Pass 2: sweep filter strictness
force_temperature — Pass 3: variance test
New native mode `staffing_inference_lakehouse` (Pass 4):
- Same composer architecture as codereview_lakehouse
- Staffing framing: coordinator producing fillable|contingent|
unfillable verdict + ranked candidate list with playbook citations
- matrix_corpus = workers_500k_v8
- Validates that modes-as-prompt-molders generalizes beyond code
- Framing explicitly says "do NOT fabricate workers" — the staffing
analog of the lakehouse mode's symbol-grounding requirement
Three sweep harnesses:
scripts/mode_pass2_corpus_sweep.ts — 4 corpora × 4 thresholds × 5 files
scripts/mode_pass3_variance.ts — 3 files × 3 temps × 5 reps
scripts/mode_pass4_staffing.ts — 5 fill requests through staffing mode
Each appends per-call rows to data/_kb/mode_experiments.jsonl which
mode_compare.ts already aggregates with grounding column.
Pass 1 (10 files × 5 modes broad sweep) currently running via the
existing scripts/mode_experiment.ts — gateway restart deferred until
it completes so the new override knobs aren't enabled mid-experiment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
52bb216c2d |
mode_compare: grounding check + control flag + emoji-tolerant section detection
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Three fixes after the playbook_only confabulation surfaced in 2026-04-26 experiment (8 'findings' on a 333-line file all citing lines 378-945 — fully fabricated from pathway-memory pattern names). (1) Aggregator regex bug — section detection failed on emoji-prefixed markdown headers like `## 🔎 Ranked Findings`. The original regex required word chars right after #{1,3}\s+, so the patches table header `## 🛠️ Concrete Patch Suggestions` was never recognized as a stop boundary, double-counting every finding. Fix: allow non-letter chars (emoji/space) between # and the keyword. (2) Grounding check — for each finding row in the response, extract backtick-quoted symbols + cited line numbers; verify symbols exist in the actual focus file and lines fall within EOF. Computes grounded/total ratio per mode. Surfaces 'OOB' (out-of-bounds) count explicitly so confabulation is visible at a glance. Confirms what hand-grading found: codereview_playbook_only's 8 findings on service.rs were 1/8 grounded with 7 OOB. (3) Control mode tagging — codereview_null and codereview_playbook_only are designed as falsifiers (baseline / lossy ceiling) and their numerical wins should never be read as recommendations. Output marks them with ⚗ glyph + warning footer. Per-mode aggregate is now sorted by groundedness, not raw count. Per-mode-vs-lakehouse comparison uses grounded findings, not raw — so confabulation can no longer score a "win". Updated SCRUM_MASTER_SPEC.md with refactor timeline pointing at the 2026-04-25/26 commits (observer fix, relevance filter, retire wire, mode router, enrichment runner, parameterized experiment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
7c47734287 |
v1/mode: parameterized runner + 5 enrichment-experiment modes
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
J's directive (2026-04-26): "Create different modes so we can really
dial in the architecture before it gets further along — pinpoint the
failures and strengths equally so I know what direction to go in.
Loop theater happens when we don't pinpoint the most accurate path."
Refactored execute() to switch on mode name → EnrichmentFlags preset.
Five native modes designed as deliberate experiments — each isolates
one architectural axis so the comparison matrix reads off what's
doing work vs what's adding latency for nothing:
codereview_lakehouse — all enrichment on (ceiling)
codereview_null — raw file + generic prompt (baseline)
codereview_isolation — file + pathway only (no matrix)
codereview_matrix_only — file + matrix only (no pathway)
codereview_playbook_only — pathway only, NO file content (lossy ceiling)
Each call appends a row to data/_kb/mode_experiments.jsonl with full
sources + response. LH_MODE_LOG_OFF=1 to suppress.
scripts/mode_experiment.ts — sweeps files × modes serially, prints
live progress with per-call enrichment stats. Defaults to OpenRouter
free model so cloud quota doesn't gate experiments.
scripts/mode_compare.ts — reads the JSONL, outputs per-file matrix
+ per-mode aggregate + mode-vs-baseline win/loss with avg finding
delta. Heuristic finding-count from markdown table rows; pathway
citation count from preamble references.
scrum_master_pipeline.ts gets a mode-runner fast path gated by
LH_USE_MODE_RUNNER=1: try /v1/mode/execute first, fall through to
the existing ladder if response < LH_MODE_MIN_CHARS (default 2000)
or anything errors. Off by default until A/B-validated.
First experiment results (2 files × 5 modes via gpt-oss-120b:free):
- codereview_null produces 12.6KB response with ZERO findings
(proves adversarial framing is load-bearing)
- codereview_playbook_only produces MORE findings than lakehouse
on average (12 vs 9) at 73% the latency — pathway memory is
the dominant signal driver
- codereview_matrix_only underperforms isolation by ~0.5 findings
while costing the same latency — matrix corpus likely
underperforming for scrum_review task class
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
86f63a083d |
v1/mode: codereview_lakehouse native runner — modes are prompt-molders
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
J's framing (2026-04-26): "Modes are how you ask ONCE and get BETTER
information — they mold the data, hyperfocus the prompt on this
codebase's needs, so the model gets it right the first time without
the cascading retry ladder."
Built the first concrete native enrichment runner (codereview_lakehouse)
that composes every context primitive the gateway exposes:
1. Focus file content (read from disk OR caller-supplied)
2. Pathway memory bug_fingerprints for this file area (ADR-021
preamble — "📚 BUGS PREVIOUSLY FOUND IN THIS FILE AREA")
3. Matrix corpus search via the task_class's matrix_corpus
4. Relevance filter (observer /relevance) drops adjacency pollution
5. Assembles ONE precise prompt with system framing
6. Single call to /v1/chat with the recommended model
POST /v1/mode/execute dispatches. Native mode → runs the composer.
Non-native mode → 501 NOT_IMPLEMENTED with hint (proxy to LLM Team
/api/run is queued).
Provider hint logic auto-routes by model name shape:
- vendor/model[:tag] → openrouter
- kimi-*/qwen3-coder*/deepseek-v*/mistral-large* → ollama_cloud
- everything else → local ollama
Live test against crates/queryd/src/delta.rs (10593 bytes, 10
historical bug fingerprints, 2 matrix chunks dropped by relevance):
- enriched_chars: 12876
- response_chars: 16346 (14 findings with confidence percentages)
- Model literally cited the pathway memory preamble in finding #7
- One call to free-tier gpt-oss:120b produced what previously
required the 9-rung escalation ladder
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
d277efbfd2 |
v1/mode: task_class → mode/model router (decision-only, phase 1)
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
HANDOVER §queued (2026-04-25): "Mode router — port LLM Team multi-model
patterns. Pick the right TOOL/MODE for each task class via the matrix,
not cascade through models."
Two-stage architecture:
1. Decision (POST /v1/mode) — pure recommendation, no execution.
Returns {mode, model, decision: {source, fallbacks, matrix_corpus,
notes}} so callers see WHY this mode was picked.
2. Execution (future POST /v1/mode/execute) — proxy to LLM Team
/api/run for modes not yet ported to native Rust runners. Not
wired in this phase.
Splitting decision from execution lets us A/B-test the routing logic
without committing to running every recommendation. The decision
function is pure enough for exhaustive unit tests (3 added).
config/modes.toml — initial map for 5 task_classes (scrum_review,
contract_analysis, staffing_inference, fact_extract, doc_drift_check)
+ a default. matrix_corpus per task is reserved for the future
matrix-informed routing pass.
VALID_MODES list (24 modes) is kept in sync manually with LLM Team's
/api/run handler at /root/llm_team_ui.py:10581. Adding a mode here
without adding it upstream returns 400 from a future proxy.
GET /v1/mode/list — operator introspection so a UI can render the
registry table without re-parsing TOML.
Live-tested: 5 task classes match, unknown classes fall through to
default, force_mode override works + validates, bogus modes return
400 with the valid_modes list.
Updates reference_llm_team_modes.md memory — earlier note claiming
"only extract is registered" was wrong (all 25 are registered).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
626f18d491 |
pathway_memory: audit-consensus → retire wire
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
When observer's hand-review explicitly rejects the output of a
hot-swap-recommended model, the matrix's recommendation was wrong
for this context. Auto-retire the trace so future agents don't
get the same poisoned recommendation in their preamble.
crates/vectord/src/pathway_memory.rs — add `trace_uid` to
HotSwapCandidate response and populate from the matched trace.
This gives consumers single-trace precision for /pathway/retire.
tests/real-world/scrum_master_pipeline.ts:
- HotSwapCandidate interface gains trace_uid
- new retirePathwayTrace() helper (fire-and-forget, fall-open)
- in the obsVerdict reject branch: if hotSwap was active AND
the rejected model is the hot-swap-recommended one AND
observer confidence ≥0.7, fire retire and null hotSwap so
post-loop replay bookkeeping doesn't double-process.
- hotSwap declared `let` (was const) so it can be nulled
Cycle verdicts ("needs different angle") don't trigger retire —
only outright rejects do. Confidence gate avoids retiring on
heuristic-fallback verdicts that come back without a confidence
number. Closes the "audit-consensus → retire" item from
HANDOVER.md.
Live-tested: insert synthetic trace → /pathway/retire by trace_uid
→ retired counter 1 → 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
e1ef185868 |
docs: add MATRIX_AGENT_HANDOVER notes + cross-link from SCRUM_MASTER_SPEC
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
The matrix-agent-validated repo split + Ansible deploy pipeline are otherwise undocumented in this repo. This new doc explains: - the relationship between lakehouse and matrix-agent-validated - where the playbook lives and what it provisions - the critical distinction: matrix-test (10.111.129.50 Incus container) is the REAL destination, while 192.168.1.145 is a smoke-test VPS only (partial deploy, no services, do not treat as production) - what landed today (observer fix, HANDOVER.md render, relevance filter) Added a cross-link block at the top of SCRUM_MASTER_SPEC.md so the canonical scrum handoff doc points at the new MATRIX_AGENT_HANDOVER doc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
0115a60072 |
observer: add /relevance heuristic filter for adjacency pollution
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Matrix retrieval often surfaces high-cosine chunks that are about
symbols the focus file IMPORTS but doesn't define. The reviewer LLM
then hallucinates those imported-crate internals as in-file content
("I see main.rs does X" when X lives in queryd::context).
mcp-server/relevance.ts — pure scorer with five signals:
path_match +1.0 chunk source/doc_id encodes focus path
defined_match +0.6 chunk text mentions focus.defined_symbols
token_overlap +0.4 jaccard of non-stopword tokens
prefix_match +0.3 shared first-2-segment prefix
import_only -0.5 mentions only imported symbols (pollution)
Default threshold 0.3 — tuned empirically on the gateway/main.rs case.
Also fixes a regex bug in the import extractor: the character class
was lowercase-only, so `use catalogd::Registry;` silently never
matched (regex backed off when it hit the uppercase R). Caught by
the test suite.
observer.ts — POST /relevance endpoint wraps filterChunks().
scrum_master_pipeline.ts — fetchMatrixContext gains optional
focusContent param; calls /relevance after collecting allHits and
before sort+top. Opt-out via LH_RELEVANCE_FILTER=0; threshold via
LH_RELEVANCE_THRESHOLD. Fall-open on observer failure.
9 unit tests, all green. Live probe on real shape correctly drops
a 0.7-cosine adjacency-pollution chunk while keeping in-focus hits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
54689d523c |
observer: fix gateway health probe — text/plain not JSON
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Same bug as matrix-agent-validated 5db0c58. observer.ts:645 did fetch().then(r => r.json()) against /health which returns text/plain "lakehouse ok". r.json() throws on non-JSON, .catch swallows to null, observer exits assuming gateway down. With systemd Restart=on-failure this crash-loops every 5s — confirmed live on matrix-test box today. Fix: r.ok ? r.text() : null. Same shape, accepts the actual content type. Sealed in pathway_memory as TypeConfusion:fetch-health-json. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
779158a09b |
scripts: chicago analyzer field-name fixes + vectorize sanitizer hardening
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Two small fixes surfaced during smoke testing: analyze_chicago_contracts.ts: permit field is contact_1_name not contact_1; reported_cost is integer-string. Fixed filter (was rejecting all 2853 permits) and contractor extraction (was empty). vectorize_raw_corpus.ts: sanitize() expanded to strip control chars + ALL backslashes (kills incomplete \uXXXX escapes) + UTF-16 surrogates (unpaired surrogates from emoji split by truncate boundary). Llm_team response cache had docs with all three pollution shapes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
6ac7f61819 |
pathway_memory: Mem0 versioning + deletion (upsert/revise/retire/history)
Per J 2026-04-25: pathway_memory was append-only — every agent run added
a new trace, bad/failed runs polluted the matrix forever, no notion of
"this is the canonical evolved playbook." Ported playbook_memory's
Phase 25/27 patterns into pathway_memory so the agent loop's matrix
converges on best-known approaches per task class instead of bloating.
Fields added to PathwayTrace (all #[serde(default)] for back-compat):
- trace_uid: stable UUID per individual trace within a bucket
- version: u32 default 1
- parent_trace_uid, superseded_at, superseded_by_trace_uid
- retirement_reason (paired with existing retired:bool)
Methods added to PathwayMemory:
- upsert(trace) → PathwayUpsertOutcome {Added|Updated|Noop}
Workflow-fingerprint dedup: ladder_attempts + final_verdict hash.
Identical workflow → bumps existing replay_count instead of duplicating.
- revise(parent_uid, new_trace) → PathwayReviseOutcome
Chains versions; rejects retired or already-superseded parents.
- retire(trace_uid, reason) → bool
Marks specific trace retired with reason. Idempotent.
- history(trace_uid) → Vec<PathwayTrace>
Walks parent_trace_uid back to root, then superseded_by forward to tip.
Cycle-safe via visited set.
Retrieval gates updated:
- query_hot_swap skips superseded_at.is_some()
- bug_fingerprints_for skips both retired AND superseded
HTTP endpoints in service.rs:
- POST /vectors/pathway/upsert
- POST /vectors/pathway/retire
- POST /vectors/pathway/revise
- GET /vectors/pathway/history/{trace_uid}
scripts/seal_agent_playbook.ts switched insert→upsert + accepts SESSION_DIR
arg so it can seal any archived session, not just iter4.
Verified live (4/4 ops):
- UPSERT first run: Added trace_uid 542ae53f
- UPSERT identical: Updated, replay_count bumped 0→1 (no duplicate)
- REVISE 542ae53f→87a70a61: parent stamped superseded_at, v2 created
- HISTORY of v2: chain_len=2, v1 superseded, v2 tip
- RETIRE iter-6 broken trace: retired=true, retirement_reason preserved
- pathway_memory.stats: total=79, retired=1, reuse_rate=0.0127
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
ed83754f20 |
raw-corpus dump + vectorization + chicago contract inference pipeline
Three new pieces, executed in order:
scripts/dump_raw_corpus.sh
- One-shot bash that creates MinIO bucket `raw` and uploads all
testing corpora as a persistent immutable test set. 365 MB total
across 5 prefixes (chicago, entities, sec, staffing, llm_team)
+ MANIFEST.json. Sources: workers_500k.parquet (309 MB),
resumes.parquet, entities.jsonl, sec_company_tickers.json,
Chicago permits last 30d (2,853 records, 5.4 MB), 9 LLM Team
Postgres tables dumped via row_to_json.
scripts/vectorize_raw_corpus.ts
- Bun script that fetches each raw-bucket source via mc, runs a
source-specific extractor into {id, text} docs, posts to
/vectors/index, polls job to completion. Verified results:
chicago_permits_v1: 3,420 chunks
entity_brief_v1: 634 chunks
sec_tickers_v1: 10,341 chunks (after extractor fix for
wrapped {rows: {...}} JSON shape)
llm_team_runs_v1: in flight, 19K+ chunks
llm_team_response_cache_v1: queued
scripts/analyze_chicago_contracts.ts
- Real inference pipeline that picks N high-cost permits with
named contractors from the raw bucket, queries all 6 contract-
analysis corpora in parallel via /vectors/search, builds a
MATRIX CONTEXT preamble, calls Grok 4.1 fast for structured
staffing analysis, hand-reviews each via observer /review,
appends to data/_kb/contract_analyses.jsonl.
tests/real-world/scrum_master_pipeline.ts
- MATRIX_CORPORA_FOR_TASK extended with two new task classes:
contract_analysis (chicago + entity_brief + sec + llm_team_runs
+ llm_team_response_cache + distilled_procedural)
staffing_inference (workers_500k_v8 + entity_brief + chicago
+ llm_team_runs + distilled_procedural)
scrum_review unchanged.
This is the first time the matrix architecture operates on real
ingested data instead of code-review smoke tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
a496ced848 |
scrum: unified matrix retriever — pull from ALL relevant KB corpora, not just pathway memory
Per J 2026-04-25 architectural correction: matrix index is the vector
indexing layer for the WHOLE knowledge base (distilled facts, procedures,
config hints, team runs, playbooks, pathway successes), not a single
narrow store. Built fetchMatrixContext(query, taskClass, filePath) that:
- Queries multiple persistent vector indexes in parallel via /vectors/search
- Collects hits per corpus + score + doc_id + 400-char excerpt
- Pulls pathway successes via existing helper, mapped to MatrixHit shape
- Sorts by score across corpora, returns top-N (default 8)
- Reports per-corpus hit counts + errors for transparency
Per-task-class corpus list (MATRIX_CORPORA_FOR_TASK):
scrum_review → distilled_factual, distilled_procedural,
distilled_config_hint, kb_team_runs_v1
(staffing data deliberately excluded — not relevant to code review)
Probed live: distilled_config_hint top hit = 0.52, distilled_procedural
top = 0.49, kb_team_runs top = 0.59. Real signal across corpora.
Replaces the narrow proven-approaches preamble with a unified
MATRIX-INDEXED CONTEXT preamble tagged with source_corpus per chunk
so the model knows what kind of context it's seeing.
LH_SCRUM_MATRIX_RETRIEVE=0 still disables for A/B testing.
Future: promote to a Rust /v1/matrix endpoint once corpora list and
ranking logic stabilize. For now TS lets us iterate fast against the
live matrix without gateway restarts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
d187bcd8ac |
scrum: stop cascading models on quality issues — single-model retry with enrichment
Architectural correction (J 2026-04-25): The 9-rung ladder was treating cascade as the strategy. That's wrong. ONE model handles the work, with same-model retries using enriched context. Cycle to a different model ONLY on PROVIDER errors (network / auth / 5xx) — never on quality issues, because quality issues mean the context needs more enrichment, not a different model. Changes: - LADDER shrunk from 11 entries to 3 (Grok 4.1 fast primary, DeepSeek V4 flash + Qwen3-235B as provider-error fallbacks). Removed Kimi K2.6, Gemini 2.5 flash, all Ollama Cloud rungs, OR free-tier rungs, local qwen3.5 — none were doing the work, all wasted attempts. They remain available as routable tools for the future mode router. - Loop restructured: separate `modelIdx` from attempt counter. Provider error → modelIdx++ (advance fallback). Observer reject / cycle / thin response → retry SAME model with rejection notes feeding into the `learning` preamble; advance fallback only after MAX_QUALITY_RETRIES (default 2) exhausted on the current model. - LH_SCRUM_MAX_QUALITY_RETRIES env to tune the per-model retry cap. What this preserves: - Tree-split (treeSplitFile) is still the ONE legitimate model-switch trigger for context-overflow, but even it just re-runs the same model against smaller chunks. - Pathway memory preamble still fires. - Hot-swap reorder still applies — when a recommended model maps to the new shorter ladder. Future direction (J 2026-04-25 note): the LLM Team multi-model modes in /root/llm_team_ui.py are a REFERENCE PATTERN for a mode router we will build INSIDE this gateway. Mimic the patterns, don't modify the LLM Team UI itself. The mode router will pick the right approach for each task class via the matrix index, not cascade through models. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
6432465e2c |
autonomous_loop: stop clobbering applier model/provider defaults
Found by running: the loop was setting LH_APPLIER_MODEL=qwen3-coder:480b explicitly via env, which clobbered the applier's NEW default of x-ai/grok-4.1-fast on openrouter. Result: applier kept hitting the throttled ollama_cloud account and producing zero patches every iter. Now LOOP_APPLIER_MODEL and LOOP_APPLIER_PROVIDER are optional overrides; when unset, scrum_applier.ts uses its own defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
4ac56564c0 |
scrum + applier + observer: switch to paid OpenRouter ladder, add Kimi K2.6 + Gemini 2.5
Ollama Cloud was throttled across all 6 cloud rungs in iters 1-9, which forced the loop into 0-review iterations even though the architecture was sound. Swapping to paid OpenRouter unblocks the test path. Ladder changes (top-of-ladder paid models, all under $0.85/M either side): - moonshotai/kimi-k2.6 ($0.74/$4.66, 256K) — capped at 25/hr - x-ai/grok-4.1-fast ($0.20/$0.50, 2M) — primary general - google/gemini-2.5-flash ($0.30/$2.50, 1M) — Google reasoning - deepseek/deepseek-v4-flash ($0.14/$0.28, 1M) — cheap workhorse - qwen/qwen3-235b-a22b-2507 ($0.07/$0.10, 262K) — cheapest big Existing rungs (Ollama Cloud + free OR + local qwen3.5) kept as fallback. Per-model rate limiter (MODEL_RATE_LIMITS in scrum_master_pipeline.ts): - Persists call timestamps to data/_kb/rate_limit_calls.jsonl so caps survive process restarts (autonomous loop spawns a fresh subprocess per iteration; without persistence each iter would reset) - O(1) writes, prune-on-read for the rolling 1h window - Capped models log "SKIP (rate-limited: cap N/hr reached)" and the ladder cycles to the next rung - J directive 2026-04-25: 25/hr on Kimi K2.6 to bound output cost Observer hand-review cloud tier swapped from ollama_cloud/qwen3-coder:480b to openrouter/x-ai/grok-4.1-fast — proven to emit precise semantic verdicts (named "AccessControl::can_access() doesn't exist" specifically in 2026-04-25 tests instead of the heuristic fallback). Applier patch emitter swapped from ollama_cloud/qwen3-coder:480b to openrouter/x-ai/grok-4.1-fast (default; LH_APPLIER_MODEL + LH_APPLIER_PROVIDER override). This was the third LLM call we missed — without it, observer accepts a review but applier never produces patches because its emitter was still hitting the throttled account. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
e79e51ed70 |
tests: autonomous_loop.ts — goal-driven scrum + applier retry harness
Wraps tests/real-world/scrum_master_pipeline.ts and scrum_applier.ts in a single autonomous loop that runs scrum → applier --commit → optional git push, observes per-iteration outcomes via observer /event, journals to data/_kb/autonomous_loops.jsonl. Stops when 2 consecutive iters land zero commits OR LOOP_MAX_ITERS reached. Env knobs: LOOP_TARGETS — comma-sep paths, default 3 high-traffic Lakehouse files LOOP_MAX_ITERS — default 3 LOOP_PUSH=1 — push branch after each commit-landing iter LOOP_BRANCH — default scrum/auto-apply-19814 (refuses to run elsewhere) LOOP_MIN_CONF — applier min confidence (default 85) LOOP_APPLIER_MODEL — default qwen3-coder:480b Causality preserved: targets pass through to LH_APPLIER_FILES so applier patches what scrum just reviewed (vs picking from global review history). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
3f166a5558 |
scrum + observer: hand-review wire — judgment moved out of the inner loop
Pre-2026-04-25 the scrum_master applied a hardcoded grounding-rate gate
inline. That baked policy into the wrong layer — semantic judgment about
whether a review is grounded belongs in the observer (which has Langfuse
traces, sees every response across the system, and can call cloud LLMs
for real evaluation). Scrum should report DATA, observer DECIDES.
What landed:
- scrum_master_pipeline.ts: removed the inline grounding-pct threshold;
every accepted candidate now POSTs to observer's /review endpoint with
{response, source_content, grounding_stats, model, attempt}. Observer
returns {verdict: accept|reject|cycle, confidence, notes}. On observer
failure, scrum falls open to accept (observer is policy, not blocker).
- mcp-server/observer.ts: new POST /review endpoint with two-tier
evaluator. Tier 1: cloud LLM (qwen3-coder:480b at temp=0) hand-reviews
with full context — response + source excerpt + grounding stats — and
emits structured verdict JSON. Tier 2: deterministic heuristic over
grounding pct + total quotes when cloud throttles, marked source:
"heuristic" so consumers can tune it later by comparing against cloud.
- Every verdict persists to data/_kb/observer_reviews.jsonl with full
input snapshot so cloud vs heuristic can be A/B compared once cloud
quota refreshes.
Verified end-to-end: smoke loop iter 1 — observer returned `cycle` on
21% grounding (cycled to next rung), `reject` on 17% (gave up). Iter 2
— `reject` on 12% and 14%. Both UNRESOLVED with honest signal instead
of polluting pathway memory with hallucinated patterns.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
c90a509f49 |
applier: LH_APPLIER_FILES env to constrain to current-iter targets
Without this, the applier loaded the latest 34 reviews and patched the highest-confidence file from history — which is meaningless when called from the autonomous loop where the intent is "review file X this iter, patch file X this iter." Now the loop passes its targets through and the applier filters eligible reviews accordingly. Causality is restored: scrum reviews file X → applier patches file X. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
9ecc5848fa |
scrum: blind-response guard + anchor-grounding post-verifier
Two signal-quality fixes for the scrum loop: 1. isBlindResponse() — detects models that emit structurally-valid review JSON containing "no source code visible / cannot verify" even when source WAS supplied. Rejects so the ladder cycles to the next rung instead of accepting the blind hallucination. 2. verifyAnchorGrounding() + appendGroundingFooter() — post-process verifier that extracts every backtick-quoted snippet from the review and checks it against the original source content. Appends a grounding footer reporting grounded vs ungrounded counts so humans can audit hallucination rate at a glance. Born from the iter where llm_team_ui.py review came back with 6/10 findings hallucinated (invented render_template_string calls, fabricated logger.exception sites, made-up SHA-256 hashing). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
b843a23433 |
mcp: contractor entity-brief drill-down + mobile UX pass
Adds /contractor page route plus /intelligence/contractor_profile endpoint that fans out across OSHA, ticker, history, parent_link, federal contracts, debarment, NLRB, ILSOS, news, diversity certs, BLS macro — single per-contractor portfolio view across every wired source. search.html: mobile responsive layout, fixed bottom dock with horizontal scroll-snap, legacy bridge row stacking, viewport overflow guards. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
a6c83b03e5 |
chore: sync Cargo.lock — toml dep for phase-42 rule loader
Pairs retroactively with
|
||
|
rootandClaude Opus 4.7
|
858954975b |
Staffing Co-Pilot UI — architecture-first enrichments + shift clock
lakehouse/auditor 2 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
J's direction: the dashboard was explanatory but not *actionable* as
a staffing-matrix console. Refactor so the architecture claims from
docs/PRD.md surface as operational signals on every contract card.
Backend (mcp-server/index.ts):
+ GET|POST /intelligence/arch_signals — probes live substrate health
so the dashboard shows instant-search latency, index shape,
playbook-memory entries, and pathway-memory (ADR-021) trace count.
Fires one fresh /vectors/hybrid probe against workers_500k_v1 so
the "instant search" number on screen is live, not cached.
* /intelligence/permit_contracts now times every hybrid call per
contract and returns search_latency_ms, so the card can display
the per-query latency pill (⚡ 342ms).
+ Per-contract computed fields returned from the backend:
search_latency_ms — real /vectors/hybrid duration
fill_probability — base_pct (by pool_size×count ratio)
+ curve [d0, d3, d7, d14, d21, d30]
with cumulative fill% per bucket
economics — avg_pay_rate, gross_revenue,
gross_margin, margin_pct,
payout_window_days [30, 45],
over_bill_count,
over_bill_pool_margin_at_risk
shifts_needed — 1st/2nd/3rd/4th inferred from
permit work_type + description regex
* Pre-existing dangling-brace bug in api() fixed (the `activeTrace`
logging block had been misplaced at module scope, referencing
variables that only existed inside the function). Restart was
failing with "Unexpected }" at line 76. Moved tracing inside the
try block where parsed/path/body/ms are in scope.
Frontend (mcp-server/search.html):
+ Top "Substrate Signals" section — 4 live tiles (instant search,
index, playbook memory, pathway matrix). Color-codes latency
(green <100ms, amber <500ms, red otherwise).
+ "24/7 Shift Coverage" section — SVG 24-hour clock with 4 colored
shift arcs (1st/2nd/3rd/4th), current-time needle, center label
showing the live shift, per-shift contract count tiles beside.
4th shift assumes weekend/split; handles 3rd-shift wrap across
midnight by splitting into two arcs.
+ Per-card architecture pills: instant-search latency, SQL-filter
pool-size with k=200 boost note, shift requirements.
+ Per-card fill-probability horizontal stacked bar with day
markers (d0/d3/d7/d14/d21/d30) and per-bucket segment shading
(green → amber → orange → red as time decays).
+ Per-card economics 4-tile grid: Est. Revenue, Est. Margin (with
% colored by health), Payout Window (30–45d standard), Over-Bill
Pool count + margin at risk.
Architecture smoke test (tests/architecture_smoke.ts, earlier commit)
still green: 11/11 pass including the new /intelligence/arch_signals
+ permit_contracts enrichments.
J specifically wanted: "shoot for the stars · hyperfocus · our
architecture is better because it self-regulates, uses hot-swap,
pulls from real data, and shows instant searches from clever
indexing." Every one of those is now a specific visible signal on
the page, not prose in the README.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
4a94da2d41 |
tests/architecture_smoke — PRD-invariant probe against 500k workers
J's reset: I'd been iterating on pipeline internals without a
driver. The PRD says staffing is the REFERENCE consumer, not the
domain driver — the architecture is the thing. This test makes
that explicit.
8 sections exercise the PRD §Shared requirements against live
production-shaped data (500k workers parquet, 50k-chunk vector
index, 768d nomic embeddings):
1. preconditions — gateway + sidecar alive
2. catalog lookup — workers_500k resolves to 500000 rows
3. SQL at scale — count(*) + geo filter on 500k rows
4. vector search — /vectors/search returns top-k
5. hybrid SQL+vector — /vectors/hybrid with sql_filter
6. playbook_memory — /vectors/playbook_memory/stats
7. pathway_memory — ADR-021 stats + bug_fingerprints
8. truth gate — DROP TABLE blocked with 403
No cloud calls. Completes in ~5 seconds. Exits non-zero on any
failure; failure messages print "these are the next things to fix."
First-run measurements against current code:
- 500k COUNT(*) = 22ms, OH-filtered = 20ms (invariant met)
- vector search p=368ms on 10-NN
- hybrid p=4662ms, returned 0 Toledo-OH hits (two signals worth
investigating: the latency AND the empty result)
- playbook_memory = 0 entries (rebuild never fired since boot)
The 11/11 pass means the substrate's contract is intact. The
measurements tell us WHERE to look next, not what to speculate.
Going forward: this script is the canary. Run it after every
substantive change. If a section flips from pass to fail, that IS
the regression; roll back or fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
4087dde780 |
execution_loop: update stale test assertion to match current prompt format
lakehouse/auditor 2 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Pre-existing failure I've been noting across this session — `executor_prompt_includes_surfaced_candidates` expected the substring "W-1 Alice Smith" but the prompt format was intentionally changed (probably in a Phase 38/39 commit) to separate doc_id from name so the executor doesn't conflate `doc_id` (vector-index key) with `workers_500k.worker_id` (integer PK). Current prompt format (line 1178 in build_executor_prompt): - name="Alice Smith" city="Toledo" state="OH" (vector doc_id=W-1) The prompt body explicitly instructs the model NOT to conflate the two IDs — the format separation is the mechanism enforcing that instruction. The OLD test assertion predated that separation. Assertion now checks the semantic contract (both tokens present, any order) instead of the exact old concatenation. Workspace test result after this commit: 343 passed, 0 failed, 0 warnings (both lib + tests). This is the last stale-test hole from the phase-audit sweep — it popped up during the 41-commit push but I was leaving it as pre-existing-unrelated. J called it: sitting broken for hours is worse than a one-line assertion update. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
951c6014ec |
gateway: boot-time probe of truth/ file-backed rules
Phase 42 PRD deliverable
|
||
|
rootandClaude Opus 4.7
|
fee094f653 |
gateway/access: wire get_role + is_enabled into HTTP routes
Two of the four #[allow(dead_code)] methods in access.rs were dead
because nothing exposed them externally. access.rs itself is fine —
list_roles, set_role, can_access all have live callers. But get_role
and is_enabled were shaped as public API with no surface to call
them through.
Fix adds two small routes under /access (where the rest of the
access surface lives):
GET /access/roles/{agent}
Calls AccessControl::get_role(agent). Returns 404 with a clear
message when the agent isn't registered so clients distinguish
"unknown agent" from "access denied." Part of P13-001
(ops tooling needs per-agent role introspection).
GET /access/enabled
Calls AccessControl::is_enabled(). Returns {"enabled": bool}.
Dashboards + ops tooling poll this to confirm auth posture of
the running gateway — distinct from /health which answers
"is the process up" vs "is access enforcement on."
#[allow(dead_code)] removed from both methods — they have live
callers now via these routes, the linter will enforce that going
forward.
Still #[allow(dead_code)] on access.rs: masked_fields + log_query.
Both need cross-crate wiring:
- masked_fields wants the agent's role + query response columns,
called in response shaping (queryd returning to gateway path)
- log_query wants post-execution audit, called after every SQL
execution on the gateway boundary
Both are P13-001 phase 2 work — need AgentIdentity plumbed through
the /query nested router before the call sites make sense. Flagged
for follow-up.
Workspace warnings still at 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
91a38dc20b |
vectord/index_registry: add last_used + build_signature (scrum iter 11)
Scrum iter 11 on crates/vectord/src/index_registry.rs flagged two
concrete field gaps (90% confidence). Both were tagged UnitMismatch
/ missing-invariant.
IndexMeta gains two Optional fields:
last_used: Option<DateTime<Utc>>
PRD 11.3 — when this index was last searched against. Callers
were reading created_at as a liveness proxy, which conflated
"built" with "used." IndexRegistry::touch_used(name) stamps the
field on every hit; incremental re-embed can now skip cold
indexes without misattributing "fresh build" to "recent use."
build_signature: Option<String>
PRD 11.3 — stable SHA-256 of (sorted source files + chunk_size
+ overlap + model_version). compute_build_signature() in the
same module is deterministic: file-order-invariant, changes on
chunk param, changes on model version. Lets incremental re-embed
answer "has anything changed since last build?" without scanning
the source Parquet.
Both fields are #[serde(default)] — the ~40 existing .json meta
files under vectors/meta/ load unchanged. Backward-compat verified
by the explicit `index_meta_deserializes_without_new_fields_backcompat`
test.
7 new tests:
- build_signature_is_deterministic
- build_signature_order_invariant (sorted internally)
- build_signature_changes_on_chunk_param
- build_signature_changes_on_model_version
- touch_used_updates_last_used
- touch_used_is_noop_on_missing_index
- index_meta_deserializes_without_new_fields_backcompat
Call-site fixes: crates/vectord/src/refresh.rs:294 and
crates/vectord/src/service.rs:244 both construct IndexMeta with
fully-literal init, default the new fields to None. One
indentation cleanup on service.rs (a pre-existing visual issue on
id_prefix: None).
Workspace warnings still at 0. touch_used() isn't wired into search
hot-path yet — follow-up commit when the search handlers can
adopt it without a broader refactor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
6532938e85 |
gateway/tools: truth gate for model-provided SQL (iter 11 CF-1+CF-2)
Scrum iter 11 flagged crates/gateway/src/tools/service.rs with two
95%-confidence critical failures:
CF-1: "Direct SQL execution from model-provided parameters without
explicit validation or sanitization" (line 68, 95% conf)
CF-2: "No permission check performed before executing SQL query;
access control is bypassed entirely" (line 102, 90% conf)
CF-1 is the real one — same security gap as queryd /sql had before
P42-002 (
|
||
|
rootandClaude Opus 4.7
|
de8fb10f52 |
phase-42: truth/ repo-root dir + TOML rule loader
lakehouse/auditor 4 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Phase 42 PRD (docs/CONTROL_PLANE_PRD.md:144): "truth/ dir at repo
root — rule files, versioned in git." Didn't exist. Landing both the
dir + its loader.
New files:
truth/
README.md — documents file format, rule shape,
composition model (file rules are
additive on top of in-code default_
truth_store), explicit non-goals
(no hot reload, no inheritance)
staffing.fill.toml — 2 staffing.fill rules:
endorsed-count-matches-target,
city-required (both Reject via
FieldEmpty)
staffing.any.toml — 1 staffing.any rule:
no-destructive-sql-in-context via
FieldContainsAny (parallel to the
queryd SQL gate we already ship)
crates/truth/src/loader.rs — load_from_dir(store, dir)
— 5 tests: happy path, duplicate-ID
rejection within files, duplicate-ID
rejection against in-code rules,
non-toml files skipped, missing-dir
error. Alphabetical file order for
reproducible error messages.
crates/truth/src/lib.rs — new pub fn all_rule_ids() helper on
TruthStore so the loader can detect
collisions without breaching the
private `rules` field.
crates/truth/Cargo.toml — adds `toml` workspace dep.
Composition model: file rules are ADDITIVE on top of what
default_truth_store() registers in code. Operators can tune
thresholds/needles/descriptions at the file layer without a code
deploy. Schema changes (new RuleCondition variants) still need a
code bump.
Integration hook (not in this commit, flagged for follow-up):
main.rs should call loader::load_from_dir(&mut store, "truth/")
after default_truth_store() so file-backed rules take effect on
gateway boot. Deliberately separate: this commit lands the
machinery; wiring it on happens when the team is ready to own
the rule file lifecycle.
Total: 37 truth tests green (was 32). Workspace warnings still 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
0b3bd28cf8 |
phase-40: Gemini + Claude provider adapters
Phase 40 PRD (docs/CONTROL_PLANE_PRD.md:82-83) listed: - crates/aibridge/src/providers/gemini.rs - crates/aibridge/src/providers/claude.rs Neither existed. Landing both now, in gateway/src/v1/ (matches the existing ollama.rs + openrouter.rs sibling pattern — aibridge's providers/ is for the adapter *trait* abstractions, v1/ holds the concrete /v1/chat dispatchers that know the wire format). gemini.rs: - POST https://generativelanguage.googleapis.com/v1beta/models/ {model}:generateContent?key=<API_KEY> - Auth: query-string key (not bearer) - Maps messages → contents+parts (Gemini's wire shape), extracts from candidates[0].content.parts[0].text - 3 tests: key resolution, body serialization (camelCase generationConfig + maxOutputTokens), prefix-strip claude.rs: - POST https://api.anthropic.com/v1/messages - Auth: x-api-key header + anthropic-version: 2023-06-01 - Carries system prompt in top-level `system` field (not messages[]). Extracts from content[0].text where type=="text" - 4 tests: key resolution, body serialization with/without system field, prefix-strip v1/mod.rs: + V1State.gemini_key + claude_key Option<String> + resolve_provider() strips "gemini/" and "claude/" prefixes + /v1/chat dispatcher handles "gemini" + "claude"/"anthropic" + 2 new resolve_provider tests (prefix + strip per adapter) main.rs: + Construct both keys at startup via resolve_*_key() helpers. Missing keys log at debug (not warn) since these are optional providers — unlike OpenRouter which is the rescue rung. Every /v1/chat error path mirrors the existing pattern: - 503 SERVICE_UNAVAILABLE when key isn't configured - 502 BAD_GATEWAY with the provider's error text when the upstream call fails - Response shape always the OpenAI-compatible ChatResponse Workspace warnings still at 0. 9 new tests pass. Pre-existing test failure `executor_prompt_includes_surfaced_ candidates` at execution_loop/mod.rs:1550 is unrelated (fails on pristine HEAD too — PR fixture divergence). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
b5b0c00efe |
phase-43: new crates/validator — trait, staffing impls, devops scaffold
lakehouse/auditor 3 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Phase 43 PRD (docs/CONTROL_PLANE_PRD.md:161) was the one audit finding
truly unimplemented — no crate, no trait, no tests, no workspace entry.
Neither PHASES.md nor the source tree had any Phase 43 presence.
Genuine greenfield gap.
Lands the scaffold as a real crate, registered in workspace Cargo.toml:
crates/validator/
src/lib.rs — Validator trait, Artifact enum (5 variants:
FillProposal, EmailDraft, Playbook,
TerraformPlan, AnsiblePlaybook), Report,
Finding, Severity, ValidationError
src/staffing/mod.rs — staffing validators module root
src/staffing/fill.rs — FillValidator (schema-level: fills array
+ per-fill {candidate_id, name}). 4 tests.
Worker-existence + status + geo checks
are TODO v2 (need catalog query handle).
src/staffing/email.rs — EmailValidator (to/body schema + SMS ≤160
+ email subject ≤78). 4 tests. PII scan +
name-consistency TODO v2.
src/staffing/playbook.rs — PlaybookValidator (operation prefix,
endorsed_names non-empty + ≤ target×2,
fingerprint present per Phase 25). 5 tests.
src/devops.rs — TerraformValidator + AnsibleValidator
scaffolds. Return Unimplemented — keeps
dispatcher shape stable, surfaces a clear
"phase 43 not wired" signal instead of
silently passing or panicking.
Total: 15 tests, all green. Covers the happy paths, the common
failure modes (missing fields, overfull arrays, length violations),
and the dispatch-error path (wrong artifact type into wrong validator).
Still open from Phase 43 (v2 work, beyond scaffold):
- FillValidator catalog integration (worker-existence, status,
geo/role match) — needs catalog handle in constructor
- EmailValidator PII scan (shared::pii::strip_pii integration) +
name-consistency cross-check
- Execution loop wiring: generate → validate → observer correction
+ retry (bounded by max_iterations=3) — spans crates, follow-up
- Observer logging: validation results to data/_observer/ops.jsonl
and data/_kb/outcomes.jsonl
- Scenario fixture tests against tests/multi-agent/playbooks/*
Workspace warnings still at 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
2f1b9c9768 |
phase-39+41: land promised artifacts — providers.toml, activation.rs, profiles/
Three PRD gaps closed in one coherent batch — all were cosmetic or
scaffold-shaped, now real files:
Phase 39 (PRD:57):
+ config/providers.toml — provider registry (name/base_url/auth/
default_model) for ollama, ollama_cloud, openrouter. Commented
stubs for gemini + claude pending adapter work. Secrets stay in
/etc/lakehouse/secrets.toml or env, NEVER inline.
Phase 41 (PRD:115):
+ crates/vectord/src/activation.rs — ActivationTracker with the
PRD-named single-flight guard ("refuse new activation if one is
pending/running"). Per-profile granularity — activating A doesn't
block B. 5 tests cover the full state machine. Handler body stays
in service.rs for now; tracker usage integration is a follow-up.
Phase 41 (PRD:113):
+ crates/shared/src/profiles/ with 4 submodules:
* execution.rs — `pub use crate::types::ModelProfile as
ExecutionProfile` (backward-compat rename per PRD)
* retrieval.rs — top_k, rerank_top_k, freshness cutoff,
playbook boost, sensitivity-gate enforcement
* memory.rs — playbook boost ceiling, history cap, doc
staleness, auto-retire-on-failure
* observer.rs — failure cluster size, alert cooldown, ring
size, langfuse forwarding
All fields `#[serde(default)]` so existing ModelProfile files
load unchanged.
Still open from the same phases:
- Gemini + Claude provider adapters (Phase 40 — 100-200 LOC each)
- Full activate_profile handler extraction into activation.rs
(Phase 41 — module-structure refactor)
- Catalogd CRUD endpoints for retrieval/memory/observer profiles
(Phase 41 — exists at list level, no create/update/delete yet)
- truth/ repo-root directory for file-backed rules (Phase 42 —
TOML loader + schema)
- crates/validator crate (Phase 43 — full greenfield)
Workspace warnings still at 0. 5 new tests, all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
021c1b557f |
agent.ts: route generateCloud through /v1/chat (Phase 44 migration)
Phase 44 PRD (docs/CONTROL_PLANE_PRD.md:204) explicitly lists
`tests/multi-agent/agent.ts::generate()` as a migration target:
every internal LLM caller must flow through /v1/chat so usage
accounting + audit trail see all traffic.
generateCloud() was bypassing the gateway entirely — direct POST to
OLLAMA_CLOUD_URL/api/generate with the bearer key. This meant:
- /v1/usage missed every agent.ts cloud call
- No gateway-side caching, rate-limiting, or cost gating
- Callers needed OLLAMA_CLOUD_KEY in env (leak risk; gateway
already owns the key)
Migration:
- Endpoint: OLLAMA_CLOUD_URL/api/generate → GATEWAY/v1/chat
- Body shape: {prompt,options.num_predict,options.temperature} →
OpenAI-compatible {messages[],temperature,max_tokens}
- provider: "ollama_cloud" explicit in the request
- Response extraction: data.response → data.choices[0].message.content
- OLLAMA_CLOUD_KEY no longer required in agent.ts env
Phase 44 gate verified: `grep localhost:3200/generate|/api/generate`
now only hits (a) the ollama_cloud.rs adapter itself (legit — it's
the gateway-side direct caller) and (b) this comment explaining the
migration history. Zero non-adapter code paths to /api/generate.
generate() (local Ollama) still goes direct to :3200 — that's the
t1_hot path. Phase 44 PRD focuses on cloud callers; hot-path local
generation deliberately stays direct for latency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
049a4b69fb |
truth: split staffing + devops into dedicated modules (Phase 42 PRD)
Phase 42 PRD (docs/CONTROL_PLANE_PRD.md:137) specified:
- crates/truth/src/staffing.rs — staffing rule shapes
- crates/truth/src/devops.rs — scaffold for DevOps long-horizon
PHASES.md marked Phase 42 done, but the rule sets lived inline in
default_truth_store() in lib.rs. Worked, but doesn't match the PRD's
module separation — and that separation matters when the long-horizon
phase fleshes out devops rules: "Keeps the dispatcher signature stable
so no refactor needed later."
Fix: extract staffing_rules() into staffing.rs (5 rules, unchanged
behavior) + create devops.rs with an empty scaffold. default_truth_store
becomes a one-line composition:
devops::devops_rules(staffing::staffing_rules(TruthStore::new()))
4 new tests in the submodules cover:
- staffing_rules registers expected count (regression guard)
- blacklisted worker fails the client-not-blacklisted rule
- missing deadline fires Reject via FieldEmpty condition
- devops scaffold is a no-op for now
Total truth tests: 28 → 32. Workspace warnings still at 0.
Still open from Phase 42 (flagged, not in this commit):
- `truth/` dir at repo root for file-backed rule loading (TOML/YAML).
Rules are in-code today; loader work is a separate feature.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
ed85620558 |
scrum: filter table-header words from bug_fingerprint extraction
Iter 11 surfaced "DeadCode:Flag" in the matrix — a noisy pattern_key where "Flag" is the table column HEADER kimi produces for structured review output, not an actual Rust identifier. Kimi's standard format on recent iters: | # | Change | Flag | Confidence | | 1 | Wire AgentIdentity into.. | Boundary.. | 92% | The extractor's KEYWORDS set already filtered Rust grammar words (self, mut, async, etc) and the FLAG_VARIANTS themselves. Adding markdown-layout words (Flag, Change, Confidence, PRD, Plan) closes the last common noise class. One-line addition — empirically validated against the iter 11 vectord trace that produced DeadCode:Flag. Future iters won't reproduce that specific noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
08cc960115 |
vectord: Phase 41 gate fixes — 202 ACCEPTED + /profile/jobs/{id} alias
Phase 41 PRD (docs/CONTROL_PLANE_PRD.md:121) gate:
"Activate a profile → returns 202 in <100ms → job completes in
background → /vectors/profile/jobs/{id} shows progress"
Two concrete mismatches to PRD:
1. activate_profile returned HTTP 200, not 202. Fix: wrap the Json
return in (StatusCode::ACCEPTED, Json(...)) so the async semantics
are visible at the status-code level.
2. The PRD quotes GET /vectors/profile/jobs/{id} but code only exposed
/vectors/jobs/{id}. Fix: add an alias route — same get_job handler,
second URL matches what the PRD's polling example documents.
Still open from Phase 41 (flagged for follow-up, bigger scope):
- crates/shared/src/profiles/ module with ExecutionProfile,
RetrievalProfile, MemoryProfile, ObserverProfile types — PRD
claims them, file doesn't exist; ModelProfile still does all
four roles today. This is a real schema-refactor, not 6-line work.
- crates/vectord/src/activation.rs with ActivationTracker — the
activation logic lives inline in service.rs; extracting it is
a module-structure change.
- Phase 37 hot-swap stress test in tests/multi-agent/run_stress.ts
Phase 3 — PRD says it must pass, current state unknown.
Workspace warnings still at 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
24b06d80b2 |
mcp: register gitea-mcp server — closes Phase 40 repo-ops gap
Phase 40 PRD (docs/CONTROL_PLANE_PRD.md:91) claimed:
"Gitea MCP reconnect — the MCP server binary still installed at
/home/profit/.bun/install/cache/gitea-mcp@0.0.10/ gets wired into
mcp-server/index.ts tool registry."
The PHASES.md checkbox marked this done, but audit found:
- gitea-mcp binary exists in bun cache (verified)
- Zero references to gitea/list_prs/open_pr in mcp-server/index.ts
- No entry for "gitea" in .mcp.json
The PRD's architectural description ("wired into mcp-server/index.ts
tool registry") is conceptually wrong — gitea-mcp is a peer MCP server
that the MCP host (Claude Code) connects to directly, not a library
to import. Correct wiring: register it in .mcp.json so Claude Code
spawns both lakehouse's MCP server AND gitea-mcp as separate children,
each exposing their own tools.
This commit adds the "gitea" entry to .mcp.json pointing at bunx
gitea-mcp with GITEA_HOST set to git.agentview.dev.
OPERATOR STEP (one-time): before restarting Claude Code, generate a
personal access token at https://git.agentview.dev/user/settings/
applications and replace the SET_ME_... placeholder in
GITEA_ACCESS_TOKEN. Token needs at minimum `read:repository,
write:issue, read:user` scopes for list_prs/open_pr/comment_on_issue.
Still open from Phase 40 (not in this commit, bigger scope):
- crates/aibridge/src/providers/gemini.rs (claimed, missing)
- crates/aibridge/src/providers/claude.rs (claimed, missing)
These are ~100-200 lines each (full HTTP adapter + auth + request
shape mapping). Flag as follow-up commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
999abd6999 |
gateway/v1: model-prefix routing closes Phase 39 PRD gate
lakehouse/auditor 4 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Phase 39 PRD (docs/CONTROL_PLANE_PRD.md:62) promised:
"/v1/chat routes by `model` field: prefix match
(e.g. openrouter/anthropic/claude-3.5-sonnet → OpenRouter;
bare names → Ollama)"
Actual behavior required clients to pass `provider: "openrouter"`
explicitly. Bare `model: "openrouter/..."` would fall through to the
"unknown provider ''" error. PRD gate never actually passed.
Fix: resolve_provider(&ChatRequest) picks (provider, effective_model):
- explicit `req.provider` wins, model passes through unchanged
- else strip "openrouter/" prefix → provider="openrouter", model
without prefix (OpenRouter API expects "openai/gpt-4o-mini",
not "openrouter/openai/gpt-4o-mini")
- else strip "cloud/" prefix → provider="ollama_cloud"
- else default provider="ollama"
Adapter calls use Cow<ChatRequest>: borrowed when no strip needed
(zero alloc), owned when we needed to build a new model string. Keeps
the hot path allocation-free for the common case.
ChatRequest gains #[derive(Clone)] — needed for the Owned variant.
5 new tests pin the resolution semantics including the
"explicit provider + prefixed model" corner case (trust the caller,
don't double-strip).
Workspace warnings unchanged at 0.
Still not shipped from Phase 39: config/providers.toml — hardcoded
match arms work fine in practice, centralizing them is cosmetic.
Flag as a follow-up if a 4th provider lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
0cf1b7c45a |
scrum_master: env-configurable tree-split threshold + shard size
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Hard-coded constants (FILE_TREE_SPLIT_THRESHOLD=6000, FILE_SHARD_SIZE=3500) were tuned for Rust source files in crates/<crate>/src/*.rs. Running the pipeline against /root/llm-team-ui/llm_team_ui.py (13K lines, ~400KB) would produce ~200 shards per review at the default size — not viable. Two env vars now: - LH_SCRUM_TREE_SPLIT_THRESHOLD — when tree-split fires (default 6000) - LH_SCRUM_SHARD_SIZE — bytes per shard (default 3500) For the big-Python case the CLAUDE.md in /root/llm-team-ui/ recommends LH_SCRUM_TREE_SPLIT_THRESHOLD=20000, LH_SCRUM_SHARD_SIZE=12000 which brings the 13K-line file down to ~35 shards — same ballpark as a typical Rust file review. No default change. Existing lakehouse runs unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
81bae108f4 |
gateway/tools: collapse ToolRegistry::new() and new_with_defaults() into one
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Two constructors existed with a subtle trap:
- `new()` had `#[allow(dead_code)]` and called `register_defaults()`
via `tokio::task::block_in_place(...)` — a sync wrapper hack around
an async method, fragile and unused.
- `new_with_defaults()` was misleadingly named — it created the empty
registry WITHOUT registering defaults, despite the name.
main.rs was doing the right thing: `new_with_defaults()` + explicit
`.register_defaults().await`. The misleading name was a landmine
for future callers.
Fix: delete the dead `new()` with its block_in_place hack, rename
`new_with_defaults()` → `new()` (Rust idiom — `new` is the canonical
constructor), add a docstring that says what you need to do after.
Single clear API.
Update the one caller in main.rs. Workspace warnings still at 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
5df4d48109 |
cleanup: drop two #[allow] attributes that were hiding real dead code
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
- ingestd/src/service.rs: top-of-file `#[allow(unused_imports)]`
was masking genuinely unused `delete` and `patch` routing
constructors in an axum import block. Removed the attribute,
trimmed the imports to only `get` and `post` (what's actually
used). Any future over-import now trips the unused_imports
lint immediately instead of being silently allowed.
- gateway/src/v1/truth.rs: `truth_router()` was a 4-line stub
wrapping a single `/context` route — carried `#[allow(dead_code)]`
because v1/mod.rs wires `get(truth::context)` directly onto its
own router, bypassing this helper. Zero callers across the
workspace. Deleted the function + allow + now-unused Router
import. Left a breadcrumb comment pointing to the real wiring.
Workspace warnings: 0 (lib + tests). Each #[allow] removed raises
the bar on future code entering these modules — the linter now
catches the same classes of bugs at PR time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
ffdc842ec3 |
ingestd: scope test-only imports into the test module
schema_evolution.rs had two `#[allow(unused_imports)]` attributes hiding
over-broad top-level imports:
- `Schema` was imported at crate level but only used in test code
- `Arc` was imported at crate level but only used in test code
- `DataType` and `SchemaRef` were actually used (28 references) — the
allow on that line was cargo-culted.
Fix: drop the allows, move Schema + Arc into the #[cfg(test)] block
where they're actually used. The non-test build no longer imports
symbols it doesn't need. Test build still works because the imports
are now in the test module's scope.
Workspace warnings still at 0 (lib + tests). Net: -3 import lines
from crate scope, +2 into test scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
12e615bb5d |
ingestd/vectord: remove two fragile unwraps on Option paths
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Both were technically safe — guarded above by map_or(true, ...) and
Some(entry) assignment respectively — but relied on multi-line
invariants that a future refactor could easily break.
- ingestd/watcher.rs:80: path.file_name().unwrap() on a path that
was already checked via map_or(true, ...) two lines up. Fix:
let-else binds filename once, no double lookup, no unwrap.
- vectord/promotion.rs:145: file.current.as_ref().unwrap() called
TWICE on the same line to log config + trial_id. Guard via
`if let Some(cur) = &file.current` so the log gracefully skips
if the invariant ever breaks instead of panicking at runtime.
Both are drop-in semantically: happy path identical, error path now
graceful-skip instead of panic. Workspace warnings still at 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
a934a76988 |
aibridge: delete deprecated estimate_tokens wrapper — fully migrated
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
|
||
|
rootandClaude Opus 4.7
|
cdc24d8bd0 |
shared: build ModelMatrix — migrate 5 call sites off deprecated estimate_tokens
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
The `aibridge::context::estimate_tokens` deprecation has been pointing
at `shared::model_matrix::ModelMatrix::estimate_tokens` for a while,
but that module didn't exist — so the deprecation was aspirational
noise, not actionable guidance.
Built the minimal target: `shared::model_matrix::ModelMatrix` with
an associated `estimate_tokens(text: &str) -> usize` method. Same
chars/4 ceiling heuristic as the deprecated helper. 6 tests cover
empty/3/4/5-char cases, multi-byte UTF-8 (emoji count as 1 char each),
and linear scaling to 400-char inputs.
Migrated 5 call sites:
- aibridge/context.rs:88 — opts.system token count
- aibridge/context.rs:89 — prompt token count
- aibridge/tree_split.rs:22 — import (now uses ModelMatrix)
- aibridge/tree_split.rs:84, 89 — truncate_scratchpad budget loop
- aibridge/tree_split.rs:282 — scratchpad post-truncation assertion
- aibridge/context.rs:183 — system-prompt budget test
Also cleaned up two parallel test warnings:
- aibridge/context.rs legacy estimate_tokens_ceiling_divides_by_four
test deleted (ModelMatrix's tests cover the same behavior now).
- vectord/playbook_memory.rs:1650 unused_mut on e_alive.
Net workspace warning count: 11 → 0 (including --tests build).
The deprecated `estimate_tokens` wrapper stays in aibridge/context.rs
for external callers. Future commits can remove it entirely once no
public API surface still references it.
The applier's warning-count gate now has a floor of 0 — any future
patch that introduces a single warning trips the gate automatically.
Previously a floor of 11 tolerated noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
fdc5123f6d |
cleanup: drop workspace warnings from 11 to 6
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Three trivial cleanups that pull the workspace baseline down by five:
- vectord/trial.rs: removed unused ObjectStore import (not referenced
anywhere in the file; cargo's unused_imports lint was flagging it
on every check). Net: -2 warnings (cascade effect from one import).
- ui/main.rs:1241: `Err(e)` with unused binding → `Err(_)`.
- ui/main.rs:1247: `let mut import_table` never mutated → `let`.
Matters because the scrum_applier's hardened warning-count gate uses
this baseline as its reject threshold. Lower baseline = lower floor
= any future patch that adds a warning trips the gate earlier.
Remaining 6 warnings are all aibridge context::estimate_tokens
deprecation notices pointing at a planned-but-unbuilt
shared::model_matrix::ModelMatrix::estimate_tokens. Fix requires
creating that type (next commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
51a1aa3ddc |
gateway/execution_loop: wire truth gate (Phase 42 step 6 — was TODO)
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Line 156 had `// --- (6) TRUTH GATE — PORT FROM Phase 42 (TODO) ---`
sitting empty for weeks. The Blocked outcome variant existed but was
marked #[allow(dead_code)] because nothing constructed it.
Now: before the main turn loop, evaluate truth rules for the request's
task_class against self.req.spec. Any rule whose condition holds AND
whose action is Reject/Block short-circuits to RespondOutcome::Blocked
with a reason citing the rule_id. Downstream finalize() already matched
Blocked at line 848 (maps to truth_block category in kb row).
Mirrors the queryd/service.rs SQL gate from
|
||
|
rootandClaude Opus 4.7
|
d122703e9a |
vectord: delete _run_embedding_job_legacy — 44 lines of explicit dead code
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Function was labeled "Legacy single-pipeline embedding (replaced by supervisor)" with a #[allow(dead_code)] attribute. Zero callers across the workspace. This is exactly what `#[allow(dead_code)]` is supposed to silently flag as "I know this is dead but I'm not committing to removing it" — so let's commit to removing it. Iter memory grep for this pattern showed 5 remaining #[allow(dead_code)] attributes in the workspace (1 here, 4 in gateway/access.rs). The four in access.rs are waiting on P13-001 (queryd → AccessControl wiring) before removing — that's cross-crate work. This one was self-contained. Net: -44 lines of dead code + comment. Workspace warnings unchanged at 11. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
3963b28b50 |
aibridge: fix glob_match — remove dead panic branch + add multi-* support
Iter 9 scrum flagged routing.rs with OffByOne + NullableConfusion risks
on the glob matcher. Two real bugs in one function:
1. The `else if parts.len() == 1` branch was dead AND panic-hazardous:
split('*') on a string containing '*' always yields ≥2 parts, so
the branch was unreachable — but if ever reached (via future
caller or split-behavior change), `parts[1]` would index out of
bounds and panic.
2. Multi-* patterns like `gpt-*-large*` fell through to exact-match
because the `parts.len() == 2` branch only handled single-*. Result:
a rule like `model_pattern: "gpt-*-oss-*"` would only match the
literal string "gpt-*-oss-*", never an actual gpt-4-oss-120b.
Fix walks parts left-to-right: prefix check, suffix check, each
interior segment must appear in order. Cursor-advance logic ensures
a mid-segment that appears before cursor (duplicate prefix) can't
falsely match.
8 new tests cover: exact match, exact mismatch, leading/trailing/bare
wildcards, multi-* in-order, multi-* wrong-order (regression guard),
and the old panic-hazard case ("a*b*c" variants) as an explicit check.
Workspace warnings unchanged at 11.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
c47523e5bd |
queryd: add latency_ms to QueryResponse (iter 9 finding #3, 88% conf)
Scrum iter 9 flagged that gateway's audit row stores null for `latency_ms` — required for PRD audit-log parity. The field didn't exist; adding it now with a single Instant captured at handler entry, populated on both response paths (empty batches + non-empty result). No behavior change for existing clients — they read the JSON and ignore unknown fields. Audit-log consumers can now surface p50/p99 latency from the response body instead of inferring from tracing. Narrow fingerprint on crates/queryd already has this as a known BoundaryViolation pattern (`latency_ms-row_count` key) — iter 10 on any queryd file will see the preamble say "this was fixed in iter 10" when it runs. Workspace warnings unchanged at 11. 7 policy tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
fd92a9a0d0 |
docs: SCRUM_MASTER_SPEC.md — single handoff artifact for the scrum loop
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Fresh-session artifact so work is recoverable if the branch is reopened
in a new Claude Code session without context. Covers:
- 9-rung ladder (kimi-k2:1t through local qwen3.5:latest)
- tree-split reducer (files >6KB sharded + map→reduce)
- schema_v4 KB rows in data/_kb/scrum_reviews.jsonl
- auto-applier 5 hardened gates (confidence, size, cargo-green,
warning-count, rationale-diff)
- pathway_memory (ADR-021) — narrow fingerprint + hot-swap gate +
semantic-correctness layer (SemanticFlag, BugFingerprint)
- HTTP surface on gateway (/vectors/pathway/*)
- current state (12 traces, 11 fingerprints, 0 hot-swaps — probation)
- commit history on scrum/auto-apply-19814 since iter-5 baseline
- how-to-run (env vars, service restarts)
- where things live (code pointers table)
- known gotchas (LLM Team mode registry, restart requirements)
Paired updates (not in this commit, live outside the repo):
- /home/profit/CLAUDE.md — active workstream pointer + notes
- /root/.claude/skills/read-mem/SKILL.md — SCRUM_MASTER_SPEC.md added
to the loading list + ADR-021 glossary
- memory/project_scrum_pipeline.md — updated with iter-9 state
- memory/feedback_semantic_correctness_via_matrix.md — updated with
end-to-end proof evidence
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
f4cff660aa |
ADR-021 Phase D fix: strip flag names + Rust keywords from pattern_keys
Iter 9 revealed two quality bugs in the extractor: 1. Kimi wraps the Flag column in backticks (\`DeadCode\`), so the flag name itself was captured as a code token. Result: pattern_keys like "DeadCode:DeadCode" that match nothing and add noise to the index. Fix: filter FLAG_VARIANTS out of token candidates. 2. Complex backtick content like \`Foo::bar(&self) -> u64\` was rejected wholesale by the identifier regex. Fallback now scans for identifier substrings and ranks by ::-qualified paths first, then length. Bonus: filter Rust keywords (self, mut, async, etc) since they're grammar, not bug-shape signal. Dry-run on iter 9 delta.rs output produces semantically meaningful keys: DeadCode:DeltaStats::tombstones_applied NullableConfusion:DeltaError-DeltaStats-apply_delta BoundaryViolation:apply_delta-journald::emit-rows_dropped_by_tombstones PseudoImpl:apply_delta-delta_ops-validate_schema These are stable under reviewer prose variation (canonical sort + top-3 slice) and precise enough to separate different bugs within the same Flag category. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
ee31424d0c |
ADR-021 Phase D: bug_fingerprint pattern extraction from reviewer output
lakehouse/auditor 4 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Fills the gap between Phase B (flags tagged) and Phase C (preamble
quotes past fingerprints): parses each reviewer line that mentions a
Flag variant, collects backtick-quoted identifiers, canonicalizes them
(sorted alphabetically, top 3), and emits a stable pattern_key of
shape `{Flag}:{tok1}-{tok2}-{tok3}`.
Stability by design: canonical sort means "row_count + QueryResponse"
and "QueryResponse + row_count" produce the same key, so variation in
reviewer prose doesn't fragment the index. Top-3 cap keeps keys short
while retaining enough signal to separate different bugs of the same
category.
Dry-run validation on iter-8 delta.rs output (crates/queryd prefix)
extracted 10 semantically meaningful fingerprints including:
- UnitMismatch:base_rows-checked_add-checked_sub
- DeadCode:queryd::delta::write_delta (P9-001 dead-function finding)
- BoundaryViolation:can_access-log_query-masked_columns (P13-001 gap)
- NullableConfusion:CompactResult-DeltaError-IntegerOverflow
Cross-cutting signal: kimi-k2:1t's finding #5 explicitly quoted the
seeded pathway memory preamble ("Pathway memory flags row_count-
file_count unit mismatch") and proposed overflow-checked arithmetic as
the fix. That is the compounding loop in action — prior bug context
shifted the reviewer's attention toward a specific instance of the
same class, which produces a specific pattern_key that will compound
further on the next iter.
Filter: identifier-shaped tokens only (A-Za-z_ / :: paths / snake_case
/ CamelCase). Skips punctuation, prose quotes, and tokens <3 chars so
generic nouns and partial words don't pollute the index.
What's still queued (Phase E):
- type_hints_used population from catalogd column types + Arrow schema
- auditor → pathway audit_consensus update wire (strict-audit gate
activation)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
0a0843b605 |
ADR-021: semantic-correctness layer lands in pathway_memory (A+B+C)
lakehouse/auditor 4 blocking issues: todo!() macro call in tests/real-world/scrum_master_pipeline.ts
Phase A — data model (vectord/src/pathway_memory.rs):
+ SemanticFlag enum (9 variants: UnitMismatch, TypeConfusion,
NullableConfusion, OffByOne, StaleReference, PseudoImpl, DeadCode,
WarningNoise, BoundaryViolation) as #[serde(tag = "kind")]
+ TypeHint { source, symbol, type_repr }
+ BugFingerprint { flag, pattern_key, example, occurrences }
+ PathwayTrace gains semantic_flags, type_hints_used, bug_fingerprints
all #[serde(default)] for back-compat deserialization of pre-ADR-021
traces on disk
+ build_pathway_vec now tokenizes flag:{variant} + bug:{flag}:{key}
so traces with different bug histories cluster separately in the
similarity gate (proven by pathway_vec_differs_when_bug_fingerprint_added
test)
Phase B — producer (scrum_master_pipeline.ts):
+ Prompt addendum: each finding must carry `**Flag: <CATEGORY>**` tag
alongside the existing Confidence: NN% tag. 9 category choices plus
`None` for improvements that aren't bug-shaped.
+ Parser extracts tagged flags from reviewer markdown; falls back to
bare-word match if reviewer omits the label. Deduplicated per trace.
+ PathwayTracePayload gains semantic_flags / type_hints_used /
bug_fingerprints fields. Wire format matches Rust serde tagged enum
so TS and Rust interop directly.
Phase C — pre-review enrichment:
+ new `/vectors/pathway/bug_fingerprints` endpoint aggregates
occurrences by (flag, pattern_key) across traces sharing a narrow
fingerprint, sorts by frequency, returns top-K.
+ scrum calls it before the ladder and prepends a PATHWAY MEMORY
preamble to the reviewer prompt ("these patterns appeared N times
on this file area before — check for recurrences"). Empty on
fresh install; grows as the matrix index learns.
Tests: 27 pathway_memory tests green (was 18). New tests:
- pathway_trace_deserializes_without_new_fields_backcompat
- semantic_flag_serializes_as_tagged_enum
- bug_fingerprint_roundtrips_through_serde
- pathway_vec_differs_when_bug_fingerprint_added
- semantic_flag_discriminates_by_variant
- bug_fingerprints_aggregate_by_pattern_key (sums occurrences, sorts desc)
- bug_fingerprints_empty_for_unseen_fingerprint
- bug_fingerprints_respects_limit
- insert_preserves_semantic_fields (roundtrip via persist + reload)
Workspace warnings unchanged at 11.
What's still queued (not this commit):
- type_hints_used population from catalogd column types + Arrow schema
- bug_fingerprint extraction from reviewer output (Phase D — for now
semantic_flags populate but the fingerprint key requires parsing
code-shape from the finding; next iteration's work)
- auditor → pathway audit_consensus update wire (explicit-fail gate)
Why this commit matters: the mechanical applier's gates are syntactic
(warning count, patch size, rationale-token alignment). The
queryd/delta.rs base_rows bug (
|
||
|
rootandClaude Opus 4.7
|
92df0e930a |
ADR-021: semantic-correctness layer on pathway_memory
Spec for the compounding-bug-grammar insight from J's feedback on the
queryd/delta.rs unit-mismatch fix (
|
||
|
rootandClaude Opus 4.7
|
86901f8def |
queryd/delta: fix CompactResult.base_rows unit mismatch (6-line fix)
lakehouse/auditor 2 blocking issues: cloud: claim not backed — "proven review pathways."
Before: `base_rows = pre_filter_rows - delta_count` subtracted a FILE count (delta_batches.len()) from a ROW count (pre_filter_rows), producing a meaningless "rough" approximation the comment acknowledged. Now: base_rows is captured directly from the pre-extend state. Same for delta_rows, which now reports actual delta row count instead of file count. Workspace baseline warnings unchanged at 11. Flagged by scrum iter 4-7 as a PRD §8.6 contract gap (upsert semantics); this closes the reporting half. Full dedup work remains queued. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
2f8b347f37 |
pathway_memory: consensus-designed sidecar + hot-swap learning loop
lakehouse/auditor 11 warnings — see review
10-probe N=3 consensus (kimi-k2:1t / gpt-oss:120b / qwen3.5:latest /
deepseek-v3.1:671b / qwen3-coder:480b / mistral-large-3:675b /
qwen3.5:397b + 2 stability re-probes; 2 openrouter probes 429'd) locked
the design across three rounds. Full JSON responses in
data/_kb/consensus_reducer_design_{mocq3akn,mocq6pi1,mocqatik}.json.
What it does
Preserves FULL backtrack context per reviewed file (ladder attempts +
latencies + reject reasons, KB chunks with provenance + cosine + rank,
observer signals, context7 bridge hits, sub-pipeline calls, audit
consensus) and indexes them by narrow fingerprint for hot-swap of
proven review pathways.
When scrum reviews a file:
1. narrow fingerprint = task_class + file_prefix + signal_class
2. query_hot_swap checks pathway memory for a match that passes
probation (≥3 replays @ ≥80% success) + audit gate + similarity
(≥0.90 cosine on normalized-metadata-token embedding)
3. if hot-swap eligible, recommended model tried first in the ladder
4. replay outcome reported back, updating the pathway's success_rate
5. pathways below 0.80 after ≥3 replays retire permanently (sticky)
6. full PathwayTrace always inserted at end of review — hot-swap
grows with use, it doesn't bootstrap from nothing
Gate design is load-bearing:
- narrow fingerprint (6 of 8 consensus models converged on the same
3-field composition; lock) — enables generalization within crate
- probation ≥3 replays — binomial tail at 80% is ~5%, below is noise
- success rate ≥0.80 — mistral + qwen3-coder independently proposed
this exact threshold across two rounds
- similarity ≥0.90 — middle of the 0.85/0.95 consensus spread
- bootstrap: null audit_consensus ALLOWED (auditor → pathway update
not wired yet; probation + success_rate gates alone enforce safety
during bootstrap; explicit audit FAIL still blocks)
- retirement is sticky — prevents oscillation on noise
Files
+ crates/vectord/src/pathway_memory.rs (new, 600 lines + 18 tests)
PathwayTrace, LadderAttempt, KbChunkRef, ObserverSignal, BridgeHit,
SubPipelineCall, AuditConsensus, HotSwapCandidate, PathwayMemory,
PathwayMemoryStats. 18/18 tests green.
Cosine + 32-bucket L2-normalized embedding; mirror of TS impl.
M crates/vectord/src/lib.rs
pub mod pathway_memory;
M crates/vectord/src/service.rs
VectorState grows pathway_memory field;
4 HTTP handlers (/pathway/insert, /pathway/query,
/pathway/record_replay, /pathway/stats).
M crates/gateway/src/main.rs
Construct PathwayMemory + load from storage on boot,
wire into VectorState.
M tests/real-world/scrum_master_pipeline.ts
Byte-matching TS bucket-hash (verified same bucket indices as
Rust); pre-ladder hot-swap query; ladder reorder on hit;
per-attempt latency capture; post-accept trace insert
(fire-and-forget); replay outcome recording;
observer /event emits pathway_hot_swap_hit, pathway_similarity,
rungs_saved per review for the VCP UI.
M ui/server.ts
/data/pathway_stats aggregates /vectors/pathway/stats +
scrum_reviews.jsonl window for the value metric.
M ui/ui.js
Three new metric cards:
· pathway reuse rate (activity: is it firing?)
· avg rungs saved (value: is it earning its keep?)
· pathways tracked (stability: retirement = learning)
What's not in this commit (queued)
- auditor → pathway audit_consensus update wire (explicit audit-fail
block activates when this lands)
- bridge_hits + sub_pipeline_calls population from context7 / LLM
Team extract results (fields wired, callers not yet)
- replay log (PathwayReplayOutcome {matched_id, succeeded, ts}) as
a separate jsonl for forensic audit of why specific replays failed
Why > summarization
Summaries discard the causal chain. With this, auditor can verify
citation provenance, applier can distinguish lucky from learned paths,
and the matrix indexing actually stores end-to-end pathways instead of
just RAG chunks — which is what J meant by "why aren't we using it
for everything."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
9cc0ceb894 |
P42-002: wire truth gate into queryd /sql + /paged SQL paths
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
The scrum master flagged crates/queryd/src/service.rs across iters 3-5
with the same finding: "raw SQL forwarded to DataFusion without schema
or policy gate; violates PRD §42-002 truth enforcement." Confidence
79-95%, gradient tier auto/dry_run. Applier couldn't touch it — the fix
is larger than 6 lines and crosses crate boundaries.
Hand-fix lands the missing enforcement point:
- truth: new RuleCondition::FieldContainsAny { field, needles } with
case-insensitive substring matching. 4 new unit tests cover the
positive, negative, missing-field, and empty-needles paths.
- truth: sql_query_guard_store() helper returns a baseline store that
rejects destructive verbs (DROP/TRUNCATE/DELETE FROM) and empty SQL.
- queryd: QueryState grows an Arc<TruthStore>; default router() loads
sql_query_guard_store; new router_with_truth(engine, store) lets
tests inject a custom store.
- queryd: sql_policy_check() runs truth.evaluate("sql_query", ctx)
before hitting DataFusion. Reject/Block actions on matched
conditions short-circuit to HTTP 403 with the rule's message.
Both /sql and /paged gated.
- queryd: 7 new tests cover block/allow/case-insensitive/false-
positive scenarios. "SELECT deleted_at FROM t" must NOT be rejected
(substring match is narrow: "delete from", not "delete").
Total: 28 truth tests green (was 24), 7 new queryd policy tests green.
Workspace baseline warnings unchanged at 11.
This is a signal-driven fix the mechanical pipeline couldn't produce
but the scrum master kept asking for. Closes one of four LOOPING files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
5e8d87bf34 |
cleanup: remove unused HashSet import from 96b46cd + tighten applier gates
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
|
||
|
rootandClaude Opus 4.7
|
25ea3de836 |
observer: fix LLM Team escalation — route to /v1/chat qwen3-coder:480b instead of dead mode
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
Discovery 2026-04-24: /api/run?mode=code_review returns "Unknown mode"
(error response from llm_team_ui.py). The 2026-04-24 observer escalation
wiring pointed at a dead endpoint and was failing silently. My earlier
claim of "9 registered LLM Team modes" came from GET probes that all
returned 405 — I interpreted that as "POST-only endpoints exist" when
it just means "GET is not allowed for anything, and on POST only `extract`
is registered."
Rewire: observer's escalateFailureClusterToLLMTeam now hits
POST /v1/chat { provider: "ollama_cloud", model: "qwen3-coder:480b", ... }
which is the same coding-specialist rung 2 of the scrum ladder that
reliably produces substantive reviews. Probe shows 1240 chars of
substantive analysis in ~8.7s.
Also tightens scrum_applier:
* MODEL default: kimi-k2:1t → qwen3-coder:480b (coding specialist)
* Size gate: 20 lines → 6 lines (surgical patches only)
* Max patches per file: 3 → 2
* Prompt: explicit forbidden-actions list (no struct renames, no
function-signature changes, no new modules) and mechanical-only
whitelist
These changes produced the first auto-applied commit (
|
||
|
root
|
96b46cdb91 |
auto-apply: 1 high-confidence fix in crates/queryd/src/service.rs
- Add basic destructive SQL filter to mitigate PRD §42-002 violation (conf 90%)
🤖 scrum_applier.ts
|
||
|
rootandClaude Opus 4.7
|
8b77d67c9c |
OpenRouter rescue ladder + tree-split reduce fix + observer→LLM Team + scrum_applier + first auto-applied patch
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
## Infrastructure (scrum loop hardening)
crates/gateway/src/v1/openrouter.rs — new OpenRouter provider
Direct HTTPS to openrouter.ai/api/v1/chat/completions with OpenAI-compatible shape.
Key resolution: OPENROUTER_API_KEY env → /home/profit/.env → /root/llm_team_config.json
(shares LLM Team UI's quota). Added after iter 5 hit repeated Ollama Cloud 502s on
kimi-k2:1t — different provider backbone as rescue rung. Unit tests pin the URL
stripping and OpenAI wire shape.
crates/gateway/src/v1/mod.rs + main.rs
Added `"openrouter" | "openrouter_free"` arm to /v1/chat dispatch.
V1State.openrouter_key loaded at startup via openrouter::resolve_openrouter_key()
mirroring the Ollama Cloud pattern. Startup log:
"v1: OpenRouter key loaded — /v1/chat provider=openrouter enabled"
tests/real-world/scrum_master_pipeline.ts
* 9-rung ladder — kimi-k2:1t → qwen3-coder:480b → deepseek-v3.1:671b →
mistral-large-3:675b → gpt-oss:120b → qwen3.5:397b → openrouter/gpt-oss-120b:free
→ openrouter/gemma-3-27b-it:free → local qwen3.5:latest.
Added qwen3-coder:480b as rung 2 after live probes confirmed it rescues
kimi-k2:1t 502s cleanly (0.9s latency, substantive reviews).
Dropped devstral-2 (displaced by qwen3-coder); dropped kimi-k2.6 (not available);
dropped minimax-m2.7 (returned 0 chars / 400 thinking tokens).
Local fallback promoted qwen3.5:latest per J's direction 2026-04-24.
* MAX_ATTEMPTS bumped 6 → 9 to accommodate the rescue tier.
* Tree-split scratchpad fixed — was concatenating shard markers directly
into the reviewer input, causing kimi-k2:1t to write titles like
"Forensic Audit Report – file.rs (shard 3)". Now uses internal §N§
markers during accumulation and runs a proper reduce step that
collapses per-shard digests into ONE coherent file-level synthesis
with markers stripped. Matches the Phase 21 aibridge::tree_split
map→reduce design. Fallback to stripped scratchpad if reducer returns thin.
tests/real-world/scrum_applier.ts — NEW (737 lines)
The auto-apply pipeline. Reads scrum_reviews.jsonl, filters rows where
gradient_tier ∈ {auto, dry_run} AND confidence_avg ≥ MIN_CONF (default 90),
asks the reviewer model for concrete old_string/new_string patch JSON,
applies via text replacement, runs cargo check after each file, commits
if green and reverts if red. Deny-list: /etc/, config/, ops/, auditor/,
docs/, data/, mcp-server/, ui/, sidecar/, scripts/. Hard caps: per-patch
confidence ≥ MIN_CONF, old_string must be exactly unique, max 20 lines per
patch. Never runs on main without explicit LH_APPLIER_BRANCH override.
Audit trail in data/_kb/auto_apply.jsonl.
Empirical behavior (dry-run over iter 4 reviews):
5 eligible files → 1 green commit-ready, 2 build-red reverts, 2 all-rejected
The build-green gate caught 2 bad patches before they'd have merged.
mcp-server/observer.ts — LLM Team code_review escalation
When a sig_hash accumulates ≥3 failures (ESCALATION_THRESHOLD), fire-and-forget
POST /api/run?mode=code_review at localhost:5000 with the failure cluster context.
Parses facts/entities/relationships/file_hints from the response. Writes to a
new data/_kb/observer_escalations.jsonl surface. Answers J's vision of the
observer triggering richer LLM Team calls when failures pile up.
Non-blocking: runs parallel to existing qwen2.5 analyzer, never replaces it.
Tracks escalated sig_hashes in a session-local Set to avoid re-hammering
LLM Team when a cluster persists across observer cycles.
crates/aibridge/src/context.rs
First auto-applied patch produced by scrum_applier.ts (dry-run path —
applier writes files in dry-run mode but doesn't commit; bug noted for
iter 6 fix). Adds #[deprecated] annotation to the inline estimate_tokens
helper pointing callers to the centralized shared::model_matrix::ModelMatrix
entry point (P21-002 — duplicate token-estimator surfaces). Cargo check
passes with the annotation (verified by applier's own build gate).
## Visual Control Plane (UI)
ui/server.ts — Bun.serve on :3950 with /data/* fan-out:
/data/services, /data/reviews, /data/metrics, /data/trust, /data/overrides,
/data/findings, /data/outcomes, /data/audit_facts, /data/file/:path,
/data/refactor_signals, /data/search?q=, /data/signal_classes,
/data/logs/:svc (journalctl tail per systemd unit), /data/scrum_log.
Bug fix: tryFetch always attempts JSON.parse before falling back to text
— observer's Bun.serve returns JSON without application/json content-type,
which was displaying stats as a raw string ("0 ops" on map) before.
ui/index.html + ui.css — dark neo-brutalist shell. 6 views:
MAP (D3 force-graph + overlays) / TRACE (per-file iter history) /
TRAJECTORY (signal-class cards + refactor-signals table + reverse-index
search box) / METRICS (every card has SOURCE + GOOD lines explaining
where the number comes from and what target trajectory means) /
KB (card grid with tooltips on every field) / CONSOLE (per-service
journalctl tabs).
ui/ui.js — polling client, D3 wiring, signal-class panel, refactor-signals
table, reverse-index search, per-service console tabs. Bug fix:
renderNodeContext had Object.entries() iterating string characters when
/health returned a plain string — now guards with typeof check so
"lakehouse ok" renders as one row instead of "0 l / 1 a / 2 k / ...".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
rootandClaude Opus 4.7
|
39a2856851 |
docs: rewrite PR #10 description to drop unfalsifiable metric claims
lakehouse/auditor 1 blocking issue: cloud: claim not backed — "journal event verified live (total_events_created 0→1 after probe)."
Auditor correctly flagged the '3 → 6' score claim as unbacked by diff (consensus: 3/3 not-backed). The claim referenced scrum_reviews.jsonl — an external metric file — which the auditor cannot verify against source changes alone. Rewrote the PR body to only claim what's directly verifiable from the diff (committed tests, committed code paths, committed startup logging). Trajectory data remains in docs/SCRUM_LOOP_NOTES.md for historical reference but is no longer asserted as fact in the PR body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
bb4a8dff34 |
test: committed verification for P9-001 journal-on-ingest behavior
lakehouse/auditor 2 blocking issues: cloud: claim not backed — "| **P9-001** (partial) | `crates/ingestd/src/service.rs` | **3 → 6** ↑↑↑ | `journal.record_ing
Responds to PR #10 auditor block (2/2 blocking: "claim not backed"): the auditor's N=3 cloud consensus flagged the "verified live" language in the description as unbacked by the diff. That was fair — the verification was a manual curl probe, not committed code. Committed verification now lives in the diff: * journal_record_ingest_increments_counter - mirrors the /ingest/file success path against an in-memory store - asserts total_events_created: 0 → 1 after record_ingest - asserts the event is retrievable by entity_id with correct fields * optional_journal_field_none_is_valid_back_compat - pins IngestState.journal as Option<Journal> - forces explicit reconsideration if a refactor makes it mandatory * journal_record_event_fields_match_adr_012_schema - pins the 11-field ADR-012 event schema against field-rot 3/3 pass. Resolves block 2. Block 1 ("no changes to ingestd/service.rs appear in the diff") was a tree-split shard-leakage false positive — the diff at lines 37-40 + 149-163 clearly adds the journal wiring; this commit moves those lines into direct test-exercised contact so the next audit cycle has fewer shards to stitch together. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
rootandClaude Opus 4.7
|
21fd3b9c61 |
Scrum-driven fixes: P5-001 auth wired, P42-001 truth evaluator, P9-001 journal on ingest
lakehouse/auditor 2 blocking issues: cloud: claim not backed — "| **P9-001** (partial) | `crates/ingestd/src/service.rs` | **3 → 6** ↑↑↑ | `journal.record_ing
Apply the highest-confidence findings from the Phase 0→42 forensic sweep
after four scrum-master iterations under the adversarial prompt. Each fix
is independently validated by a later scrum iteration scoring the same
file higher under the same bar.
Code changes
────────────
P5-001 — crates/gateway/src/auth.rs + main.rs
api_key_auth was marked #[allow(dead_code)] and never wrapped around
the router, so `[auth] enabled=true` logged a green message and
enforced nothing. Now wired via from_fn_with_state, with constant-time
header compare and /health exempted for LB probes.
P42-001 — crates/truth/src/lib.rs
TruthStore::check() ignored RuleCondition entirely — signature looked
like enforcement, body returned every action unconditionally. Added
evaluate(task_class, ctx) that actually walks FieldEquals / FieldEmpty /
FieldGreater / Always against a serde_json::Value via dot-path lookup.
check() kept for back-compat. Tests 14 → 24 (10 new exercising real
pass/fail semantics). serde_json moved to [dependencies].
P9-001 (partial) — crates/ingestd/src/service.rs
Added Optional<Journal> to IngestState + a journal.record_ingest() call
on /ingest/file success. Gateway wires it with `journal.clone()` before
the /journal nest consumes the original. First-ever internal mutation
journal event verified live (total_events_created 0→1 after probe).
Iter-4 scrum scored these files higher under same prompt:
ingestd/src/service.rs 3 → 6 (P9-001 visible)
truth/src/lib.rs 3 → 4 (P42-001 visible)
gateway/src/auth.rs 3 → 4 (P5-001 visible)
gateway/src/execution_loop 4 → 6 (indirect)
storaged/src/federation 3 → 4 (indirect)
Infrastructure additions
────────────────────────
* tests/real-world/scrum_master_pipeline.ts
- cloud-first ladder: kimi-k2:1t → deepseek-v3.1:671b → mistral-large-3:675b
→ gpt-oss:120b → devstral-2:123b → qwen3.5:397b (deep final thinker)
- LH_SCRUM_FORENSIC env: injects SCRUM_FORENSIC_PROMPT.md as adversarial preamble
- LH_SCRUM_PROPOSAL env: per-iter fix-wave doc override
- Confidence extraction (markdown + JSON), schema v4 KB rows with:
verdict, critical_failures_count, verified_components_count,
missing_components_count, output_format, gradient_tier
- Model trust profile written per file-accept to data/_kb/model_trust.jsonl
- Fire-and-forget POST to observer /event so by_source.scrum appears in /stats
* mcp-server/observer.ts — unchanged in shape, confirmed receiving scrum events
* ui/ — new Visual Control Plane on :3950
- Bun.serve with /data/{services,reviews,metrics,trust,overrides,findings,file,refactor_signals,search,logs/:svc,scrum_log}
- Views: MAP (D3 graph, 5 overlays) / TRACE (per-file iter timeline) /
TRAJECTORY (refactor signals + reverse index search) / METRICS (explainers
with SOURCE + GOOD lines) / KB (card grid with tooltips) / CONSOLE (per-service
journalctl tail, tabs for gateway/sidecar/observer/mcp/ctx7/auditor/langfuse)
- tryFetch always attempts JSON.parse (fix for observer returning JSON without content-type)
- renderNodeContext primitive-vs-object guard (fix for gateway /health string)
* docs/SCRUM_FIX_WAVE.md — iter-specific scope directing the scrum
* docs/SCRUM_FORENSIC_PROMPT.md — adversarial audit prompt (verdict/critical/verified schema)
* docs/SCRUM_LOOP_NOTES.md — iteration observations + fix-next-loop queue
* docs/SYSTEM_EVOLUTION_LAYERS.md — Layers 1-10 roadmap (trust profiling, execution DNA, drift sentinel, etc)
Measurements across iterations
──────────────────────────────
iter 1 (soft prompt, gpt-oss:120b): mean score 5.00/10
iter 3 (forensic, kimi-k2:1t): mean score 3.56/10 (−1.44 — bar raised)
iter 4 (same bar, post fixes): mean score 4.00/10 (+0.44 — fixes landed)
Score movement iter3→iter4: ↑5 ↓1 =12
21/21 first-attempt accept by kimi-k2:1t in iter 4
20/21 emitted forensic JSON (richer signal than markdown)
16 verified_components captured (proof-of-life, new metric)
Permission Gradient distribution: 0 auto · 16 dry_run · 4 sim · 1 block
Observer loop: by_source {scrum: 21, langfuse: 1985, phase24_audit: 1}
v1/usage: 224 requests, 477K tokens, all tracked
Signal classes per file (iter 3 → iter 4):
CONVERGING: 1 (ingestd/service.rs — fix clearly landed)
LOOPING: 4 (catalogd/registry, main, queryd/service, vectord/index_registry)
ORBITING: 1 (truth — novel findings surfacing as surface ones fix)
PLATEAU: 9 (scores flat with high confidence — diminishing returns)
MIXED: 6
Loop thesis status
──────────────────
A file's score rises only when the scrum confirms a real fix landed.
No false positives yet across 3 iterations. Fixes applied to 3 files all
raised their independent scores under the same adversarial prompt. Loop
is measurable, not hand-wavy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
root
|
4251e94531 |
Update PHASES.md: Phase 41 + Guard fixes
- Phase 41: ProfileType enum, per-type endpoints - Guard: scrumaudit.py, fixed watcher.sh + pr-reviewer.md |
||
|
root
|
f59ddbebd4 |
Phase 41: Profile System Expansion
- ProfileType enum: Execution, Retrieval, Memory, Observer - Per-type endpoints: /profiles/retrieval, /profiles/memory, /profiles/observer - profile_type field on ModelProfile - All tests pass |
||
|
root
|
e442d401d2 | Update Cargo.lock | ||
|
root
|
55f8e0fe6e |
Phase 40: Routing Engine + Policy
- RoutingEngine with RouteDecision (model_pattern → provider) - config/routing.toml: rules, fallback chain, cost gating - Per-provider Usage tracking in /v1/usage response - 12 gateway tests green |
||
|
root
|
e27a17e950 |
Phase 39: Provider Adapter Refactor
- ProviderAdapter trait with chat(), embed(), unload(), health() - OllamaAdapter wrapping existing AiClient - OpenRouterAdapter for openrouter.ai API integration - provider_key() routing by model prefix (openrouter/*, etc) |
||
|
root
|
e2ccddd8d2 | Test updates: scenarios manifest + nine_consecutive_audits | ||
|
root
|
5ff3213a37 | Update Cargo.lock | ||
|
root
|
21e8015b60 |
Phase 37: Hot-swap async + Phase 38: Universal API skeleton
- JobTracker extended with JobType::ProfileActivation + Embed - activate_profile returns job_id immediately, work spawns in background - /v1/chat, /v1/usage, /v1/sessions endpoints (OpenAI-compatible) - Langfuse trace integration (Phase 40 early deliverable) - 12 gateway unit tests green, curl gates pass |
||
|
root
|
640db8c63c |
Phase 26 — Mem0 upsert + Letta geo hot cache
Closes the two remaining 2026-era memory findings. Both are
optimizations per J's framing — not load-bearing, but good data
hygiene + future-proofing at scale.
MEM0 UPSERT (data hygiene):
Before: /seed always appended. A scenario re-running the same
operation on the same day wrote duplicate entries, inflating the
playbook corpus with near-identical rows.
Now: upsert_entry(new) inspects existing non-retired entries and
decides ADD / UPDATE / NOOP:
ADD → no matching (operation, day, city, state) tuple, append
UPDATE → match exists with different names → merge (union, stable
order), refresh timestamp, keep original playbook_id so
citations stay valid
NOOP → match exists with identical names → skip, return id
Day-granularity keying on timestamp YYYY-MM-DD means intraday
re-seeds dedup but tomorrow's same-operation is a fresh ADD. Retired
entries don't block new seeds — they're out of scope anyway.
Seed endpoint returns {outcome: {mode, playbook_id, merged_names?},
entries_after}. Append=false retains old replace-all semantics.
5 unit tests pass: first_seed_is_add, identical_reseed_is_noop,
same_day_different_names_updates_and_merges, different_day_same_op_is_add,
retired_entry_doesnt_block_new_seed.
Live verified: three successive seeds with (Alice), (Alice),
(Alice, Bob) left entry count unchanged at 1936 with merged names
{Alejandro, Lauren, Alice, Bob}. Previously would have been 3
appends.
LETTA GEO HOT CACHE (scale primitive):
Added geo_index: HashMap<(city_lower, state_upper), Vec<usize>>
alongside PlaybookMemoryState. Rebuilt on every mutation: set_entries,
retire_one, retire_on_schema_drift, upsert_entry, load_from_storage.
compute_boost_for_filtered_with_role now uses the index for O(1) geo
lookup instead of scanning all entries. At current scale (1.9K) the
scan was sub-ms; at 100K+ the scan becomes the dominant cost. The
hot cache future-proofs without adding an LRU abstraction.
Retired entries excluded from index; valid_until still checked on the
hot path since it can elapse between rebuilds.
Owns cloned PlaybookEntries in the geo_filtered vector so the state
read-lock is released before cosine scoring — avoids lock contention
on the scoring path.
Memory-findings progress: 5 of 5 shipped.
✓ Multi-strategy parallel retrieval (Phase 19 refinement)
✓ Input normalization + unified /memory/query (Phase 24 TS)
✓ Zep validity windows (Phase 25)
✓ Mem0 UPSERT (Phase 26 today)
✓ Letta geo hot cache (Phase 26 today)
All 18 playbook_memory tests pass.
|
||
|
root
|
138592dc56 |
Spec v2 — all chapters aligned with Phases 19-25
Full audit pass on devop.live/lakehouse/spec. Five chapters were
stale, one had an outright incorrect line. Scope was bigger than
ch6 alone — J asked "you want to update all" and the honest answer
was yes.
Ch 1 (Repository layout):
- mcp-server row gains /memory/query, /models/matrix, /system/summary,
observer.ts with :3800 listener
- tests/multi-agent/ row lists all new files: kb.ts, normalize.ts,
memory_query.ts, gen_scenarios.ts, gen_staffer_demo.ts, and the
colocated unit tests (kb.test.ts, normalize.test.ts)
- NEW config/ row documents models.json as the 5-tier matrix
- data/ row enumerates the four learning-loop directories:
_kb/, _playbook_lessons/, _observer/, _chunk_cache/
Ch 3 (Measurement & indexing):
- NEW "Model matrix (Phase 20)" subsection — 5-tier table (T1 hot /
T2 review / T3 overview / T4 strategic / T5 gatekeeper), per-tier
primary model, frequency, the think:false mechanical finding
called out with the 650-token reasoning-budget example
- NEW "Continuation primitive (Phase 21)" paragraph
- NEW "Per-staffer tool_level (Phase 23)" section with full/local/
basic/minimal mapping and the 46pt fill-rate delta from the 36-run
demo
Ch 7 (Scale story):
- FIX: playbook_memory growth bullet was claiming "No TTL or merge
policy" — Phase 25 added retirement via valid_until +
schema_fingerprint + /retire endpoint. Rewritten to name current
state (1936 entries, active vs retired split exposed).
Ch 8 (Error surfaces):
- Five new rows added to the failure-mode table:
* Zero-supply city → cloud rescue (Phase 22 item B) with the
Gary IN → South Bend IN concrete example
* LLM truncation → generateContinuable (Phase 21)
* Schema migration → /vectors/playbook_memory/retire (Phase 25)
* Observer unreachable → scenario silent-skip + append journal
survivability
Ch 9 (Per-staffer context):
- NEW "Staffer identity + competence-weighted retrieval (Phase 23)"
section with the competence_score formula and findNeighbors
weighted_score
- NEW "Auto-discovered reliable-performer labels" section naming
Rachel D. Lewis (18 endorsements) and Angela U. Ward (19) as
concrete output of 36-run demo
Ch 10 (A day in the life):
- Added 17:15 timeline entry — Kim using /memory/query with natural
language, regex normalizer extracting role/city/count in 0ms
- 17:00 entry updated to mention KB indexing + pathway recommendation
+ observer stream
- 22:00 entry updated to mention detectErrorCorrections nightly scan
Ch 11 (Known limits & non-goals):
- FIX: "playbook_memory compaction" bullet rewritten since retirement
is now wired; reframed as the honest Mem0 UPDATE/NOOP gap
- Added Letta hot cache deferred item with honest "cheap at 1.9K,
will bite at 100K" framing
- Added Chunking cache (Phase 21 Rust port) deferred item
- Added Observer → autotune feedback wire deferred item (Phase 26+)
Footer bumped v1 2026-04-20 → v2 2026-04-21 with Phase list.
Verified all updates live on devop.live/lakehouse/spec.
|
||
|
root
|
e0a843d1a5 |
Phase 25 — validity windows + playbook retirement
Addresses the load-bearing memory gap J flagged: playbook entries
had timestamps but no retirement semantic. When a schema migration
changed a column or a seasonal contract ended, stale playbooks kept
boosting candidates silently. Zep 2026-era finding — temporal
validity is the single highest-value memory-hygiene primitive.
SCHEMA (PlaybookEntry gains four optional fields, serde default):
schema_fingerprint — SHA-256 over dataset (column, type) tuples at
seed time. Missing = legacy entry, never
auto-retired on drift.
valid_until — RFC3339 hard expiry. compute_boost skips
entries past this moment.
retired_at — Set by retire_one or retire_on_schema_drift.
Retired entries excluded from all boost
calculations but kept in journal.
retirement_reason — Human-readable: "schema_drift: ...",
"expired: ...", "manual: ..."
RETRIEVAL PATH (compute_boost_for_filtered_with_role):
Before geo+cosine, active_entries filter removes anything retired
OR past valid_until. Uses chrono::Utc::now() once per call, no per-
entry clock queries.
NEW METHODS on PlaybookMemory:
retire_one(playbook_id, reason)
retire_on_schema_drift(city, state, current_fp, reason) — idempotent,
scopes by (city, state) so a Nashville migration doesn't touch
Chicago. Skips legacy entries with no fingerprint.
status_counts() -> (total, retired, failures)
HTTP ENDPOINTS:
POST /vectors/playbook_memory/retire
{playbook_id, reason} → retire by id
{city, state, current_schema_fingerprint, reason} → schema drift
GET /vectors/playbook_memory/status
{total, active, retired, failures}
SEED REQUEST extended with optional schema_fingerprint + valid_until
so the orchestrator (scenario.ts) can pass the current schema hash
when seeding, without a round trip through catalogd.
UNIT TESTS (5/5 pass): retire_one_marks_entry_and_persists,
retired_entries_do_not_boost, expired_valid_until_is_skipped,
schema_drift_retires_mismatched_fingerprints_only,
schema_drift_skips_other_cities.
LIVE VERIFIED: /status on current state = 1936 entries, 43 failures.
POST /retire with a sample playbook_id → "retired":1, /status now
reports active=1935, retired=1.
Memory-findings progress: 3 of 5 shipped.
✓ Multi-strategy parallel retrieval (Phase 19 refinement)
✓ Input normalization + unified /memory/query (Phase 24 TS)
✓ Zep-style validity windows (Phase 25, tonight)
⏳ Mem0 UPDATE / DELETE / NOOP ops (dedup same-(op,date) seeds)
⏳ Letta working-memory hot cache (not biting at 1.5K entries)
|
||
|
root
|
3fb3a60da4 |
Spec ch6 rewrite — 3 learning paths → 7 + honest gap list
J flagged the spec out of alignment with what's built. Ch6 now reflects the full current architecture: - Path 1 (playbook boost) — formula kept; geo+role prefilter refinement called out with measured 14× citation lift - Path 2 (pattern discovery) — unchanged - Path 3 (autotune agent) — unchanged - Path 4 (KB + pathway recommender) — Phase 22, file layout documented - Path 5 (cloud rescue on failure) — Phase 22 item B, verified stress_01 example cited - Path 6 (staffer competence-weighted retrieval) — Phase 23, competence_score formula included, cross-staffer auto- discovered worker labels (Rachel D. Lewis 18× endorsements) - Path 7 (observer outcome ingest) — Phase 24, :3800 HTTP listener + ops.jsonl append journal Input normalizer + unified /memory/query surface documented as the "seamless with whatever input" answer, with the 319ms natural-language latency number. Honest gaps kept visible in the spec itself, not hidden: - Zep validity windows (most load-bearing remaining) - Mem0 UPDATE/DELETE/NOOP ops - Letta working-memory hot cache Live at https://devop.live/lakehouse/spec#ch6 after service restart. Verified post-deploy: geo+role prefilter, 14× delta, validity windows gap all present in served HTML. |
||
|
root
|
52561d10d3 |
Input normalizer + unified memory query — "seamless with whatever input"
J asked directly: "did we implement our memory findings so that our
knowledge base and our configuration playbook [work] seamlessly with
whatever input they're given?" Honest answer tonight was "one of five
findings shipped, normalizer is the blocker." This closes that gap.
NORMALIZER (tests/multi-agent/normalize.ts):
Accepts structured JSON, natural language, or mixed. Returns canonical
NormalizedInput { role, city, state, count, client, deadline, intent,
confidence, extraction_method, missing_fields } for any downstream
consumer.
Three-tier path:
1. Structured fast-path — already-shaped input skips LLM
2. Regex path — "need 3 welders in Nashville, TN" parses without LLM.
City/state parser tightened to 1-3 capitalized words + "in {city}"
anchor preference + case-exact full-state-name variants to prevent
"Forklift Operators in Chicago" being captured as the city name
3. LLM fallback — qwen3 local with think:false + 400 max_tokens for
inputs the regex can't handle
Unit tests (tests/multi-agent/normalize.test.ts): 9/9 pass. Covers
structured fast-path, misplacement→rescue intent, state-name→abbrev
conversion, regex extraction from natural language, plural role +
full state name edge case, rescue intent keyword precedence, partial
input reporting missing fields, empty object fallthrough, async/sync
parity on clean inputs.
UNIFIED MEMORY QUERY (tests/multi-agent/memory_query.ts):
One function, five parallel fan-outs, one bundle returned:
- playbook_workers — hybrid_search via gateway with use_playbook_memory
- pathway_recommendation — KB recommender for this sig
- neighbor_signatures — K-NN sigs weighted by staffer competence
- prior_lessons — T3 overseer lessons filtered by city/state
- top_staffers — competence-sorted leaderboard
- discovered_patterns — top workers endorsed across past playbooks
for this (role, city, state)
- latency_ms — per-source + total
Every branch is best-effort: one source down doesn't break the bundle.
HTTP ENDPOINT (mcp-server/index.ts):
POST /memory/query with body {input: <anything>} → MemoryQueryResult
Returns the same shape the TS function does. Typed with types.ts for
future UI consumption.
VERIFIED:
curl POST /memory/query with structured {role,city,state,count}
→ extraction_method=structured, 10 playbook workers, top score 0.878
curl POST /memory/query with "I need 3 welders in Nashville, TN"
→ extraction_method=regex (no LLM call), 319ms total, 8 endorsements
for Lauren Gomez auto-discovered as top Nashville Welder
Honest remaining gaps (documented for next phase):
- Mem0 ADD/UPDATE/DELETE/NOOP — we still only ADD + mark_failed
- Zep validity windows — playbook entries have timestamps but no
retirement semantic
- Letta working-memory / hot cache — every query scans all 1560
playbook entries
- Memory profiles / scoped queries — global pool, no per-staffer
private subsets
2 of 5 findings now shipped (multi-strategy retrieval in Rust, input
normalization + unified query in TS). The remaining 3 are architectural
additions queued as Phase 25 items — validity windows first since it's
the most load-bearing for long-running systems.
|
||
|
root
|
b95dd86556 |
Phase 24 — observer HTTP ingest + scenario outcome streaming
Closes the gap J flagged: observer wraps MCP:3700, scenarios hit
gateway:3100 directly, observer idle at 0 ops across 3600+ cycles.
Now scenarios POST per-event outcomes to observer's new HTTP ingest
on :3800, observer consumes them alongside MCP-wrapped ops, ERROR_
ANALYZER and PLAYBOOK_BUILDER loops see the full picture.
observer.ts:
- Bun.serve() HTTP listener on OBSERVER_PORT (default 3800):
GET /health — basic + ring depth
GET /stats — total / success / failure / by_source / recent
scenario ops digest
POST /event — accept scenario outcome, shape it into ObservedOp
with source="scenario" + staffer_id + sig_hash +
event_kind + role/city/state + rescue flags
- recordExternalOp() — shared ring-buffer insert so the main analyzer
+ playbook builder don't care where the op came from
- ObservedOp extended with provenance fields
persistOp() FIX — old path POSTed to /ingest/file?name=observed_operations
which REPLACES the dataset (flagged in feedback_ingest_replace_semantics.md).
Every op was silently wiping all prior ops. Replaced with append to
data/_observer/ops.jsonl so the historical trace is durable across
analyzer cycles and process restarts.
scenario.ts:
- OBSERVER_URL env (default http://localhost:3800)
- postObserverEvent() helper with 2s AbortSignal.timeout so observer
being down doesn't block scenario flow
- Per-event POST after ctx.results.push(result), carrying staffer_id,
sig_hash (via imported computeSignature), event_kind + role + city
+ state + count + rescue_attempted / rescue_succeeded + truncated
output_summary
VERIFIED:
curl POST /event → {"accepted":true,"ring_size":1}
curl GET /stats → {"total":1,"successes":1,"by_source":{"scenario":1},
"recent_scenario_ops":[{...staffer_id,kind,role}]}
Final v3 demo leaderboard (9 runs per staffer, cumulative 3 batches):
James (local): 92.9% fill, 36.8 cites, score 0.775 — RANK 1
Maria (full): 81.0% fill, 26.2 cites, score 0.727
Sam (basic): 61.9% fill, 28.2 cites, score 0.640
Alex (minimal): 59.5% fill, 32.2 cites, score 0.631
Honest finding: Alex has MORE citations than Sam despite NO T3 and NO
rescue. Playbook inheritance alone is firing hardest when overseer is
absent. The 59.5% fill rate (up from 0% when qwen2.5 was executor)
proves cloud-exec + playbook inheritance is the floor the architecture
delivers.
Local gpt-oss:20b T3 outperforms cloud gpt-oss:120b T3 by 12pt fill
rate on this workload — cloud overseer paying latency+variance for
no measurable gain, worth flagging in next models.json tune.
|
||
|
root
|
137aed64fb |
Coherence pass — PRD/PHASES updates, config snapshot wired, unit tests
J flagged the audit: "make sure everything flows coherently, no pseudocode or unnecessary patches or ignoring any particular part of what we built." This is that pass. PRD.md updates: - Phase 19 refinement block — geo-filter + role-prefilter WIRED with citation density numbers (0.32 → 1.38, and 2 → 28 on same scenario). - Phase 20 rewrite — mistral dropped, qwen3.5 + qwen3 local hot path, think:false as the key mechanical finding, kimi-k2.6 upgrade path. - Phase 21 status block — think plumbing + cloud executor routing added after original commit. - Phase 22 item B (cloud rescue) — pivot sanitizer, rescue verified 1/3 on stress_01. - Phase 23 NEW — staffer identity + tool_level + competence-weighted retrieval + kb_staffer_report. Auto-discovered worker labels called out with real numbers (Rachel Lewis 12× across 4 staffers). - Phase 24 NEW — Observer/Autotune integration gap DOCUMENTED, not fixed. Observer has been idle at 0 ops for 3600+ cycles because scenarios hit gateway:3100 directly, bypassing MCP:3700 which the observer wraps. This is the honest "we're not using it in these tests" signal J surfaced. Fix deferred; gap visible now. PHASES.md: - Appended Phases 20-23 as checked, Phase 24 as unchecked gap. - Updated footer count: 102 unit tests across all layers. - Latest line updated with 14× citation lift + 46.4pt tool-asymmetry finding. scenario.ts: - snapshotConfig() was defined but never called. Now fires at every scenario start with a stable sha256 hash over the active model set + tool_level + cloud flags. config_snapshots.jsonl finally populates, which the error_corrections diff path needs to work correctly. kb.test.ts (new): 4 signature invariant tests — stability across unrelated fields (date, contract, staffer), sensitivity to role/city/ count changes, digest shape. All pass under `bun test`. service.rs: 6 Rust extractor tests for extract_target_geo + extract_target_role — basic, missing-state-returns-none, word boundary (civilian != city), multi-word role, absent role, quoted value parse. All pass under `cargo test -p vectord --lib extractor_tests`. Dangling items now honestly documented rather than silently pending: - Chunking cache (config/models.json SPEC, not wired) — flagged - Playbook versioning (SPEC, not wired) — flagged - Observer integration (WIRED but disconnected) — new Phase 24 |
||
|
root
|
ad0edbe29c |
Cloud kimi-k2.5 executor for weak tiers + multi-strategy playbook retrieval
Two coupled changes from the 2026 agent-memory research + tool
asymmetry findings.
SCENARIO (weak-tier cloud substitute):
qwen2.5 collapsed to 0/14 across the basic/minimal tool_levels.
Replace with cloud kimi-k2.5 on Ollama Cloud — same family as k2.6
(pro-tier locked today, on J's upgrade path). Plumb cloud flag
through ACTIVE_EXECUTOR_CLOUD / ACTIVE_REVIEWER_CLOUD into
generateContinuable so executor/reviewer can route to cloud when
tool_level requires. think:false supported by Kimi family.
Tool level mapping (revised):
full — qwen3.5 local + qwen3 local + cloud gpt-oss:120b T3 + rescue
local — qwen3.5 local + qwen3 local + local gpt-oss:20b T3 + rescue
basic — kimi-k2.5 cloud + qwen3 local + local T3, no rescue
minimal — kimi-k2.5 cloud + qwen3 local, no T3, no rescue.
Playbook inheritance alone on the decision path.
This is the honest version of J's "minimal tools still works via
inheritance" hypothesis — with the executor no longer broken at the
tokenizer level, we can actually measure whether playbook retrieval
substitutes for missing overseers.
PLAYBOOK_MEMORY (multi-strategy retrieval):
Zep / Mem0 research shows multi-strategy rerank (semantic + keyword +
graph + temporal) outperforms single-strategy cosine. Lakehouse now
has a two-tier:
1. Exact (role, city, state) match: skip cosine, assign similarity=1.0,
take up to top_k/2+1 slots. These are identity-class neighbors —
the strongest possible signal.
2. Cosine fallback within the same (city, state) but different role:
fills remaining slots.
Exposed as compute_boost_for_filtered_with_role(target_geo, target_role).
Backwards-compatible: compute_boost_for_filtered forwards with role=None
so existing callers keep their current behavior.
Service.rs wires both: extract_target_geo and extract_target_role pull
from the executor's SQL filter. grab_eq_value is factored out of
extract_target_geo so both lookups share one parser. Diagnostic log
now prints target_role alongside target_geo for every hybrid_search:
playbook_boost: boosts=88 sources=39 parsed=39 matched=5
target_geo=Some(("Nashville", "TN")) target_role=Some("Welder")
Verified: Nashville Welder query returns 5/10 boosted workers in
top_k with clean role+geo provenance.
Research sources: atlan.com Agent Memory Frameworks 2026, Mem0 paper
(arxiv 2504.19413), Zep/Graphiti LongMemEval comparison, ossinsight
Agent Memory Race 2026.
kimi-k2.6 on current key returns 403 — pro-tier upgrade required.
kimi-k2.5 is the substitute today; swap to k2.6 by renaming one line
in applyToolLevel once the subscription lands.
|
||
|
root
|
5e89407939 |
Phase 23 refinement — per-staffer tool_level variance
Staffer.tool_level now controls which subsystems a specific run gets:
full — qwen3.5 + qwen3 + cloud T3 + cloud rescue
local — qwen3.5 + qwen3 + local gpt-oss:20b T3 + rescue
basic — qwen2.5 + qwen2.5 + local T3, no rescue
minimal — qwen2.5 + qwen2.5, NO T3, NO rescue. Playbook
inheritance only.
applyToolLevel() mutates module-scoped ACTIVE_* slots each run from the
env defaults, so prior staffer's overrides never leak. Hot-path code
reads ACTIVE_EXECUTOR / ACTIVE_REVIEWER / ACTIVE_T3_DISABLED /
ACTIVE_OVERVIEW_CLOUD / ACTIVE_RETRY_ON_FAIL instead of the baked
constants.
The architectural question this answers: does playbook_memory
inheritance carry enough knowledge to let a weakly-tooled coordinator
still produce usable outcomes? "Minimal" Alex runs qwen2.5 exec + no
reviewer overseer + no cloud rescue. If Alex still fills events at a
reasonable rate, the playbook system is the real knowledge carrier —
the senior stack is nice-to-have, not the sine qua non.
Demo personas mapped:
Maria (senior, 48mo, full)
James (mid, 14mo, local)
Sam (junior, 4mo, basic)
Alex (trainee, 1mo, minimal)
Same 3 contracts (Nashville downtown, Joliet warehouse, Indianapolis
assembly) across all four → 12 runs. KB + kb_staffer_report.py
leaderboard already wired; competence_score will now reflect real tool
asymmetry instead of LLM sampling variance.
|
||
|
root
|
6b71c8e9b2 |
Phase 23 — contract terms + staffer identity + competence-weighted retrieval
Matrix-index the "who handled this" dimension so top staffers become
the training signal and juniors inherit their playbooks automatically
via the boost pipeline. Auto-discovered indicators emerge from
comparing trajectories across staffers on similar contracts — that was
always the architectural point; this wires the last piece.
ContractTerms:
- deadline, budget_total_usd, budget_per_hour_max, local_bonus_per_hour,
local_bonus_radius_mi, fill_requirement ("paramount" | "preferred")
- Attached to ScenarioSpec, propagated into T3 checkpoint + cloud
rescue prompts so cloud reasons about trade-offs (pivot within bonus
radius first; respect per-hour cap; split across cities when
fill_requirement=paramount).
Staffer:
- {id, name, tenure_months, role: senior|mid|junior|trainee}
- On ScenarioSpec; logged at scenario start; attached to KB outcome
- Recomputed StafferStats written to data/_kb/staffers.jsonl after
every run: total_runs, fill_rate, avg_turns, avg_citations,
rescue_rate, competence_score.
- Competence formula: 0.45*fill_rate + 0.20*turn_efficiency +
0.20*citation_density + 0.15*rescue_rate. Normalized to 0..1.
findNeighbors now returns weighted_score = cosine × best_staffer_competence
(floored at 0.3 so high-similarity low-competence neighbors still
surface). pathway_recommender prompt shows the top staffer's identity
so cloud knows WHOSE playbook it's synthesizing from.
Demo infrastructure:
- tests/multi-agent/gen_staffer_demo.ts: 4 personas (Maria senior,
James mid, Sam junior, Alex trainee) × 3 contracts (Nashville Welder,
Joliet Warehouse, Indianapolis Assembly). 12 scenarios total.
- scripts/run_staffer_demo.sh: runs the 12 sequentially with
LH_OVERVIEW_CLOUD=1. Post-run calls kb_staffer_report.py.
- scripts/kb_staffer_report.py: leaderboard + cross-staffer worker
overlap (names endorsed by ≥2 staffers → auto-discovered high-value
workers). Top vs bottom differential.
gen_scenarios.ts (Phase 22 generator) also now emits contract terms
on 70% of generated specs — future KB batches populate with realistic
constraint patterns instead of bare role+city+count.
Stress scenario from item A intentionally NOT the production test.
Real staffing has constraints; Nashville contract + staffer demo is
the honest test of whether the architecture produces measurable
differential between coordinator skill levels.
Demo batch launched — 12 runs × ~3min each ≈ 40min unattended. Report
emitted after batch.
|
||
|
root
|
a7fc8e2256 |
Item B — cloud-rescue retry on event failure
When a scenario event fails (drift abort or other error) and
LH_RETRY_ON_FAIL is on (default when cloud T3 is enabled), ask cloud
for a concrete pivot — new city, role, or count — then re-run the
event with the remediation's fields. Capped at 1 retry per event so a
genuinely-impossible scenario can't burn budget.
requestCloudRemediation(event, result):
- Feeds the same diagnostic bundle T3 checkpoints get (SQL filters,
row counts, SQL errors, reviewer drift reasons, gap signals).
- Prompt demands structured JSON: {retry, new_city, new_role,
new_count, rationale}.
- Cloud is instructed to pivot to NEAREST alternate city when
zero-supply detected, broaden role when uniquely scarce, reduce
count when clearly unachievable, or return retry=false when no
pivot seems viable.
EventResult additions:
- retry_attempt, retry_remediation (with rationale + cloud_model +
duration), retry_result (full inner result shape), original_event.
- If retry succeeded, it becomes the primary result and original_event
preserves what was attempted first. If retry also failed, the
primary stays the failure and retry is recorded alongside.
Sanitizer on cloud output: model sometimes emits "Hammond, IN" in
new_city with "IN" in a non-existent new_state field, producing
"Hammond, IN, IN" downstream. Split new_city on comma, take first
token as city, extract state if present after the comma. Original
event's state is the fallback.
VERIFIED on stress_01.json with LH_OVERVIEW_CLOUD=1:
Without rescue (item A baseline): 1/5 events ok
With rescue (item B): 3/5 events ok
Gary IN misplacement: drift → cloud proposed South Bend IN → retry
filled 1/1. Rationale stored in retry_remediation for forensics.
Known limits surfaced (future work):
- City-field mangling failed one rescue before the sanitizer landed;
next run will use the fix.
- Cloud picks alternate cities without knowing ground-truth supply.
Flint → Saginaw pivoted but Saginaw also had sparse Welders.
Future: expose a /vectors/supply-estimate endpoint cloud can consult
before proposing a pivot.
|
||
|
root
|
c21b261877 |
Item A — stress scenario + enriched T3 diagnostic prompt
Proves cloud passthrough works end-to-end AND fixes the diagnostic
quality problem that first run surfaced.
STRESS SCENARIO (tests/multi-agent/scenarios/stress_01.json):
Five genuinely hard events with varied failure modes:
- Gary, IN 5× Electrician: ZERO supply (city not in workers_500k)
- Peoria, IL 8× Safety Coordinator: scarce role, initial pool only 5
- Flint, MI 3× Welder: ZERO supply
- Grand Rapids, MI 4× Tool & Die Maker: scarce but solvable
- Gary, IN 1× Electrician misplacement: repeats event 1's impossibility
FIRST RUN (stress v1) — cloud passthrough works, diagnosis vague:
T3 checkpoint: "Potential drift flags for upcoming role"
Lesson: "Before dispatching, query pool status. Update turn counter..."
Generic tactical advice that doesn't address the real problem.
Root cause: T3 prompt only saw outcome summary, not the raw
SQL/pool/drift signals the executor had in its log.
DIAGNOSTIC FIX:
- Added LogEntry[] `sharedLog` parameter to runAgentFill so the caller
retains the trace even when runAgentFill throws drift-abort.
- EventResult gained `diagnostic_log` field populated on both OK and
FAIL paths.
- extractDiagnostics() pulls SQL filters, hybrid_search row counts,
SQL errors, and reviewer drift notes from the log.
- Checkpoint prompt now includes FAILURE FORENSICS block for failed
events: SQL filters attempted, row counts, errors, drift reasons,
and an explicit teaching note about zero-supply detection.
- Cross-day lesson prompt flags each event with [ZERO-SUPPLY: pivot
city needed] tag when drift reasons mention "no match"/"no
candidates"/"0 rows". PRIORITY clause in the prompt tells the model
its lesson MUST name alternate cities when that tag appears.
SECOND RUN (stress v2 with enriched prompt) — cloud diagnosis sharp:
T3 after Flint: risk="Zero candidate supply for Welder in Flint"
hint="search Welder×3 in Saginaw, MI (≈30 mi) or
expand role to Metal Fabricator"
T3 after Gary: risk="Zero supply for Electrician in Gary, IN"
hint="Pivot to Chicago, IL (≈40 min); broaden to
Electrical Technician within 60 min radius"
Lesson: specific, per-city, with distances, role-broadening
fallback, and pre-loading strategy — actionable for item B retry.
Cloud 120b call latencies consistent: 4.8-8.0s per prompt. Cloud
passthrough proven under stress.
Fill outcomes unchanged (1/5 — correct rejection of three impossible
events + one propagating JSON emission edge case on retry pivot
reasoning). The knowledge to rescue them now exists in the lesson;
item B wires the retry.
|
||
|
root
|
a663698571 |
Item 3 — geo-filtered playbook boost; diagnostic logging
ROOT CAUSE (found via instrumentation, not hunch): After a 20-scenario corpus batch, only 6/40 successful (role, city) combos ever triggered playbook_memory citations on subsequent runs. Added `playbook_boost:` tracing::info! line in vectord::service to log boost map size vs candidate pool vs match count. One query revealed: boosts=170 sources=50 parsed=50 matched=0 170 endorsed workers came back from compute_boost_for — but zero were in the 50-candidate Toledo pool. The boost map was pulling globally- ranked semantic neighbors (top-100 playbooks across ALL cities), dominated by Kansas City / Chicago / Detroit forklift playbooks the Toledo SQL filter would never admit. The mechanism was correct at the per-playbook level; the problem was pool intersection. FIX (surgical, not cap-tuning): - playbook_memory::compute_boost_for_filtered(): accepts optional (city, state) filter. When set, skips playbooks from other geos BEFORE cosine-ranking, so top-k is within the target city. - Backwards-compatible: compute_boost_for() calls the filtered variant with None — existing callers unchanged. - service::hybrid_search(): extracts target (city, state) from the executor's SQL filter via a small parser (extract_target_geo), passes to compute_boost_for_filtered. VERIFIED: Before fix: boosts=170 sources=50 parsed=50 matched=0 (0% hit) After fix: boosts=36 sources=50 parsed=50 matched=11 (22% hit) Top-k=10 now has 7/10 boosted workers with 2-3 citations each. Boost values 0.075-0.113 on cosine scores 0.67-0.74 — meaningful reorder without saturation. scripts/kb_measure.py: Aggregator that reads data/_kb/*.jsonl and playbooks/*/results.json, reports fill rate, citation density, recommender confidence trend, and zero-citation-ok combos (item 3 target signal). Used to measure before/after on bigger batches. Diagnostic logging stays — the class of "boosts computed but not matched" bug can recur if the SQL filter format ever drifts, and without the counter it's invisible. Every hybrid_search with use_playbook_memory=true now logs its boost stats. |
||
|
root
|
330cb90f99 |
Lift k cap, drop ornamental reason field, scenario generator
ITEM 1 — k CAP + REASON FIELD The hybrid_search default k was hard-coded to 10. For multi-fill events (5× expansion, 4× emergency) that's pool=10 → propose 5-of-10, half the candidates become the answer with no room for rejection. Executor prompt now instructs k to scale with target_count: k = max(count*5, 20), cap 80. Default helper bumped 10 → 20. Fill.reason dropped from required to optional. Nothing downstream ever consumed it — resolveWorkerIds, sealSale, retrospective all use candidate_id and name. Models loved to write 100-150 char justifications per fill; on 4+ fills that blew the JSON budget before the structure closed. Test 1 run result after this change: FIRST EVER 5/5 on the Riverfront Steel scenario, 13 total turns across 5 events. The event that failed last run (emergency 4×Loader with truncated reason-field continuation) now clears in 2 turns. Progression: mistral baseline: 0/5 qwen3.5 + continuation + think:false: 4/5 qwen3.5 + k=20 + no-reason: 5/5 ✓ ITEM 2 — SCENARIO GENERATOR (NOT YET TESTED E2E) tests/multi-agent/gen_scenarios.ts emits N deterministic ScenarioSpecs with varied clients (15 companies), cities (20 Midwest cities known to exist in workers_500k), role mixes (14 industrial staffing roles, weighted realistic), and event sequences. Each gets a unique sig_hash so the KB populates with distinct neighbor signatures. scripts/run_kb_batch.sh runs all generated specs sequentially against scenario.ts, logs per-scenario outcomes, and reports KB state at the end. Each run takes ~2-4min; 20-30 scenarios = 1-2hr unattended. Next: test the generator+batch on a small N (3-5) to verify KB populates correctly and pathway recommendations start getting neighbor signal instead of cold-starts. Then item 3 (Rust re-weighting of hybrid_search by playbook_memory success). |
||
|
root
|
9c1400d738 |
Phase 22 — Internal Knowledge Library (KB)
Meta-layer over Phase 19 playbook_memory. Phase 19 answers "which WORKERS worked for this event"; KB answers "which CONFIG worked for this playbook signature" — model choice, budget hints, pathway notes, error corrections. tests/multi-agent/kb.ts: - computeSignature(): stable sha256 hash of the (kind, role, count, city, state) tuple sequence. Same scenario shape → same sig. - indexRun(): extracts sig, embeds spec digest via sidecar, appends outcome record, upserts signature to data/_kb/signatures.jsonl. - findNeighbors(): cosine-ranks the k most-similar signatures from prior runs for a target spec. - detectErrorCorrections(): scans outcomes for same-sig fail→succeed pairs, diffs the model set, logs to error_corrections.jsonl. - recommendFor(): feeds target digest + k-NN neighbors + recent corrections to the overview model, gets back a structured JSON recommendation (top_models, budget_hints, pathway_notes), appends to pathway_recommendations.jsonl. JSON-shape constrained so the executor can inherit it mechanically. - loadRecommendation(): at scenario start, pulls newest rec matching this sig (or nearest). scenario.ts: - Reads KB recommendation at startup (alongside prior lessons). - Injects pathway_notes into guidanceFor() executor context. - After retrospective, indexes the run + synthesizes next rec. Cold-start behavior: first run with no history writes a low-confidence "no prior data" rec so the signal that something was attempted is captured. Second run gets "low confidence, 0 neighbors" until a third distinct sig gives the embedder something to compare against — hence the upcoming scenario generator. VERIFIED: - data/_kb/ populated after one scenario run: 1 outcome (sig=4674…, 4/5 ok, 16 turns total), 1 signature, 2 recs (cold + post-run). - Recommendation JSON-parsed cleanly from gpt-oss:20b overview model. PRD Phase 22 added with file layout, cycle description, and the rationale for file-based MVP → Rust port progression that matches how Phase 21 primitives shipped. What's NOT here yet (batched follow-ups per J's request, tested between each): - Lift the k=10 hybrid_search cap to adaptive k=max(count*5, 20) - Scenario generator to bulk-populate KB with varied signatures - Rust re-weighting: push playbook_memory success signal INTO hybrid_search scoring, not just post-hoc boost |