checkpoint: matrix-agent-validated (2026-04-25)

Architectural snapshot of the lakehouse codebase at the point where the
full matrix-driven agent loop with Mem0 versioning + deletion was
validated end-to-end.

WHAT THIS REPO IS
A clean single-commit snapshot of the lakehouse code. Heavy test data
(.parquet datasets, vector indexes) excluded — see REPLICATION.md for
regen path. Full lakehouse history at git.agentview.dev/profit/lakehouse.

WHAT WAS PROVEN
- Vector retrieval across multi-corpora matrix (chicago_permits + entity
  briefs + sec_tickers + distilled procedural + llm_team runs)
- Observer hand-review (cloud + heuristic fallback) gating each candidate
- Local-model agent loop (qwen3.5:latest) with tool use + scratchpad
- Playbook seal on success → next-iter retrieval surfaces it as preamble
- Mem0 versioning + deletion in pathway_memory:
    * UPSERT: ADD on new workflow, UPDATE bumps replay_count on identical
    * REVISE: chains versions, parent.superseded_at + superseded_by stamped
    * RETIRE: marks specific trace retired with reason, excluded from retrieval
    * HISTORY: walks chain root→tip, cycle-safe

KEY DIRECTORIES
- crates/vectord/src/pathway_memory.rs — Mem0 ops live here
- crates/vectord/src/playbook_memory.rs — original Mem0 reference
- tests/agent_test/ — local-model agent harness + PRD + session archives
- scripts/dump_raw_corpus.sh — MinIO bucket dump (raw test corpus)
- scripts/vectorize_raw_corpus.ts — corpus → vector indexes
- scripts/analyze_chicago_contracts.ts — real inference pipeline
- scripts/seal_agent_playbook.ts — Mem0 upsert from agent traces

Replication: see REPLICATION.md for Debian 13 clean install + cloud-only
adaptation (no local Ollama).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
profit
2026-04-25 19:43:27 -05:00
co-authored by Claude Opus 4.7
commit ac01fffd9a
325 changed files with 82720 additions and 0 deletions
@@ -0,0 +1,26 @@
worker_id,name,role,city,state,email,phone,skills,certifications,availability,reliability,archetype
W-1001,Sarah Johnson,Forklift Operator,Chicago,IL,sarah.johnson@example.com,(312) 555-0101,pallet jack|hazmat|loading dock,OSHA-10|Forklift,0.92,0.88,reliable
W-1002,Michael Chen,Welder,Toledo,OH,m.chen@example.com,(419) 555-0234,TIG|MIG|blueprint reading|grinder,OSHA-10|AWS D1.1,0.71,0.94,specialist
W-1003,Maria Rodriguez,Warehouse Associate,Chicago,IL,maria.r@example.com,(312) 555-0311,inventory|RF scanner|pick-to-light,OSHA-10,0.85,0.80,reliable
W-1004,David Park,Machine Operator,Detroit,MI,dpark@example.com,(313) 555-0412,CNC|gauge R&R|lean manufacturing,OSHA-10|First Aid,0.60,0.91,specialist
W-1005,Jennifer Lopez,Loader,Chicago,IL,j.lopez@example.com,(312) 555-0501,loading dock|team lead|cold storage,OSHA-10|Forklift,0.95,0.76,communicator
W-1006,Robert Williams,Maintenance Tech,Milwaukee,WI,rwilliams@example.com,(414) 555-0612,electrical|PLC|hydraulics|CMMS,OSHA-10|LOTO,0.45,0.93,specialist
W-1007,Amanda Taylor,Quality Tech,Toledo,OH,a.taylor@example.com,(419) 555-0701,ISO 9001|calibration|root cause analysis,OSHA-10|Six Sigma Green,0.80,0.87,leader
W-1008,Carlos Mendoza,Electrician,Chicago,IL,c.mendoza@example.com,(312) 555-0808,conduit|motor controls|troubleshooting,OSHA-30|NEC,0.88,0.96,specialist
W-1009,Kim Nguyen,Forklift Operator,Detroit,MI,k.nguyen@example.com,(313) 555-0915,pallet jack|shipping|team lead,OSHA-10|Forklift,0.73,0.82,flexible
W-1010,James O'Brien,Welder,Milwaukee,WI,jobrien@example.com,(414) 555-1021,TIG|pipe welding|confined space,OSHA-10|AWS D1.1,0.90,0.79,reliable
W-1011,Priya Patel,Production Worker,Chicago,IL,priya.p@example.com,(312) 555-1118,line work|first article|labeling,OSHA-10,0.66,0.85,reliable
W-1012,Thomas Anderson,Material Handler,Toledo,OH,tanderson@example.com,(419) 555-1225,RF scanner|pallet jack|receiving,OSHA-10|Forklift,0.82,0.77,flexible
W-1013,Lisa Nakamura,Warehouse Associate,Detroit,MI,l.nakamura@example.com,(313) 555-1312,inventory|Excel|packaging,OSHA-10,0.77,0.89,communicator
W-1014,Brandon Moore,Machine Operator,Milwaukee,WI,b.moore@example.com,(414) 555-1405,CNC|SPC|conveyor ops,OSHA-10,0.56,0.94,specialist
W-1015,Emily Zhang,Assembler,Chicago,IL,emily.z@example.com,(312) 555-1502,assembly|gauge R&R|line lead,OSHA-10|Six Sigma Yellow,0.91,0.81,leader
W-1016,Marcus Johnson,Forklift Operator,Toledo,OH,mjohnson@example.com,(419) 555-1609,pallet jack|hazmat|overhead crane,OSHA-10|Forklift|Hazmat,0.68,0.88,reliable
W-1017,Anita Brooks,Line Lead,Detroit,MI,a.brooks@example.com,(313) 555-1715,team lead|training|SPC,OSHA-30|Six Sigma Green,0.52,0.97,leader
W-1018,Dmitri Volkov,Welder,Chicago,IL,d.volkov@example.com,(312) 555-1820,TIG|MIG|blueprint reading,OSHA-10|AWS D1.1|Confined Space,0.84,0.83,specialist
W-1019,Rachel Kim,Loader,Milwaukee,WI,r.kim@example.com,(414) 555-1911,loading dock|first aid|bilingual,OSHA-10|First Aid,0.79,0.84,communicator
W-1020,Samuel Park,Maintenance Tech,Toledo,OH,spark@example.com,(419) 555-2012,electrical|PLC|troubleshooting|CMMS,OSHA-10|LOTO,0.70,0.92,specialist
W-1021,Jordan Williams,Quality Tech,Detroit,MI,j.williams@example.com,(313) 555-2117,ISO 9001|calibration|SPC,OSHA-10|Six Sigma Green,0.83,0.86,leader
W-1022,Natalia Soto,Warehouse Associate,Chicago,IL,n.soto@example.com,(312) 555-2221,RF scanner|inventory|team lead,OSHA-10,0.89,0.90,communicator
W-1023,Henry Chen,Machine Operator,Milwaukee,WI,h.chen@example.com,(414) 555-2318,CNC|conveyor ops|root cause,OSHA-10,0.64,0.93,specialist
W-1024,Ava Martinez,Forklift Operator,Toledo,OH,a.martinez@example.com,(419) 555-2411,pallet jack|bilingual|cold storage,OSHA-10|Forklift,0.87,0.80,flexible
W-1025,Tyler Rodriguez,Production Worker,Chicago,IL,t.rodriguez@example.com,(312) 555-2515,line work|packaging|quality inspection,OSHA-10,0.74,0.86,reliable
1 worker_id name role city state email phone skills certifications availability reliability archetype
2 W-1001 Sarah Johnson Forklift Operator Chicago IL sarah.johnson@example.com (312) 555-0101 pallet jack|hazmat|loading dock OSHA-10|Forklift 0.92 0.88 reliable
3 W-1002 Michael Chen Welder Toledo OH m.chen@example.com (419) 555-0234 TIG|MIG|blueprint reading|grinder OSHA-10|AWS D1.1 0.71 0.94 specialist
4 W-1003 Maria Rodriguez Warehouse Associate Chicago IL maria.r@example.com (312) 555-0311 inventory|RF scanner|pick-to-light OSHA-10 0.85 0.80 reliable
5 W-1004 David Park Machine Operator Detroit MI dpark@example.com (313) 555-0412 CNC|gauge R&R|lean manufacturing OSHA-10|First Aid 0.60 0.91 specialist
6 W-1005 Jennifer Lopez Loader Chicago IL j.lopez@example.com (312) 555-0501 loading dock|team lead|cold storage OSHA-10|Forklift 0.95 0.76 communicator
7 W-1006 Robert Williams Maintenance Tech Milwaukee WI rwilliams@example.com (414) 555-0612 electrical|PLC|hydraulics|CMMS OSHA-10|LOTO 0.45 0.93 specialist
8 W-1007 Amanda Taylor Quality Tech Toledo OH a.taylor@example.com (419) 555-0701 ISO 9001|calibration|root cause analysis OSHA-10|Six Sigma Green 0.80 0.87 leader
9 W-1008 Carlos Mendoza Electrician Chicago IL c.mendoza@example.com (312) 555-0808 conduit|motor controls|troubleshooting OSHA-30|NEC 0.88 0.96 specialist
10 W-1009 Kim Nguyen Forklift Operator Detroit MI k.nguyen@example.com (313) 555-0915 pallet jack|shipping|team lead OSHA-10|Forklift 0.73 0.82 flexible
11 W-1010 James O'Brien Welder Milwaukee WI jobrien@example.com (414) 555-1021 TIG|pipe welding|confined space OSHA-10|AWS D1.1 0.90 0.79 reliable
12 W-1011 Priya Patel Production Worker Chicago IL priya.p@example.com (312) 555-1118 line work|first article|labeling OSHA-10 0.66 0.85 reliable
13 W-1012 Thomas Anderson Material Handler Toledo OH tanderson@example.com (419) 555-1225 RF scanner|pallet jack|receiving OSHA-10|Forklift 0.82 0.77 flexible
14 W-1013 Lisa Nakamura Warehouse Associate Detroit MI l.nakamura@example.com (313) 555-1312 inventory|Excel|packaging OSHA-10 0.77 0.89 communicator
15 W-1014 Brandon Moore Machine Operator Milwaukee WI b.moore@example.com (414) 555-1405 CNC|SPC|conveyor ops OSHA-10 0.56 0.94 specialist
16 W-1015 Emily Zhang Assembler Chicago IL emily.z@example.com (312) 555-1502 assembly|gauge R&R|line lead OSHA-10|Six Sigma Yellow 0.91 0.81 leader
17 W-1016 Marcus Johnson Forklift Operator Toledo OH mjohnson@example.com (419) 555-1609 pallet jack|hazmat|overhead crane OSHA-10|Forklift|Hazmat 0.68 0.88 reliable
18 W-1017 Anita Brooks Line Lead Detroit MI a.brooks@example.com (313) 555-1715 team lead|training|SPC OSHA-30|Six Sigma Green 0.52 0.97 leader
19 W-1018 Dmitri Volkov Welder Chicago IL d.volkov@example.com (312) 555-1820 TIG|MIG|blueprint reading OSHA-10|AWS D1.1|Confined Space 0.84 0.83 specialist
20 W-1019 Rachel Kim Loader Milwaukee WI r.kim@example.com (414) 555-1911 loading dock|first aid|bilingual OSHA-10|First Aid 0.79 0.84 communicator
21 W-1020 Samuel Park Maintenance Tech Toledo OH spark@example.com (419) 555-2012 electrical|PLC|troubleshooting|CMMS OSHA-10|LOTO 0.70 0.92 specialist
22 W-1021 Jordan Williams Quality Tech Detroit MI j.williams@example.com (313) 555-2117 ISO 9001|calibration|SPC OSHA-10|Six Sigma Green 0.83 0.86 leader
23 W-1022 Natalia Soto Warehouse Associate Chicago IL n.soto@example.com (312) 555-2221 RF scanner|inventory|team lead OSHA-10 0.89 0.90 communicator
24 W-1023 Henry Chen Machine Operator Milwaukee WI h.chen@example.com (414) 555-2318 CNC|conveyor ops|root cause OSHA-10 0.64 0.93 specialist
25 W-1024 Ava Martinez Forklift Operator Toledo OH a.martinez@example.com (419) 555-2411 pallet jack|bilingual|cold storage OSHA-10|Forklift 0.87 0.80 flexible
26 W-1025 Tyler Rodriguez Production Worker Chicago IL t.rodriguez@example.com (312) 555-2515 line work|packaging|quality inspection OSHA-10 0.74 0.86 reliable