checkpoint: matrix-agent-validated (2026-04-25)

Architectural snapshot of the lakehouse codebase at the point where the full matrix-driven agent loop with Mem0 versioning + deletion was validated end-to-end. WHAT THIS REPO IS A clean single-commit snapshot of the lakehouse code. Heavy test data (.parquet datasets, vector indexes) excluded — see REPLICATION.md for regen path. Full lakehouse history at git.agentview.dev/profit/lakehouse. WHAT WAS PROVEN - Vector retrieval across multi-corpora matrix (chicago_permits + entity briefs + sec_tickers + distilled procedural + llm_team runs) - Observer hand-review (cloud + heuristic fallback) gating each candidate - Local-model agent loop (qwen3.5:latest) with tool use + scratchpad - Playbook seal on success → next-iter retrieval surfaces it as preamble - Mem0 versioning + deletion in pathway_memory: * UPSERT: ADD on new workflow, UPDATE bumps replay_count on identical * REVISE: chains versions, parent.superseded_at + superseded_by stamped * RETIRE: marks specific trace retired with reason, excluded from retrieval * HISTORY: walks chain root→tip, cycle-safe KEY DIRECTORIES - crates/vectord/src/pathway_memory.rs — Mem0 ops live here - crates/vectord/src/playbook_memory.rs — original Mem0 reference - tests/agent_test/ — local-model agent harness + PRD + session archives - scripts/dump_raw_corpus.sh — MinIO bucket dump (raw test corpus) - scripts/vectorize_raw_corpus.ts — corpus → vector indexes - scripts/analyze_chicago_contracts.ts — real inference pipeline - scripts/seal_agent_playbook.ts — Mem0 upsert from agent traces Replication: see REPLICATION.md for Debian 13 clean install + cloud-only adaptation (no local Ollama). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 19:43:27 -05:00
commit ac01fffd9a
325 changed files with 82720 additions and 0 deletions
@@ -0,0 +1,23 @@
+{
+  "id": "6d0002ef-28de-4b92-abed-0bfb7ab5fb6f",
+  "name": "candidates",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/candidates.parquet",
+      "size_bytes": 10592165,
+      "created_at": "2026-03-28T01:54:06.898505222Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:06.898506121Z",
+  "updated_at": "2026-03-28T01:54:06.898506121Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "73a4ef71-5b2a-4124-a2c2-fcefdde99624",
+  "name": "job_orders",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/job_orders.parquet",
+      "size_bytes": 905534,
+      "created_at": "2026-03-28T01:54:07.007332060Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:07.007332775Z",
+  "updated_at": "2026-03-28T01:54:07.007332775Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "b56694bb-f4f7-447d-b4d6-ac89103a6a3a",
+  "name": "call_log",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/call_log.parquet",
+      "size_bytes": 35951077,
+      "created_at": "2026-03-28T01:54:15.457573021Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:15.457573701Z",
+  "updated_at": "2026-03-28T01:54:15.457573701Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "daa5a11b-6898-40ba-a811-3bd4b6f5b750",
+  "name": "email_log",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/email_log.parquet",
+      "size_bytes": 16768671,
+      "created_at": "2026-03-28T01:54:17.356056562Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:17.356057529Z",
+  "updated_at": "2026-03-28T01:54:17.356057529Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "dd396ee4-3a1a-4939-af60-84d409eebed4",
+  "name": "clients",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/clients.parquet",
+      "size_bytes": 21971,
+      "created_at": "2026-03-28T01:54:06.904379300Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:06.904379793Z",
+  "updated_at": "2026-03-28T01:54:06.904379793Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "f85285e3-111a-4a55-aa58-05fd740ed862",
+  "name": "timesheets",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/timesheets.parquet",
+      "size_bytes": 17539932,
+      "created_at": "2026-03-28T01:54:11.947773535Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:11.947778065Z",
+  "updated_at": "2026-03-28T01:54:11.947778065Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}
@@ -0,0 +1,23 @@
+{
+  "id": "ffa72582-dc11-4454-a46e-35839a7d04bb",
+  "name": "placements",
+  "schema_fingerprint": "auto",
+  "objects": [
+    {
+      "bucket": "data",
+      "key": "datasets/placements.parquet",
+      "size_bytes": 1213820,
+      "created_at": "2026-03-28T01:54:07.109218264Z"
+    }
+  ],
+  "created_at": "2026-03-28T01:54:07.109219070Z",
+  "updated_at": "2026-03-28T01:54:07.109219070Z",
+  "description": "",
+  "owner": "",
+  "sensitivity": null,
+  "columns": [],
+  "lineage": null,
+  "freshness": null,
+  "tags": [],
+  "row_count": null
+}