profit/lakehouse
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Phase 23 refinement — per-staffer tool_level variance

Browse Source 
Staffer.tool_level now controls which subsystems a specific run gets:

  full     — qwen3.5 + qwen3 + cloud T3 + cloud rescue
  local    — qwen3.5 + qwen3 + local gpt-oss:20b T3 + rescue
  basic    — qwen2.5 + qwen2.5 + local T3, no rescue
  minimal  — qwen2.5 + qwen2.5, NO T3, NO rescue. Playbook
             inheritance only.

applyToolLevel() mutates module-scoped ACTIVE_* slots each run from the
env defaults, so prior staffer's overrides never leak. Hot-path code
reads ACTIVE_EXECUTOR / ACTIVE_REVIEWER / ACTIVE_T3_DISABLED /
ACTIVE_OVERVIEW_CLOUD / ACTIVE_RETRY_ON_FAIL instead of the baked
constants.

The architectural question this answers: does playbook_memory
inheritance carry enough knowledge to let a weakly-tooled coordinator
still produce usable outcomes? "Minimal" Alex runs qwen2.5 exec + no
reviewer overseer + no cloud rescue. If Alex still fills events at a
reasonable rate, the playbook system is the real knowledge carrier —
the senior stack is nice-to-have, not the sine qua non.

Demo personas mapped:
  Maria (senior, 48mo, full)
  James (mid, 14mo, local)
  Sam (junior, 4mo, basic)
  Alex (trainee, 1mo, minimal)

Same 3 contracts (Nashville downtown, Joliet warehouse, Indianapolis
assembly) across all four → 12 runs. KB + kb_staffer_report.py
leaderboard already wired; competence_score will now reflect real tool
asymmetry instead of LLM sampling variance.
This commit is contained in:
root
2026-04-20 22:50:05 -05:00
parent 6b71c8e9b2
commit 5e89407939
14 changed files with 119 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/multi-agent/gen_staffer_demo.ts
+8 -4
View File
@@ -8,11 +8,15 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
// Per-staffer tool_level mirrors the real-world asymmetry: senior gets
// the most powerful stack, trainee gets the least. The architectural
// question is whether the playbook inheritance is strong enough to let
// the trainee still produce usable outcomes when the big tools are off.
const STAFFERS = [
{ id: "S-001", name: "Maria Chen", tenure_months: 48, role: "senior" as const },
{ id: "S-002", name: "James Park", tenure_months: 14, role: "mid" as const },
{ id: "S-003", name: "Sam Torres", tenure_months: 4, role: "junior" as const },
{ id: "S-004", name: "Alex Rivera", tenure_months: 1, role: "trainee" as const },
{ id: "S-001", name: "Maria Chen", tenure_months: 48, role: "senior" as const, tool_level: "full" as const },
{ id: "S-002", name: "James Park", tenure_months: 14, role: "mid" as const, tool_level: "local" as const },
{ id: "S-003", name: "Sam Torres", tenure_months: 4, role: "junior" as const, tool_level: "basic" as const },
{ id: "S-004", name: "Alex Rivera", tenure_months: 1, role: "trainee" as const, tool_level: "minimal" as const },
];
// Three contract shapes — one downtown assembly, one warehouse ramp,