phase 44 (part 1): migrate TS callers to /v1/chat + add regression guard
lakehouse/auditor 16 blocking issues: cloud: claim not backed — "Verified end-to-end:"

Migrates the four TypeScript /generate callers to the gateway's
/v1/chat surface so every LLM call lands on /v1/usage and Langfuse:

  tests/multi-agent/agent.ts::generate()      provider="ollama"
  tests/agent_test/agent_harness.ts::callAgent provider="ollama"
  bot/propose.ts::generateProposal             provider="ollama_cloud"
  mcp-server/observer.ts (error analysis)      provider="ollama"

Each migration follows the same pattern as the prior generateCloud()
migration (already on /v1/chat from 2026-04-24): replace
`fetch(SIDECAR/generate)` with `fetch(GATEWAY/v1/chat)`, swap the
prompt-style body for OpenAI-compat messages array, extract
content from `choices[0].message.content` instead of `text`.

Same upstream models in every case — gateway is the new home for
the call, transport otherwise unchanged.

Adds scripts/check_phase44_callers.sh — fail-loud regression guard
that exits non-zero if any non-adapter file fetches /generate or
api/generate. Adapter files (crates/gateway, crates/aibridge,
sidecar/) are exempt. Pre-tightening regex flagged prose mentions
in comments; the shipped regex requires `fetch(...)` or
`client.post(...)` shape so comments don't trip it.

Verification:
  bun build mcp-server/observer.ts                       compiles
  bun build tests/multi-agent/agent.ts                   compiles
  bun build tests/agent_test/agent_harness.ts            compiles
  bun build bot/propose.ts                               compiles
  ./scripts/check_phase44_callers.sh                     ✅ clean
  systemctl restart lakehouse-observer                   active

Phase 44 part 2 (deferred):
  - crates/aibridge/src/client.rs:118 still posts to sidecar /generate
    directly. AiClient is the foundational Rust LLM caller used by
    8+ vectord modules; migrating it is a workspace-wide refactor
    that needs its own commit. Plan: keep AiClient as the local-
    transport layer for the gateway's `provider=ollama` arm, but
    introduce a thin `/v1/chat` wrapper for external callers (vectord
    autotune, agent, rag, refresh, supervisor, playbook_memory).
  - tests/real-world/hard_task_escalation.ts: comment mentions
    /api/generate but doesn't actually call it. Comment is left
    intentionally as historical context; regex no longer flags it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-27 07:33:06 -05:00
co-authored by Claude Opus 4.7
parent bfe1ea9d1c
commit f6af0fd409
5 changed files with 108 additions and 26 deletions
+8 -7
View File
@@ -161,16 +161,17 @@ const TOOL_SCHEMA = `Available tools (call by emitting JSON like: {"tool": "name
// ─── AGENT LOOP ───
async function callAgent(messages: Array<{role: string; content: string}>): Promise<string> {
// think:false disables hidden reasoning so all generated tokens go to
// visible response. qwen3.5:latest defaults to thinking and silently
// burns the token budget otherwise.
const r = await fetch(`${SIDECAR}/generate`, {
// Phase 44 migration (2026-04-27): /v1/chat instead of direct sidecar
// /generate so /v1/usage tracks the call, Langfuse traces it.
// think:false still disables hidden reasoning so generated tokens
// go to visible response — qwen3.5:latest defaults to thinking.
const r = await fetch(`${GATEWAY}/v1/chat`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
model: AGENT_MODEL,
prompt: messages.map(m => `${m.role.toUpperCase()}:\n${m.content}`).join("\n\n") + "\n\nASSISTANT:\n",
stream: false,
provider: "ollama",
messages,
max_tokens: 1500,
think: false,
}),
@@ -178,7 +179,7 @@ async function callAgent(messages: Array<{role: string; content: string}>): Prom
});
if (!r.ok) throw new Error(`agent ${r.status}: ${(await r.text()).slice(0, 200)}`);
const j: any = await r.json();
return String(j.text ?? j.response ?? "").trim();
return String(j?.choices?.[0]?.message?.content ?? "").trim();
}
function extractToolCall(response: string): { tool: string; args: any } | null {
+13 -5
View File
@@ -372,26 +372,34 @@ export async function generate(model: string, prompt: string, opts: {
bypass_budget?: boolean;
think?: boolean;
} = {}): Promise<string> {
// Phase 44 migration (2026-04-27): was hitting `${SIDECAR}/generate`
// directly, bypassing the gateway's /v1/usage accounting + Langfuse
// tracing. Now flows through /v1/chat with provider="ollama" so
// every local call is observable + auditable. Sidecar transport is
// unchanged — gateway just owns the call.
assertContextBudget(model, prompt, {
system: opts.system,
max_tokens: opts.max_tokens,
bypass: opts.bypass_budget,
});
const messages: Array<{ role: string; content: string }> = [];
if (opts.system) messages.push({ role: "system", content: opts.system });
messages.push({ role: "user", content: prompt });
const body: Record<string, any> = {
model,
prompt,
messages,
provider: "ollama",
temperature: opts.temperature ?? 0.3,
max_tokens: opts.max_tokens ?? 800,
};
if (opts.system) body.system = opts.system;
if (opts.think !== undefined) body.think = opts.think;
const r = await http<any>("POST", `${SIDECAR}/generate`, body);
const text = typeof r.text === "string" ? r.text : "";
const r = await http<any>("POST", `${GATEWAY}/v1/chat`, body);
const text = r?.choices?.[0]?.message?.content ?? "";
// Do NOT throw on empty. Thinking models (gpt-oss, qwen3.5) burn the
// max_tokens budget on hidden reasoning and emit "" when budget was
// too tight. generateContinuable detects empty + continues with more
// budget. Callers that expected non-empty can check themselves.
return text;
return typeof text === "string" ? text : "";
}
// Cloud generate — routes through the lakehouse gateway's /v1/chat