phase 44 (part 1): migrate TS callers to /v1/chat + add regression guard
lakehouse/auditor 16 blocking issues: cloud: claim not backed — "Verified end-to-end:"
lakehouse/auditor 16 blocking issues: cloud: claim not backed — "Verified end-to-end:"
Migrates the four TypeScript /generate callers to the gateway's /v1/chat surface so every LLM call lands on /v1/usage and Langfuse: tests/multi-agent/agent.ts::generate() provider="ollama" tests/agent_test/agent_harness.ts::callAgent provider="ollama" bot/propose.ts::generateProposal provider="ollama_cloud" mcp-server/observer.ts (error analysis) provider="ollama" Each migration follows the same pattern as the prior generateCloud() migration (already on /v1/chat from 2026-04-24): replace `fetch(SIDECAR/generate)` with `fetch(GATEWAY/v1/chat)`, swap the prompt-style body for OpenAI-compat messages array, extract content from `choices[0].message.content` instead of `text`. Same upstream models in every case — gateway is the new home for the call, transport otherwise unchanged. Adds scripts/check_phase44_callers.sh — fail-loud regression guard that exits non-zero if any non-adapter file fetches /generate or api/generate. Adapter files (crates/gateway, crates/aibridge, sidecar/) are exempt. Pre-tightening regex flagged prose mentions in comments; the shipped regex requires `fetch(...)` or `client.post(...)` shape so comments don't trip it. Verification: bun build mcp-server/observer.ts compiles bun build tests/multi-agent/agent.ts compiles bun build tests/agent_test/agent_harness.ts compiles bun build bot/propose.ts compiles ./scripts/check_phase44_callers.sh ✅ clean systemctl restart lakehouse-observer active Phase 44 part 2 (deferred): - crates/aibridge/src/client.rs:118 still posts to sidecar /generate directly. AiClient is the foundational Rust LLM caller used by 8+ vectord modules; migrating it is a workspace-wide refactor that needs its own commit. Plan: keep AiClient as the local- transport layer for the gateway's `provider=ollama` arm, but introduce a thin `/v1/chat` wrapper for external callers (vectord autotune, agent, rag, refresh, supervisor, playbook_memory). - tests/real-world/hard_task_escalation.ts: comment mentions /api/generate but doesn't actually call it. Comment is left intentionally as historical context; regex no longer flags it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -161,16 +161,17 @@ const TOOL_SCHEMA = `Available tools (call by emitting JSON like: {"tool": "name
|
||||
// ─── AGENT LOOP ───
|
||||
|
||||
async function callAgent(messages: Array<{role: string; content: string}>): Promise<string> {
|
||||
// think:false disables hidden reasoning so all generated tokens go to
|
||||
// visible response. qwen3.5:latest defaults to thinking and silently
|
||||
// burns the token budget otherwise.
|
||||
const r = await fetch(`${SIDECAR}/generate`, {
|
||||
// Phase 44 migration (2026-04-27): /v1/chat instead of direct sidecar
|
||||
// /generate so /v1/usage tracks the call, Langfuse traces it.
|
||||
// think:false still disables hidden reasoning so generated tokens
|
||||
// go to visible response — qwen3.5:latest defaults to thinking.
|
||||
const r = await fetch(`${GATEWAY}/v1/chat`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model: AGENT_MODEL,
|
||||
prompt: messages.map(m => `${m.role.toUpperCase()}:\n${m.content}`).join("\n\n") + "\n\nASSISTANT:\n",
|
||||
stream: false,
|
||||
provider: "ollama",
|
||||
messages,
|
||||
max_tokens: 1500,
|
||||
think: false,
|
||||
}),
|
||||
@@ -178,7 +179,7 @@ async function callAgent(messages: Array<{role: string; content: string}>): Prom
|
||||
});
|
||||
if (!r.ok) throw new Error(`agent ${r.status}: ${(await r.text()).slice(0, 200)}`);
|
||||
const j: any = await r.json();
|
||||
return String(j.text ?? j.response ?? "").trim();
|
||||
return String(j?.choices?.[0]?.message?.content ?? "").trim();
|
||||
}
|
||||
|
||||
function extractToolCall(response: string): { tool: string; args: any } | null {
|
||||
|
||||
@@ -372,26 +372,34 @@ export async function generate(model: string, prompt: string, opts: {
|
||||
bypass_budget?: boolean;
|
||||
think?: boolean;
|
||||
} = {}): Promise<string> {
|
||||
// Phase 44 migration (2026-04-27): was hitting `${SIDECAR}/generate`
|
||||
// directly, bypassing the gateway's /v1/usage accounting + Langfuse
|
||||
// tracing. Now flows through /v1/chat with provider="ollama" so
|
||||
// every local call is observable + auditable. Sidecar transport is
|
||||
// unchanged — gateway just owns the call.
|
||||
assertContextBudget(model, prompt, {
|
||||
system: opts.system,
|
||||
max_tokens: opts.max_tokens,
|
||||
bypass: opts.bypass_budget,
|
||||
});
|
||||
const messages: Array<{ role: string; content: string }> = [];
|
||||
if (opts.system) messages.push({ role: "system", content: opts.system });
|
||||
messages.push({ role: "user", content: prompt });
|
||||
const body: Record<string, any> = {
|
||||
model,
|
||||
prompt,
|
||||
messages,
|
||||
provider: "ollama",
|
||||
temperature: opts.temperature ?? 0.3,
|
||||
max_tokens: opts.max_tokens ?? 800,
|
||||
};
|
||||
if (opts.system) body.system = opts.system;
|
||||
if (opts.think !== undefined) body.think = opts.think;
|
||||
const r = await http<any>("POST", `${SIDECAR}/generate`, body);
|
||||
const text = typeof r.text === "string" ? r.text : "";
|
||||
const r = await http<any>("POST", `${GATEWAY}/v1/chat`, body);
|
||||
const text = r?.choices?.[0]?.message?.content ?? "";
|
||||
// Do NOT throw on empty. Thinking models (gpt-oss, qwen3.5) burn the
|
||||
// max_tokens budget on hidden reasoning and emit "" when budget was
|
||||
// too tight. generateContinuable detects empty + continues with more
|
||||
// budget. Callers that expected non-empty can check themselves.
|
||||
return text;
|
||||
return typeof text === "string" ? text : "";
|
||||
}
|
||||
|
||||
// Cloud generate — routes through the lakehouse gateway's /v1/chat
|
||||
|
||||
Reference in New Issue
Block a user