Compare commits

...
Author SHA1 Message Date
ClawandClaude Opus 5 cbc24a4b19 feat(gateway/v1): real SSE streaming on the openrouter lane; honest 400 elsewhere
lakehouse/auditor 13 blocking issues: cloud: claim not backed — "pretending to have streamed. Verified live: chat.completion.chunk frames from"
The gateway accepted `stream: true` and returned a single JSON body — the code
said 'Phase 39+ wires real streaming' and it never happened. A streaming client
waits for SSE frames that never arrive and reports 'no response': that is what
broke openclaw's gateway lane today and cost six orchestrator rounds before the
cause was found.

Now: openrouter/openrouter_free forward the upstream SSE byte-for-byte; every
other lane rejects stream=true with a 400 that names the lane, instead of
pretending to have streamed. Verified live: chat.completion.chunk frames from
:3100, and the non-streaming path unchanged. Streamed calls skip usage metering
and Langfuse (stated in the code and logged per call) — metering needs the final
usage chunk and belongs in its own change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 07:33:47 -05:00
ClawandClaude Opus 5 dd74a1ef74 fix(gateway/v1): pass OpenAI tool calling through on the openrouter lane; reject tools loudly elsewhere
lakehouse/auditor 11 blocking issues: cloud: claim not backed — "Verified live after deploy: tool_calls returned (finish_reason tool_calls); plain request"
/v1/chat/completions deserialized into a struct without tools/tool_choice and messages without
tool_calls/tool_call_id, so serde silently DROPPED them. Tool-using agents routed through :3100
(openclaw, 2026-09-18) never saw their tools and fabricated tool output as text.

- ChatRequest: tools + tool_choice (serde default). Message: tool_calls, tool_call_id, name
  (skip_serializing_if none -> plain-text JSON is byte-identical; test pins it).
- openrouter lane forwards them and returns the model's tool_calls (content null on a
  tool-call-only turn, finish_reason passed through).
- every other provider: HTTP 400 when a request uses tools, instead of silent stripping.

Verified live after deploy: tool_calls returned (finish_reason tool_calls); plain request
unchanged; ollama + tools -> 400; multi-turn tool result round-trip answered from the tool.
cargo test -p gateway --lib: 111 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 09:08:29 -05:00
rootandClaude Fable 5 9d5107fb4a auditor: alt lane (Haiku) off drained opencode onto OpenRouter
lakehouse/auditor 10 blocking issues: cloud: claim not backed — "password). Verified live: /api/v1/user auths as profit, pulls list 200s."
PR #14 audit 2 flipped to the alt lane and hit opencode 401
CreditsError (balance drained). Alt defaults move to
openrouter/anthropic/claude-haiku-4.5 (probed live through the
gateway); maxTokensFor learns vendor-prefixed claude ids.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:40:13 -05:00
rootandClaude Fable 5 8bdceda883 gateway: OR_TIMEOUT_SECS env-overridable — 180s wall starved kimi_architect
lakehouse/auditor 8 blocking issues: cloud: claim not backed — "password). Verified live: /api/v1/user auths as profit, pulls list 200s."
The auditor's kimi_architect second pass (~180K-char prompt via
openrouter/moonshotai/kimi-k2.6) died at exactly the hardcoded 180s
client timeout on PR #14's drill audit, surfacing as 'error decoding
response body'. Ceiling now reads OR_TIMEOUT_SECS (default unchanged);
host drop-in sets 900s to match the auditor's own call budget.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:31:22 -05:00
rootandClaude Fable 5 5972cd2fbc auditor: repoint Gitea client at git.warren.help — agentview host is dead
lakehouse/auditor 9 blocking issues: cloud: claim not backed — "password). Verified live: /api/v1/user auths as profit, pulls list 200s."
git.agentview.dev has served a Traefik default cert + API 404 since
Aug 25 (VPS consolidation moved repos to warren); listOpenPrs failed
every 90s cycle since. Host default moves to git.warren.help and auth
now resolves per-host from ~/.git-credentials with fallback to the
origin remote's embedded credentials, sent as Basic (works for PAT or
password). Verified live: /api/v1/user auths as profit, pulls list 200s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 04:06:54 -05:00
rootandClaude Fable 5 0661c2cb83 auditor: sweep model lanes off lapsed Ollama Cloud onto OpenRouter
Ollama Cloud sub lapsed 2026-09-02 — every ollama_cloud model 402s.
Consensus lane -> openrouter/deepseek/deepseek-chat-v3.1 (same lineage),
tie-breaker -> openrouter/x-ai/grok-4.3 (4.1-fast deprecated upstream),
kimi_architect -> openrouter/moonshotai/kimi-k2.6 via provider default
so the Kimi<->Haiku alternation stays intact. maxTokensFor learns the
vendor-prefixed kimi id. All three lanes + pr_audit mode runner probed
live through the gateway before landing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 04:05:13 -05:00
rootandClaude Fable 5 c97f52de82 gateway/v1: 402 affordability clamp-retry on the openrouter lane
Stripping max_tokens on a 402 made things worse (the precheck then
prices the model's own 65k maximum). Instead: parse the 'can only
afford N' budget from the error and retry once clamped to N when
N >= 256 — small answers survive a thin balance; anything less
surfaces the 402 verbatim, because the real fix is funding the
account the key belongs to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 17:12:53 -05:00
rootandClaude Fable 5 6de15ad5f5 gateway/v1: tolerate reasoning-model responses on the openrouter lane
Reasoning models (GLM-5.x, deepseek-v4-pro) return content:null plus a
reasoning field when max_tokens exhausts mid-thought; ORMessageResp's bare
String made serde reject the whole body ('invalid openrouter response'),
taking down every reasoning model on the lane — including phase2's
configured z-ai/glm-5.2. content/reasoning are now Options; content stays
the only text returned (empty when null, so callers' schema validation +
retry budgets handle it deterministically); reasoning-only responses log a
warn with finish_reason. Unit test pins both shapes.

Found during the 2026-08-28 VIX founder-lane dogfood.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 17:07:12 -05:00
9 changed files with 427 additions and 72 deletions
+10 -5
View File
@@ -28,11 +28,16 @@ const GATEWAY = process.env.LH_GATEWAY_URL ?? "http://localhost:3100";
// retries — verified with trivial 8-token probes). Swapped default to
// deepseek-v3.1:671b which is proven working end-to-end through the
// pr_audit mode runner during Phase 5 distillation acceptance testing.
// kimi-k2:1t can be re-selected via LH_AUDITOR_REVIEW_MODEL env when
// the upstream returns. Tie-breaker stays grok-4.1-fast (different
// vendor lineage so consensus + tie-break won't fail-correlate).
const MODEL = process.env.LH_AUDITOR_REVIEW_MODEL ?? "deepseek-v3.1:671b";
const TIEBREAKER_MODEL = process.env.LH_AUDITOR_TIEBREAKER_MODEL ?? "x-ai/grok-4.1-fast";
//
// 2026-09-02 update: Ollama Cloud subscription lapsed — every
// ollama_cloud model 402s ("requires a subscription"). J's call: sweep
// the auditor to OpenRouter. Same deepseek lineage via OpenRouter;
// tie-breaker moves to grok-4.3 because OpenRouter 404s grok-4.1-fast
// as deprecated. Both probed live through the gateway before landing.
// Vendor lineages stay distinct so consensus + tie-break won't
// fail-correlate.
const MODEL = process.env.LH_AUDITOR_REVIEW_MODEL ?? "openrouter/deepseek/deepseek-chat-v3.1";
const TIEBREAKER_MODEL = process.env.LH_AUDITOR_TIEBREAKER_MODEL ?? "openrouter/x-ai/grok-4.3";
const N_CONSENSUS = Number(process.env.LH_AUDITOR_CONSENSUS_N ?? 3);
const AUDIT_DISCREPANCIES_JSONL = "/home/profit/lakehouse/data/_kb/audit_discrepancies.jsonl";
// 40KB comfortably fits the consensus models' context windows
+15 -11
View File
@@ -51,13 +51,15 @@ const CALL_TIMEOUT_MS = 900_000;
const CACHE_TTL_MS = 24 * 60 * 60 * 1000;
const MAX_DIFF_CHARS = 180_000;
const MAX_PRIOR_FINDINGS = 50;
// Default provider/model = ollama_cloud/kimi-k2.6. Pre-2026-04-27 we
// went direct to api.kimi.com, but Ollama Cloud Pro now exposes the
// same model legitimately, so we route there to avoid User-Agent
// gating. The api.kimi.com path (provider=kimi) remains wired in the
// gateway as a fallback for when Ollama Cloud is upstream-broken.
const KIMI_PROVIDER = process.env.LH_AUDITOR_KIMI_PROVIDER ?? "ollama_cloud";
const KIMI_MODEL = process.env.LH_AUDITOR_KIMI_MODEL ?? "kimi-k2.6";
// Default provider/model = openrouter/moonshotai/kimi-k2.6.
// History: pre-2026-04-27 direct api.kimi.com (User-Agent gating),
// then ollama_cloud/kimi-k2.6 while the sub was active. 2026-09-02:
// Ollama Cloud subscription lapsed (402 on every cloud model), J's
// call was to sweep the auditor to OpenRouter — same Moonshot model,
// probed live through the gateway. The api.kimi.com path
// (provider=kimi) remains wired in the gateway as a fallback.
const KIMI_PROVIDER = process.env.LH_AUDITOR_KIMI_PROVIDER ?? "openrouter";
const KIMI_MODEL = process.env.LH_AUDITOR_KIMI_MODEL ?? "moonshotai/kimi-k2.6";
// Cross-lineage alternation. 2026-04-27 J's call: Opus is too
// expensive to auto-fire (~$0.30/audit). Kimi K2.6 via Go-sub is
// effectively free; Haiku 4.5 via Zen is ~$0.04. Alternate between
@@ -75,8 +77,10 @@ const KIMI_MODEL = process.env.LH_AUDITOR_KIMI_MODEL ?? "kimi-k2.6";
// Override the alternation entirely with LH_AUDITOR_KIMI_MODEL
// (forces one model regardless of audit count); set
// LH_AUDITOR_KIMI_ALT_MODEL to the alternate.
const ALT_MODEL = process.env.LH_AUDITOR_KIMI_ALT_MODEL ?? "claude-haiku-4-5";
const ALT_PROVIDER = process.env.LH_AUDITOR_KIMI_ALT_PROVIDER ?? "opencode";
// 2026-09-02: opencode lane 401s (CreditsError, balance drained) — alt
// lane joins the OpenRouter sweep. Same Anthropic lineage, probed live.
const ALT_MODEL = process.env.LH_AUDITOR_KIMI_ALT_MODEL ?? "anthropic/claude-haiku-4.5";
const ALT_PROVIDER = process.env.LH_AUDITOR_KIMI_ALT_PROVIDER ?? "openrouter";
const FORCE_DEFAULT = process.env.LH_AUDITOR_KIMI_MODEL !== undefined && process.env.LH_AUDITOR_KIMI_MODEL !== "";
function selectModel(diffLen: number, auditIndex: number = 0): { provider: string; model: string; promoted: boolean } {
@@ -110,8 +114,8 @@ const MAX_TOKENS_OVERRIDE = Number(process.env.LH_AUDITOR_KIMI_MAX_TOKENS) || 0;
function maxTokensFor(model: string): number {
if (MAX_TOKENS_OVERRIDE > 0) return MAX_TOKENS_OVERRIDE;
if (model.startsWith("claude-opus")) return 32_000;
if (model.startsWith("claude-haiku") || model.startsWith("claude-sonnet")) return 8_192;
if (model.startsWith("kimi-")) return 128_000;
if (model.startsWith("claude-haiku") || model.startsWith("claude-sonnet") || model.includes("/claude-haiku") || model.includes("/claude-sonnet")) return 8_192;
if (model.startsWith("kimi-") || model.includes("/kimi-")) return 128_000;
if (model.startsWith("gpt-5") || model.startsWith("o1") || model.startsWith("o3") || model.startsWith("o4")) return 32_000;
return 16_000;
}
+25 -11
View File
@@ -9,28 +9,42 @@
import { readFile } from "node:fs/promises";
import type { PrSnapshot } from "./types.ts";
const HOST = process.env.GITEA_HOST ?? "https://git.agentview.dev";
// 2026-09-02: repo home moved to git.warren.help in the VPS
// consolidation; git.agentview.dev now dead-ends at a Traefik default
// cert + 404 (auditor had been failing listOpenPrs every cycle since
// Aug 25). Credentials come from ~/.git-credentials for HOST's domain,
// falling back to the user:pass embedded in this repo's origin remote.
// Basic auth works for both a PAT and a password on Gitea.
const HOST = process.env.GITEA_HOST ?? "https://git.warren.help";
const OWNER = "profit";
const REPO = "lakehouse";
const CRED_FILE = "/home/profit/.git-credentials";
const GIT_CONFIG = "/home/profit/lakehouse/.git/config";
let cachedPat: string | null = null;
let cachedAuth: string | null = null;
async function getPat(): Promise<string> {
if (cachedPat) return cachedPat;
const raw = await readFile(CRED_FILE, "utf8");
for (const line of raw.split("\n")) {
const m = line.match(/^https:\/\/[^:]+:([^@]+)@git\.agentview\.dev/);
if (m) { cachedPat = m[1]; return m[1]; }
async function getAuthHeader(): Promise<string> {
if (cachedAuth) return cachedAuth;
const hostDomain = new URL(HOST).host.replace(/\./g, "\\.");
const credRe = new RegExp(`^https://([^:]+):([^@]+)@${hostDomain}`);
for (const file of [CRED_FILE, GIT_CONFIG]) {
let raw: string;
try { raw = await readFile(file, "utf8"); } catch { continue; }
for (const line of raw.split("\n")) {
const m = line.trim().replace(/^url = /, "").match(credRe);
if (m) {
cachedAuth = `Basic ${Buffer.from(`${decodeURIComponent(m[1])}:${decodeURIComponent(m[2])}`).toString("base64")}`;
return cachedAuth;
}
}
}
throw new Error(`no Gitea PAT in ${CRED_FILE}`);
throw new Error(`no Gitea credentials for ${HOST} in ${CRED_FILE} or ${GIT_CONFIG}`);
}
async function giteaFetch(path: string, init: RequestInit = {}): Promise<Response> {
const pat = await getPat();
const url = `${HOST}/api/v1${path}`;
const headers = new Headers(init.headers);
headers.set("Authorization", `token ${pat}`);
headers.set("Authorization", await getAuthHeader());
if (init.body && !headers.has("content-type")) {
headers.set("content-type", "application/json");
}
+4
View File
@@ -453,6 +453,8 @@ impl ExecutionLoop {
stream: Some(false),
think: Some(think),
provider: Some("ollama_cloud".into()),
tools: None,
tool_choice: None,
};
let resp = crate::v1::ollama_cloud::chat(key, &req).await
.map_err(|e| format!("ollama_cloud: {e}"))?;
@@ -693,6 +695,8 @@ impl ExecutionLoop {
stream: Some(false),
think: Some(false),
provider: Some("ollama".into()),
tools: None,
tool_choice: None,
};
let resp = crate::v1::ollama::chat(&self.state.ai_client, &chat_req).await
.map_err(|e| format!("ollama overseer: {e}"))?;
+1 -1
View File
@@ -140,7 +140,7 @@ pub async fn chat(
model,
choices: vec![Choice {
index: 0,
message: Message { role: "assistant".into(), content: serde_json::Value::String(text) },
message: Message { role: "assistant".into(), content: serde_json::Value::String(text), ..Default::default() },
finish_reason: choice.finish_reason.unwrap_or_else(|| "stop".into()),
}],
usage: UsageBlock {
+122 -10
View File
@@ -148,10 +148,22 @@ pub fn router(state: V1State) -> Router {
/// providers can take it verbatim. `Message::text()` flattens for
/// places that need a plain string (Ollama prompt assembly, char
/// counts, the assistant's own response synthesis).
#[derive(Serialize, Deserialize, Clone, Debug)]
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
pub struct Message {
pub role: String,
/// `Value::Null` is legal: an assistant turn that only makes tool calls.
#[serde(default)]
pub content: serde_json::Value,
/// OpenAI tool-calling fields (2026-09-19). Passed through verbatim on the OpenRouter lane;
/// omitted from JSON when unset, so plain-text traffic is byte-identical to before. Before
/// this, serde silently DROPPED them — tool-using agents routed through :3100 never saw their
/// tools and fabricated tool output (autobun KB finding infra.lakehouse_gateway.tools).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<serde_json::Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
}
impl Message {
@@ -163,6 +175,7 @@ impl Message {
Self {
role: role.into(),
content: serde_json::Value::String(body.into()),
..Default::default()
}
}
/// Flatten content to a plain string. Strings pass through; content-
@@ -195,9 +208,8 @@ pub struct ChatRequest {
pub temperature: Option<f64>,
#[serde(default)]
pub max_tokens: Option<u32>,
/// Accepted for shape-compat but ignored in the thin slice —
/// Phase 38 returns non-streaming even when the client asked for it.
/// Phase 39+ wires real streaming.
/// Real SSE passthrough on the openrouter lane (2026-09-20); other lanes reject
/// stream=true with a 400 rather than silently returning a non-streamed body.
#[serde(default)]
pub stream: Option<bool>,
/// Non-OpenAI extension. Passes through to the provider's thinking
@@ -215,6 +227,21 @@ pub struct ChatRequest {
/// from the model name; Phase 38/39 requires explicit selection.
#[serde(default)]
pub provider: Option<String>,
/// OpenAI tool definitions + choice (2026-09-19). Forwarded on the OpenRouter lane only;
/// any other provider REJECTS them with 400 instead of silently stripping them.
#[serde(default)]
pub tools: Option<serde_json::Value>,
#[serde(default)]
pub tool_choice: Option<serde_json::Value>,
}
impl ChatRequest {
/// True if the request uses tool calling anywhere (definitions or tool-call history).
pub fn uses_tools(&self) -> bool {
self.tools.is_some()
|| self.tool_choice.is_some()
|| self.messages.iter().any(|m| m.role == "tool" || m.tool_calls.is_some() || m.tool_call_id.is_some())
}
}
#[derive(Serialize)]
@@ -319,9 +346,47 @@ mod resolve_provider_tests {
stream: None,
think: None,
provider: provider.map(|s| s.to_string()),
tools: None,
tool_choice: None,
}
}
/// Streaming is a per-lane capability, and the routing decision must be visible in a test:
/// before 2026-09-20 the gateway accepted stream=true on every lane and returned a single
/// JSON body, so a client waiting for SSE frames hung until its own timeout.
#[test]
fn streaming_is_openrouter_only() {
let mut r = mk_req(None, "openrouter/z-ai/glm-5.3-flash");
r.stream = Some(true);
assert_eq!(resolve_provider(&r).0, "openrouter", "openrouter lane streams");
let mut local = mk_req(None, "qwen3.5:latest");
local.stream = Some(true);
assert_eq!(
resolve_provider(&local).0,
"ollama",
"a local model resolves to ollama, where the handler rejects stream=true with a 400 \
instead of silently returning a non-streamed body"
);
}
#[test]
fn uses_tools_detects_definitions_and_history() {
let mut r = mk_req(None, "openrouter/z-ai/glm-5.3-flash");
assert!(!r.uses_tools());
r.tools = Some(serde_json::json!([]));
assert!(r.uses_tools());
let mut r2 = mk_req(None, "qwen3.5:latest");
r2.messages.push(Message { role: "tool".into(), content: "ok".into(), tool_call_id: Some("c1".into()), ..Default::default() });
assert!(r2.uses_tools());
}
#[test]
fn plain_message_serialization_unchanged() {
let m = Message::new_text("user", "hi");
assert_eq!(serde_json::to_string(&m).unwrap(), r#"{"role":"user","content":"hi"}"#);
}
#[test]
fn explicit_provider_wins() {
let r = mk_req(Some("openrouter"), "qwen3.5:latest");
@@ -378,19 +443,27 @@ async fn chat(
State(state): State<V1State>,
headers: axum::http::HeaderMap,
Json(req): Json<ChatRequest>,
) -> Result<Json<ChatResponse>, (StatusCode, String)> {
) -> Result<axum::response::Response, (StatusCode, String)> {
use axum::response::IntoResponse as _;
if req.messages.is_empty() {
return Err((StatusCode::BAD_REQUEST, "messages must be non-empty".into()));
}
if req.stream.unwrap_or(false) {
tracing::warn!("/v1/chat: stream=true requested but Phase 38 returns non-streaming");
}
// Provider resolution: explicit `req.provider` wins; otherwise
// infer from a model-name prefix. Phase 39 PRD gate example:
// `model: "openrouter/openai/gpt-4o-mini"` → provider "openrouter",
// adapter gets the stripped "openai/gpt-4o-mini".
let (provider, effective_model) = resolve_provider(&req);
// Tool calling is only wired on the OpenRouter lane. Refuse loudly elsewhere — silently
// dropping tools made agents fabricate tool results (2026-09-19).
if req.uses_tools() && !matches!(provider.as_str(), "openrouter" | "openrouter_free") {
return Err((
StatusCode::BAD_REQUEST,
format!(
"tools/tool_calls are only supported on the openrouter lane via this gateway \
(resolved provider: {provider}); use an OpenRouter model id (e.g. openrouter/<vendor>/<model>)"
),
));
}
let start_time = chrono::Utc::now();
let start_instant = std::time::Instant::now();
@@ -406,6 +479,45 @@ async fn chat(
std::borrow::Cow::Owned(cloned)
};
// STREAMING (2026-09-20). Phase 38 accepted `stream: true` and returned a single JSON body;
// a client waiting for SSE frames hangs and then reports "no response". Forward the upstream
// SSE verbatim on the lane that supports it, and say so plainly on the lanes that do not
// instead of pretending. Streamed calls skip usage metering + Langfuse (see chat_stream).
if req.stream.unwrap_or(false) {
if !matches!(provider.as_str(), "openrouter" | "openrouter_free") {
return Err((
StatusCode::BAD_REQUEST,
format!(
"stream=true is only supported on the openrouter lane via this gateway \
(resolved provider: {provider}); resend with stream=false or use an \
OpenRouter model id"
),
));
}
let key = state.openrouter_key.as_deref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"OPENROUTER_API_KEY not configured".to_string(),
))?;
let upstream = openrouter::chat_stream(key, &*req_for_adapter)
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, format!("openrouter stream: {e}")))?;
tracing::info!(
target: "v1.chat",
provider = %provider,
model = %effective_model,
"streaming passthrough — usage metering and Langfuse are skipped for this call",
);
let body = axum::body::Body::from_stream(upstream.bytes_stream());
return Ok((
[
(axum::http::header::CONTENT_TYPE, "text/event-stream"),
(axum::http::header::CACHE_CONTROL, "no-cache"),
],
body,
)
.into_response());
}
let (resp, used_provider) = match provider.as_str() {
"ollama" | "local" | "" => {
let r = ollama::chat(&state.ai_client, &*req_for_adapter)
@@ -589,7 +701,7 @@ async fn chat(
provider_usage.total_tokens += resp.usage.total_tokens as u64;
}
Ok(Json(resp))
Ok(Json(resp).into_response())
}
async fn usage(State(state): State<V1State>) -> impl IntoResponse {
+10 -10
View File
@@ -113,10 +113,10 @@ mod tests {
#[test]
fn flatten_separates_system_from_turns() {
let msgs = vec![
Message { role: "system".into(), content: "Rules here.".into() },
Message { role: "user".into(), content: "Q1".into() },
Message { role: "assistant".into(), content: "A1".into() },
Message { role: "user".into(), content: "Q2".into() },
Message { role: "system".into(), content: "Rules here.".into(), ..Default::default() },
Message { role: "user".into(), content: "Q1".into(), ..Default::default() },
Message { role: "assistant".into(), content: "A1".into(), ..Default::default() },
Message { role: "user".into(), content: "Q2".into(), ..Default::default() },
];
let (system, prompt) = flatten_messages(&msgs);
assert_eq!(system, "Rules here.");
@@ -129,9 +129,9 @@ mod tests {
#[test]
fn flatten_concatenates_multiple_system_messages() {
let msgs = vec![
Message { role: "system".into(), content: "First.".into() },
Message { role: "system".into(), content: "Second.".into() },
Message { role: "user".into(), content: "Hi".into() },
Message { role: "system".into(), content: "First.".into(), ..Default::default() },
Message { role: "system".into(), content: "Second.".into(), ..Default::default() },
Message { role: "user".into(), content: "Hi".into(), ..Default::default() },
];
let (system, _) = flatten_messages(&msgs);
assert_eq!(system, "First.\nSecond.");
@@ -139,7 +139,7 @@ mod tests {
#[test]
fn flatten_with_no_system_returns_empty_system() {
let msgs = vec![Message { role: "user".into(), content: "hi".into() }];
let msgs = vec![Message { role: "user".into(), content: "hi".into(), ..Default::default() }];
let (system, prompt) = flatten_messages(&msgs);
assert!(system.is_empty());
assert!(prompt.contains("user: hi"));
@@ -147,11 +147,11 @@ mod tests {
#[test]
fn estimate_tokens_chars_div_4_ceiling() {
let msgs = vec![Message { role: "user".into(), content: "abcdefgh".into() }];
let msgs = vec![Message { role: "user".into(), content: "abcdefgh".into(), ..Default::default() }];
// 8 chars / 4 = 2, with ceiling → 2
assert_eq!(estimate_prompt_tokens(&msgs), 2);
let msgs2 = vec![Message { role: "user".into(), content: "abcdefghi".into() }];
let msgs2 = vec![Message { role: "user".into(), content: "abcdefghi".into(), ..Default::default() }];
// 9 chars → (9+3)/4 = 3 (ceiling)
assert_eq!(estimate_prompt_tokens(&msgs2), 3);
}
+1 -1
View File
@@ -145,7 +145,7 @@ pub async fn chat(
model,
choices: vec![Choice {
index: 0,
message: Message { role: "assistant".into(), content: serde_json::Value::String(text) },
message: Message { role: "assistant".into(), content: serde_json::Value::String(text), ..Default::default() },
finish_reason: choice.finish_reason.unwrap_or_else(|| "stop".into()),
}],
usage: UsageBlock {
+239 -23
View File
@@ -20,7 +20,19 @@ use serde::{Deserialize, Serialize};
use super::{ChatRequest, ChatResponse, Choice, Message, UsageBlock};
const OR_BASE_URL: &str = "https://openrouter.ai/api/v1";
const OR_TIMEOUT_SECS: u64 = 180;
// Default request ceiling. 180s starves long-context calls (kimi_architect's
// ~180K-char second pass died at exactly this wall on 2026-09-02, surfacing
// as a misleading "error decoding response body") — override per host with
// OR_TIMEOUT_SECS. The auditor budgets up to 900s client-side.
const OR_TIMEOUT_SECS_DEFAULT: u64 = 180;
fn or_timeout_secs() -> u64 {
std::env::var("OR_TIMEOUT_SECS")
.ok()
.and_then(|v| v.trim().parse::<u64>().ok())
.filter(|&s| s > 0)
.unwrap_or(OR_TIMEOUT_SECS_DEFAULT)
}
pub fn resolve_openrouter_key() -> Option<String> {
if let Ok(k) = std::env::var("OPENROUTER_API_KEY") {
@@ -49,6 +61,52 @@ pub fn resolve_openrouter_key() -> Option<String> {
None
}
/// Streaming passthrough (2026-09-20). The gateway used to accept `stream: true` and return a
/// single non-streamed JSON body — a client waiting for SSE frames hangs until its own timeout
/// and reports "no response" (this is what broke openclaw's gateway lane). We now forward the
/// upstream SSE byte-for-byte.
///
/// The caller streams the body straight to its client, so nothing here parses the chunks:
/// usage metering and Langfuse tracing are SKIPPED for streamed calls, and the handler logs
/// that. Metering streamed responses needs the final usage chunk and belongs in its own change.
pub async fn chat_stream(key: &str, req: &ChatRequest) -> Result<reqwest::Response, String> {
let model = req.model.strip_prefix("openrouter/").unwrap_or(&req.model).to_string();
let body = ORChatBody {
model,
messages: req.messages.iter().map(|m| ORMessage {
role: m.role.clone(),
content: m.content.clone(),
tool_calls: m.tool_calls.clone(),
tool_call_id: m.tool_call_id.clone(),
name: m.name.clone(),
}).collect(),
tools: req.tools.clone(),
tool_choice: req.tool_choice.clone(),
max_tokens: Some(req.max_tokens.unwrap_or(800)),
temperature: req.temperature.unwrap_or(0.3),
stream: true,
};
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(or_timeout_secs()))
.build()
.map_err(|e| format!("build client: {e}"))?;
let resp = client
.post(format!("{}/chat/completions", OR_BASE_URL))
.bearer_auth(key)
.header("HTTP-Referer", "https://vcp.devop.live")
.header("X-Title", "Lakehouse Scrum")
.json(&serde_json::to_value(&body).map_err(|e| format!("serialize body: {e}"))?)
.send()
.await
.map_err(|e| format!("openrouter.ai unreachable: {e}"))?;
if !resp.status().is_success() {
let code = resp.status();
let text = resp.text().await.unwrap_or_else(|_| "?".into());
return Err(format!("openrouter {code}: {}", text.chars().take(300).collect::<String>()));
}
Ok(resp)
}
pub async fn chat(
key: &str,
req: &ChatRequest,
@@ -57,7 +115,7 @@ pub async fn chat(
// form so OpenRouter sees the raw model id (e.g. "openai/gpt-oss-120b:free").
let model = req.model.strip_prefix("openrouter/").unwrap_or(&req.model).to_string();
let body = ORChatBody {
let mut body = ORChatBody {
model: model.clone(),
// Pass content through verbatim — preserves OpenAI's multimodal
// content-parts shape (`[{type:"text",text:"..."}, ...]`) so the
@@ -65,30 +123,78 @@ pub async fn chat(
messages: req.messages.iter().map(|m| ORMessage {
role: m.role.clone(),
content: m.content.clone(),
tool_calls: m.tool_calls.clone(),
tool_call_id: m.tool_call_id.clone(),
name: m.name.clone(),
}).collect(),
max_tokens: req.max_tokens.unwrap_or(800),
tools: req.tools.clone(),
tool_choice: req.tool_choice.clone(),
max_tokens: Some(req.max_tokens.unwrap_or(800)),
temperature: req.temperature.unwrap_or(0.3),
stream: false,
};
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(OR_TIMEOUT_SECS))
.timeout(Duration::from_secs(or_timeout_secs()))
.build()
.map_err(|e| format!("build client: {e}"))?;
let t0 = std::time::Instant::now();
let resp = client
.post(format!("{}/chat/completions", OR_BASE_URL))
.bearer_auth(key)
// OpenRouter recommends Referer + Title for attribution; absent
// headers do not fail the call but help us see our traffic in
// their dashboard.
.header("HTTP-Referer", "https://vcp.devop.live")
.header("X-Title", "Lakehouse Scrum")
.json(&body)
.send()
.await
.map_err(|e| format!("openrouter.ai unreachable: {e}"))?;
let send = |client: reqwest::Client, key: String, body_json: serde_json::Value| async move {
client
.post(format!("{}/chat/completions", OR_BASE_URL))
.bearer_auth(key)
// OpenRouter recommends Referer + Title for attribution; absent
// headers do not fail the call but help us see our traffic in
// their dashboard.
.header("HTTP-Referer", "https://vcp.devop.live")
.header("X-Title", "Lakehouse Scrum")
.json(&body_json)
.send()
.await
.map_err(|e| format!("openrouter.ai unreachable: {e}"))
};
let mut resp = send(
client.clone(),
key.to_string(),
serde_json::to_value(&body).map_err(|e| format!("serialize body: {e}"))?,
)
.await?;
// OpenRouter's affordability precheck reserves max_tokens × price against
// the account balance. Measured 2026-08-28: stripping the cap makes it
// WORSE (the precheck then prices the model's own maximum, e.g. 65536) —
// so a low-balance 402 surfaces verbatim and the fix is funding the
// account the key belongs to, not request-shape games. One clamp retry:
// when the error names an affordable budget ≥256 tokens, retry once at
// that budget so small answers still get through on a thin balance.
if resp.status() == reqwest::StatusCode::PAYMENT_REQUIRED && body.max_tokens.is_some() {
let err_body = resp.text().await.unwrap_or_else(|_| "?".into());
let afford = err_body
.split("can only afford ")
.nth(1)
.and_then(|s| s.split(|c: char| !c.is_ascii_digit()).next())
.and_then(|n| n.parse::<u32>().ok());
match afford {
Some(n) if n >= 256 => {
tracing::warn!(
target: "v1.chat",
provider = "openrouter",
model = %model,
afford = n,
"402 affordability precheck — clamping max_tokens to the stated budget",
);
body.max_tokens = Some(n);
resp = send(
client.clone(),
key.to_string(),
serde_json::to_value(&body).map_err(|e| format!("serialize body: {e}"))?,
)
.await?;
}
_ => return Err(format!("openrouter.ai 402 Payment Required: {}", err_body)),
}
}
let status = resp.status();
if !status.is_success() {
@@ -102,7 +208,24 @@ pub async fn chat(
let latency_ms = t0.elapsed().as_millis();
let choice = parsed.choices.into_iter().next()
.ok_or_else(|| "openrouter returned no choices".to_string())?;
let text = choice.message.content;
let tool_calls = choice.message.tool_calls.clone();
let reasoning_only = choice.message.content.is_none()
&& choice.message.reasoning.is_some()
&& tool_calls.is_none();
let text = choice.message.content.unwrap_or_default();
if reasoning_only {
// The model spent its whole budget thinking. Return well-formed empty
// content (callers' schema validation + retry budgets handle it
// deterministically) rather than a decode error that reads as a
// gateway fault — and never pass thinking-text off as an answer.
tracing::warn!(
target: "v1.chat",
provider = "openrouter",
model = %model,
finish = %choice.finish_reason.as_deref().unwrap_or("?"),
"reasoning-only response (content null) — raise max_tokens or use a non-reasoning model",
);
}
let prompt_tokens = parsed.usage.as_ref().map(|u| u.prompt_tokens).unwrap_or_else(|| {
let chars: usize = req.messages.iter().map(|m| m.text().chars().count()).sum();
@@ -129,7 +252,17 @@ pub async fn chat(
model,
choices: vec![Choice {
index: 0,
message: Message { role: "assistant".into(), content: serde_json::Value::String(text) },
message: Message {
role: "assistant".into(),
// OpenAI shape: a tool-call-only turn carries `content: null`.
content: if tool_calls.is_some() && text.is_empty() {
serde_json::Value::Null
} else {
serde_json::Value::String(text)
},
tool_calls,
..Default::default()
},
finish_reason: choice.finish_reason.unwrap_or_else(|| "stop".into()),
}],
usage: UsageBlock {
@@ -146,13 +279,27 @@ pub async fn chat(
struct ORChatBody {
model: String,
messages: Vec<ORMessage>,
max_tokens: u32,
#[serde(skip_serializing_if = "Option::is_none")]
max_tokens: Option<u32>,
temperature: f64,
stream: bool,
#[serde(skip_serializing_if = "Option::is_none")]
tools: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
tool_choice: Option<serde_json::Value>,
}
#[derive(Serialize)]
struct ORMessage { role: String, content: serde_json::Value }
struct ORMessage {
role: String,
content: serde_json::Value,
#[serde(skip_serializing_if = "Option::is_none")]
tool_calls: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
tool_call_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
name: Option<String>,
}
#[derive(Deserialize)]
struct ORChatResponse {
@@ -169,7 +316,21 @@ struct ORChoice {
}
#[derive(Deserialize)]
struct ORMessageResp { content: String }
struct ORMessageResp {
// Reasoning models (GLM-5.x, deepseek-v4-pro) return `content: null` plus a
// `reasoning` field when they exhaust max_tokens mid-thought. A bare String
// here made serde reject the whole body ("invalid openrouter response",
// 2026-08-28) — which took down every reasoning model on the lane,
// including phase2's configured glm-5.2. Content stays the only text we
// ever return; reasoning is observability, never an answer.
#[serde(default)]
content: Option<String>,
#[serde(default)]
reasoning: Option<String>,
/// OpenAI tool calls requested by the model (passed back to the client verbatim).
#[serde(default)]
tool_calls: Option<serde_json::Value>,
}
#[derive(Deserialize)]
struct ORUsage { prompt_tokens: u32, completion_tokens: u32 }
@@ -178,6 +339,21 @@ struct ORUsage { prompt_tokens: u32, completion_tokens: u32 }
mod tests {
use super::*;
#[test]
fn reasoning_model_null_content_parses() {
// GLM-5.x / deepseek-v4-pro shape: content null + reasoning present
// (the 2026-08-28 decode failure). Must parse, not error.
let body = r#"{"choices":[{"message":{"role":"assistant","content":null,"reasoning":"thinking…"},"finish_reason":"length"}],"usage":{"prompt_tokens":5,"completion_tokens":10}}"#;
let parsed: ORChatResponse = serde_json::from_str(body).expect("null content must parse");
let choice = &parsed.choices[0];
assert!(choice.message.content.is_none());
assert_eq!(choice.message.reasoning.as_deref(), Some("thinking…"));
// And the normal shape still parses with reasoning absent.
let normal = r#"{"choices":[{"message":{"role":"assistant","content":"OK"},"finish_reason":"stop"}]}"#;
let parsed: ORChatResponse = serde_json::from_str(normal).expect("normal shape");
assert_eq!(parsed.choices[0].message.content.as_deref(), Some("OK"));
}
#[test]
fn resolve_openrouter_key_does_not_panic() {
// Smoke test — all three sources may or may not be set depending
@@ -190,17 +366,57 @@ mod tests {
let body = ORChatBody {
model: "openai/gpt-oss-120b:free".into(),
messages: vec![
ORMessage { role: "user".into(), content: "review this".into() },
ORMessage { role: "user".into(), content: "review this".into(),
tool_calls: None, tool_call_id: None, name: None },
],
max_tokens: 800,
max_tokens: Some(800),
temperature: 0.3,
stream: false,
tools: None,
tool_choice: None,
};
let json = serde_json::to_string(&body).unwrap();
assert!(json.contains("\"model\":\"openai/gpt-oss-120b:free\""));
assert!(json.contains("\"messages\""));
assert!(json.contains("\"max_tokens\":800"));
assert!(json.contains("\"stream\":false"));
// no tool fields on plain traffic (byte-compat with pre-2026-09-19 bodies)
assert!(!json.contains("tools") && !json.contains("tool_call"));
}
#[test]
fn chat_body_forwards_tools_and_tool_messages() {
let tools = serde_json::json!([{"type":"function","function":{"name":"exec","parameters":{"type":"object"}}}]);
let body = ORChatBody {
model: "z-ai/glm-5.3-flash".into(),
messages: vec![
ORMessage { role: "assistant".into(), content: serde_json::Value::Null,
tool_calls: Some(serde_json::json!([{"id":"c1","type":"function","function":{"name":"exec","arguments":"{}"}}])),
tool_call_id: None, name: None },
ORMessage { role: "tool".into(), content: "ok".into(), tool_calls: None,
tool_call_id: Some("c1".into()), name: Some("exec".into()) },
],
max_tokens: Some(100),
temperature: 0.0,
stream: false,
tools: Some(tools),
tool_choice: Some(serde_json::json!("auto")),
};
let json = serde_json::to_string(&body).unwrap();
assert!(json.contains("\"tools\":[{"));
assert!(json.contains("\"tool_choice\":\"auto\""));
assert!(json.contains("\"tool_call_id\":\"c1\""));
assert!(json.contains("\"tool_calls\":[{"));
}
#[test]
fn response_with_tool_calls_parses() {
let raw = r#"{"choices":[{"message":{"content":null,"tool_calls":[{"id":"c1","type":"function","function":{"name":"exec","arguments":"{}"}}]},"finish_reason":"tool_calls"}]}"#;
let parsed: ORChatResponse = serde_json::from_str(raw).unwrap();
let c = &parsed.choices[0];
assert!(c.message.content.is_none());
assert!(c.message.tool_calls.is_some());
assert_eq!(c.finish_reason.as_deref(), Some("tool_calls"));
}
#[test]