v1: accept OpenAI multimodal content shape (array-of-parts)
lakehouse/auditor 1 blocking issue: todo!() macro call in tests/real-world/scrum_master_pipeline.ts

Modern OpenAI clients (pi-ai, openai SDK 6.x, langchain-js, the official
agents) send `messages[].content` as an array of content parts:
`[{type:"text", text:"..."}, {type:"image_url", ...}]`. Our gateway
typed `content` as plain `String` and 422'd those calls.

Fix: `Message.content` is now `serde_json::Value` so requests
deserialize regardless of shape. `Message::text()` flattens
content-parts arrays (concat'd `text` fields, non-text parts skipped)
for places that need a plain string — Ollama prompt assembly, char
counts, the assistant's own response synthesis. `Message::new_text()`
constructs string-content messages without writing the wrapper at
each call site. Forwarders (openrouter) clone content through
verbatim so providers see exactly what the client sent.

Verified end-to-end: Pi CLI (`pi --print --provider openrouter`)
landed a clean 1902-token request through `/v1/chat/completions`,
routed to OpenRouter as `openai/gpt-oss-120b:free`, response in
1.62s, Langfuse trace `v1.chat:openrouter` recorded with provider
tag. Same path that any tool using the official openai SDK takes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-26 17:56:46 -05:00
co-authored by Claude Opus 4.7
parent 3a0b37ed93
commit 540a9a27ee
7 changed files with 71 additions and 33 deletions
+43 -3
View File
@@ -97,10 +97,50 @@ pub fn router(state: V1State) -> Router {
// -- Shared types (OpenAI-compatible) --
/// OpenAI-compatible message. `content` accepts either a plain string or
/// an array of content parts (the modern multimodal shape:
/// `[{type:"text", text:"..."}, {type:"image_url", ...}]`). We store as
/// `serde_json::Value` to preserve client shape on forward; downstream
/// providers can take it verbatim. `Message::text()` flattens for
/// places that need a plain string (Ollama prompt assembly, char
/// counts, the assistant's own response synthesis).
#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct Message {
pub role: String,
pub content: String,
pub content: serde_json::Value,
}
impl Message {
/// Construct a plain text message — the common shape for callers
/// that don't need multimodal content. Wraps the body in
/// `serde_json::Value::String` so downstream serializers see the
/// canonical OpenAI shape.
pub fn new_text(role: impl Into<String>, body: impl Into<String>) -> Self {
Self {
role: role.into(),
content: serde_json::Value::String(body.into()),
}
}
/// Flatten content to a plain string. Strings pass through; content-
/// part arrays concatenate the `text` fields with newlines and skip
/// non-text parts (images etc.) — Phase 38/39 callers are text-only,
/// real multimodal forwarding is queued.
pub fn text(&self) -> String {
match &self.content {
serde_json::Value::String(s) => s.clone(),
serde_json::Value::Array(parts) => {
let mut out = String::new();
for p in parts {
if let Some(t) = p.get("text").and_then(|v| v.as_str()) {
if !out.is_empty() { out.push('\n'); }
out.push_str(t);
}
}
out
}
other => other.to_string(),
}
}
}
#[derive(Deserialize, Debug, Clone)]
@@ -380,7 +420,7 @@ async fn chat(
// untouched.
if let Some(lf) = &state.langfuse {
let output = resp.choices.first()
.map(|c| c.message.content.clone())
.map(|c| c.message.text())
.unwrap_or_default();
lf.emit_chat(langfuse_trace::ChatTrace {
provider: used_provider.clone(),
@@ -452,7 +492,7 @@ mod tests {
assert_eq!(r.model, "qwen3.5:latest");
assert_eq!(r.messages.len(), 2);
assert_eq!(r.messages[0].role, "system");
assert_eq!(r.messages[1].content, "Hi");
assert_eq!(r.messages[1].text(), "Hi");
assert_eq!(r.temperature, Some(0.2));
assert_eq!(r.max_tokens, Some(100));
}