Phase 40 (early): Langfuse tracing on /v1/chat — observability recovery

The lost stack J flagged was partly already present: Langfuse
container has been running 2 days with the staffing project, SDK
installed, mcp-server tracing gw:/* routes. What was missing was
Rust-side /v1/chat emission — the new Phase 38/39 code bypassed
Langfuse entirely.

This commit bridges it. Fire-and-forget HTTP POST to
http://localhost:3001/api/public/ingestion (batch {trace-create +
generation-create}) on every chat call. Non-blocking — spawned
tokio task, response latency unaffected. Trace failures log warn
and drop, never propagate.

Verified end-to-end after restart:
- Log line "v1: Langfuse tracing enabled" at startup
- /v1/chat local (qwen3.5:latest) → v1.chat:ollama trace appears
  with lat=0.41s, 24+6 tokens
- /v1/chat cloud (gpt-oss:120b) → v1.chat:ollama_cloud trace appears
  with lat=1.87s, 73+87 tokens
- mcp-server's existing gw:/log + gw:/intelligence/* traces
  continue to flow into the same project unchanged

Files:
- crates/gateway/src/v1/langfuse_trace.rs (new, 195 LOC) — thin
  client, no SDK. reqwest Basic Auth. ChatTrace payload + event
  serializer. from_env_or_defaults() resolver matches
  mcp-server/tracing.ts conventions (pk-lf-staffing / sk-lf-
  staffing-secret / localhost:3001)
- crates/gateway/src/v1/mod.rs — V1State.langfuse field, emission
  after successful provider call (post-dispatch, pre-usage-update)
- crates/gateway/src/main.rs — resolve + log at startup

Tests: 12/12 green (9 prior + 3 for langfuse_trace: ingestion-batch
serialization, uuid generator uniqueness, env resolver shape).

Recovered piece #1 of 3 from the lost-stack narrative. Still open:
- Langfuse → observer :3800 pipe (Phase 40 mid-deliverable)
- Gitea MCP reconnect in mcp-server/index.ts (Phase 40 late)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
profit
2026-04-22 03:04:28 -05:00
co-authored by Claude Opus 4.7
parent 6316433062
commit 75a0f424ef
3 changed files with 273 additions and 0 deletions
+35
View File
@@ -13,6 +13,7 @@
pub mod ollama;
pub mod ollama_cloud;
pub mod langfuse_trace;
use axum::{
Router,
@@ -33,6 +34,10 @@ pub struct V1State {
/// Ollama Cloud bearer token. Loaded at startup via
/// `ollama_cloud::resolve_cloud_key()`. None = cloud routes 503.
pub ollama_cloud_key: Option<String>,
/// Phase 40 early deliverable — Langfuse client. None = tracing
/// disabled (keys missing or container unreachable). Traces are
/// fire-and-forget: never block the response path.
pub langfuse: Option<langfuse_trace::LangfuseClient>,
}
#[derive(Default, Clone, Serialize)]
@@ -127,6 +132,9 @@ async fn chat(
}
let provider = req.provider.as_deref().unwrap_or("ollama").to_ascii_lowercase();
let start_time = chrono::Utc::now();
let start_instant = std::time::Instant::now();
let resp = match provider.as_str() {
"ollama" | "local" | "" => ollama::chat(&state.ai_client, &req)
.await
@@ -148,6 +156,33 @@ async fn chat(
}
};
let end_time = chrono::Utc::now();
let latency_ms = start_instant.elapsed().as_millis() as u64;
// Phase 40 — emit Langfuse trace. Fire-and-forget: the clone is
// cheap (Arc inside), the tokio::spawn never blocks us, a dead
// Langfuse just logs a warn. Client-visible response latency is
// untouched.
if let Some(lf) = &state.langfuse {
let output = resp.choices.first()
.map(|c| c.message.content.clone())
.unwrap_or_default();
lf.emit_chat(langfuse_trace::ChatTrace {
provider: provider.clone(),
model: resp.model.clone(),
input: req.messages.clone(),
output,
prompt_tokens: resp.usage.prompt_tokens,
completion_tokens: resp.usage.completion_tokens,
temperature: req.temperature,
max_tokens: req.max_tokens,
think: req.think,
start_time: start_time.to_rfc3339(),
end_time: end_time.to_rfc3339(),
latency_ms,
});
}
{
let mut u = state.usage.write().await;
u.requests += 1;