profit/golangLAKEHOUSE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

shared: auto-emit Langfuse trace+span per HTTP request — closes OPEN #2

Browse Source 
Adds langfuseMiddleware in internal/shared so every daemon's
shared.Run gets free production-traffic trace visibility when
LANGFUSE_URL + LANGFUSE_PUBLIC_KEY + LANGFUSE_SECRET_KEY are set.
Same env names + file shape as the multi_coord_stress driver, so
operators ship one /etc/lakehouse/langfuse.env across the deploy.

Wiring is auth-gated: middleware runs INSIDE the RequireAuth group,
so 401s from credential-stuffing don't pollute traces. /health is
exempt so LB probes don't either. Missing env vars → nil client →
middleware is a passthrough no-op (fail-open per ADR-005 5.1).

Bundled deploy:
- langfuse.env.example template (mode 0640, root:lakehouse)
- 11 systemd units gain `EnvironmentFile=-/etc/lakehouse/langfuse.env`
  (leading - so missing file = OK)
- REPLICATION.md bootstrap section documents setup

Tests (4): nil passthrough, /health bypass, real-request emission,
status-writer wrapping. All green.

STATE_OF_PLAY OPEN list: 5 rows → 4 rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-30 19:55:42 -05:00
co-authored by Claude Opus 4.7
parent 5a3364f539
commit 68d9e554b0
17 changed files with 311 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/etc-lakehouse/langfuse.env.example
+19
View File
@@ -0,0 +1,19 @@
# /etc/lakehouse/langfuse.env — Langfuse trace observability.
#
# Mode 0640, root:lakehouse. Loaded by every daemon's systemd unit
# via EnvironmentFile=-/etc/lakehouse/langfuse.env. The `-` prefix
# means "missing file is OK" — Langfuse is optional. If the file is
# missing or any of the three env vars below is empty, every daemon
# runs without Langfuse and with no logged warnings (best-effort).
#
# When all three are set, every authenticated HTTP request to every
# daemon emits one Langfuse trace + span, browseable at LANGFUSE_URL.
# /health is exempt to keep LB probes from polluting traces.
#
# Same env vars + same shape as the file the multi_coord_stress
# driver reads — operators ship one langfuse.env across the whole
# deploy.
LANGFUSE_URL=http://localhost:3001
LANGFUSE_PUBLIC_KEY=pk-lf-staffing
LANGFUSE_SECRET_KEY=sk-lf-staffing-secret
deploy/systemd/lakehouse-catalogd.service
+1
View File
@@ -15,6 +15,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-chatd.service
+1
View File
@@ -19,6 +19,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
# chatd reads provider key files via paths in lakehouse.toml [chatd]
# (ollama_cloud_key_file etc.) — each is its own EnvironmentFile so
# operators can rotate one provider without restarting others.
deploy/systemd/lakehouse-embedd.service
+1
View File
@@ -18,6 +18,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-gateway.service
+1
View File
@@ -17,6 +17,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-ingestd.service
+1
View File
@@ -15,6 +15,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-matrixd.service
+1
View File
@@ -15,6 +15,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-observerd.service
+1
View File
@@ -17,6 +17,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-pathwayd.service
+1
View File
@@ -14,6 +14,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-queryd.service
+1
View File
@@ -18,6 +18,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict
deploy/systemd/lakehouse-storaged.service
+1
View File
@@ -21,6 +21,7 @@ RestartSec=5
# Empty AUTH_TOKEN is fine for loopback-only deploys (matches
# requireAuthOnNonLoopback gate at startup).
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
# Hardening — minimum needed for the daemon to read its config
# + write its log + open its bind port.
deploy/systemd/lakehouse-vectord.service
+1
View File
@@ -17,6 +17,7 @@ Restart=on-failure
RestartSec=5
EnvironmentFile=-/etc/lakehouse/auth.env
EnvironmentFile=-/etc/lakehouse/langfuse.env
NoNewPrivileges=true
ProtectSystem=strict