phase 4: chatd — multi-provider LLM dispatcher (ollama / cloud / openrouter / opencode / kimi)
new cmd/chatd on :3220 routes /v1/chat to the right provider based
on model-name prefix or :cloud suffix. closes the architectural gap
named in lakehouse.toml [models]: tiers map to model IDs, but until
phase 4 there was no service that could actually CALL those models
from go.
routing rules (registry.Resolve):
ollama/<m> → local Ollama (prefix stripped)
ollama_cloud/<m> → Ollama Cloud
<m>:cloud → Ollama Cloud (suffix variant — kimi-k2.6:cloud)
openrouter/<v>/<m> → OpenRouter (prefix stripped, OpenAI-compat)
opencode/<m> → OpenCode unified Zen+Go
kimi/<m> → Kimi For Coding (api.kimi.com/coding/v1)
bare names → local Ollama (default)
provider implementations:
- internal/chat/types.go Provider interface, Request/Response, errors
- internal/chat/registry.go prefix + :cloud suffix dispatch
- internal/chat/ollama.go local Ollama via /api/chat (think=false default)
- internal/chat/ollama_cloud.go Ollama Cloud via /api/generate (Bearer auth)
- internal/chat/openai_compat.go shared OpenAI Chat Completions for the
OpenRouter/OpenCode/Kimi family
- internal/chat/builder.go BuildRegistry from BuilderInput;
ResolveKey reads env then .env file fallback
config:
- ChatdConfig in internal/shared/config.go with bind, ollama_url,
per-provider key env names + .env fallback paths, timeout
- Gateway gains chatd_url + /v1/chat + /v1/chat/* routes
- lakehouse.toml [chatd] block with /etc/lakehouse/<provider>.env defaults
tests (19 in internal/chat):
- registry: prefix + :cloud + errors + telemetry + provider listing
- ollama: happy path + prefix strip + format=json + 500 mapping +
flatten_messages
- openai_compat: happy path + format=json + 429 mapping + zero-choices
think=false default in ollama + ollama_cloud — local hot path skips
reasoning, low-budget callers (the playbook_lift judge at max_tokens=10)
get direct answers instead of empty content + done_reason=length.
proven via chatd_smoke acceptance.
acceptance gate: scripts/chatd_smoke.sh — 6/6 PASS:
1. /v1/chat/providers lists exactly registered providers (1 in dev mode)
2. bare model → ollama default with content + token counts + latency
3. explicit ollama/<m> → prefix stripped at upstream
4. <m>:cloud without ollama_cloud registered → 404 (no silent fall-through)
5. unknown/<m> → falls through to default → upstream 502 (no prefix rewrite)
6. missing model field → 400
just verify: PASS (vet + 30 packages × short tests + 9 smokes).
chatd_smoke is a domain smoke (not in just verify, mirrors matrix /
observer / pathway pattern).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -15,6 +15,7 @@ embedd_url = "http://127.0.0.1:3216"
|
||||
pathwayd_url = "http://127.0.0.1:3217"
|
||||
matrixd_url = "http://127.0.0.1:3218"
|
||||
observerd_url = "http://127.0.0.1:3219"
|
||||
chatd_url = "http://127.0.0.1:3220"
|
||||
|
||||
[storaged]
|
||||
bind = "127.0.0.1:3211"
|
||||
@@ -70,6 +71,36 @@ bind = "127.0.0.1:3219"
|
||||
# /var/lib/lakehouse/observer/ops.jsonl so ops survive restart.
|
||||
persist_path = ""
|
||||
|
||||
[chatd]
|
||||
# LLM chat dispatcher (Phase 4). Routes /v1/chat to the right provider
|
||||
# based on model name prefix or :cloud suffix:
|
||||
# ollama/<model> → local Ollama (no auth)
|
||||
# ollama_cloud/<model> → Ollama Cloud
|
||||
# <model>:cloud → Ollama Cloud (suffix variant)
|
||||
# openrouter/<vendor>/<m> → OpenRouter
|
||||
# opencode/<model> → OpenCode (Zen+Go unified)
|
||||
# kimi/<model> → Kimi For Coding
|
||||
# bare names (e.g. qwen3.5:latest) → local Ollama (default)
|
||||
bind = "127.0.0.1:3220"
|
||||
ollama_url = "http://localhost:11434"
|
||||
|
||||
# Per-provider key resolution: env var first, then .env file fallback.
|
||||
# Empty file path skips the file lookup. systemd EnvironmentFile=
|
||||
# loads these natively, so service runtime sees the env vars.
|
||||
ollama_cloud_key_env = "OLLAMA_CLOUD_KEY"
|
||||
openrouter_key_env = "OPENROUTER_API_KEY"
|
||||
opencode_key_env = "OPENCODE_API_KEY"
|
||||
kimi_key_env = "KIMI_API_KEY"
|
||||
|
||||
ollama_cloud_key_file = "/etc/lakehouse/ollama_cloud.env"
|
||||
openrouter_key_file = "/etc/lakehouse/openrouter.env"
|
||||
opencode_key_file = "/etc/lakehouse/opencode.env"
|
||||
kimi_key_file = "/etc/lakehouse/kimi.env"
|
||||
|
||||
# Per-call timeout (seconds). Cloud reasoning models can take >60s
|
||||
# for long prompts, so 180 is the default.
|
||||
timeout_secs = 180
|
||||
|
||||
[s3]
|
||||
endpoint = "http://localhost:9000"
|
||||
region = "us-east-1"
|
||||
|
||||
Reference in New Issue
Block a user