phase 4: chatd — multi-provider LLM dispatcher (ollama / cloud / openrouter / opencode / kimi)
new cmd/chatd on :3220 routes /v1/chat to the right provider based
on model-name prefix or :cloud suffix. closes the architectural gap
named in lakehouse.toml [models]: tiers map to model IDs, but until
phase 4 there was no service that could actually CALL those models
from go.
routing rules (registry.Resolve):
ollama/<m> → local Ollama (prefix stripped)
ollama_cloud/<m> → Ollama Cloud
<m>:cloud → Ollama Cloud (suffix variant — kimi-k2.6:cloud)
openrouter/<v>/<m> → OpenRouter (prefix stripped, OpenAI-compat)
opencode/<m> → OpenCode unified Zen+Go
kimi/<m> → Kimi For Coding (api.kimi.com/coding/v1)
bare names → local Ollama (default)
provider implementations:
- internal/chat/types.go Provider interface, Request/Response, errors
- internal/chat/registry.go prefix + :cloud suffix dispatch
- internal/chat/ollama.go local Ollama via /api/chat (think=false default)
- internal/chat/ollama_cloud.go Ollama Cloud via /api/generate (Bearer auth)
- internal/chat/openai_compat.go shared OpenAI Chat Completions for the
OpenRouter/OpenCode/Kimi family
- internal/chat/builder.go BuildRegistry from BuilderInput;
ResolveKey reads env then .env file fallback
config:
- ChatdConfig in internal/shared/config.go with bind, ollama_url,
per-provider key env names + .env fallback paths, timeout
- Gateway gains chatd_url + /v1/chat + /v1/chat/* routes
- lakehouse.toml [chatd] block with /etc/lakehouse/<provider>.env defaults
tests (19 in internal/chat):
- registry: prefix + :cloud + errors + telemetry + provider listing
- ollama: happy path + prefix strip + format=json + 500 mapping +
flatten_messages
- openai_compat: happy path + format=json + 429 mapping + zero-choices
think=false default in ollama + ollama_cloud — local hot path skips
reasoning, low-budget callers (the playbook_lift judge at max_tokens=10)
get direct answers instead of empty content + done_reason=length.
proven via chatd_smoke acceptance.
acceptance gate: scripts/chatd_smoke.sh — 6/6 PASS:
1. /v1/chat/providers lists exactly registered providers (1 in dev mode)
2. bare model → ollama default with content + token counts + latency
3. explicit ollama/<m> → prefix stripped at upstream
4. <m>:cloud without ollama_cloud registered → 404 (no silent fall-through)
5. unknown/<m> → falls through to default → upstream 502 (no prefix rewrite)
6. missing model field → 400
just verify: PASS (vet + 30 packages × short tests + 9 smokes).
chatd_smoke is a domain smoke (not in just verify, mirrors matrix /
observer / pathway pattern).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
@@ -29,6 +29,7 @@ type Config struct {
|
||||
Pathwayd PathwaydConfig `toml:"pathwayd"`
|
||||
Matrixd MatrixdConfig `toml:"matrixd"`
|
||||
Observerd ObserverdConfig `toml:"observerd"`
|
||||
Chatd ChatdConfig `toml:"chatd"`
|
||||
S3 S3Config `toml:"s3"`
|
||||
Models ModelsConfig `toml:"models"`
|
||||
Log LogConfig `toml:"log"`
|
||||
@@ -68,6 +69,7 @@ type GatewayConfig struct {
|
||||
PathwaydURL string `toml:"pathwayd_url"`
|
||||
MatrixdURL string `toml:"matrixd_url"`
|
||||
ObserverdURL string `toml:"observerd_url"`
|
||||
ChatdURL string `toml:"chatd_url"`
|
||||
}
|
||||
|
||||
// EmbeddConfig drives the embed service. ProviderURL points at the
|
||||
@@ -111,6 +113,36 @@ type MatrixdConfig struct {
|
||||
VectordURL string `toml:"vectord_url"`
|
||||
}
|
||||
|
||||
// ChatdConfig drives the chat dispatcher service (cmd/chatd) — Phase 4.
|
||||
// Routes /v1/chat to the right provider based on model-name prefix
|
||||
// or :cloud suffix. Per-provider API keys come from env vars (or
|
||||
// /etc/lakehouse/<provider>.env files); empty keys leave the provider
|
||||
// unregistered so requests for that provider 404 cleanly.
|
||||
//
|
||||
// OllamaURL is the local Ollama upstream (no auth). Empty disables
|
||||
// the local Ollama provider — useful in deployments that don't run
|
||||
// Ollama on the box (cloud-only operation).
|
||||
type ChatdConfig struct {
|
||||
Bind string `toml:"bind"`
|
||||
OllamaURL string `toml:"ollama_url"`
|
||||
// API-key env var names. Default to the conventional names.
|
||||
// Operators can rename these for environments using different env
|
||||
// var conventions, but the defaults match /etc/lakehouse/*.env
|
||||
// files that systemd already loads.
|
||||
OllamaCloudKeyEnv string `toml:"ollama_cloud_key_env"`
|
||||
OpenRouterKeyEnv string `toml:"openrouter_key_env"`
|
||||
OpenCodeKeyEnv string `toml:"opencode_key_env"`
|
||||
KimiKeyEnv string `toml:"kimi_key_env"`
|
||||
// Optional .env file paths — a fallback when the env var isn't set
|
||||
// in the process environment. Same keys, "KEY=value" lines.
|
||||
OllamaCloudKeyFile string `toml:"ollama_cloud_key_file"`
|
||||
OpenRouterKeyFile string `toml:"openrouter_key_file"`
|
||||
OpenCodeKeyFile string `toml:"opencode_key_file"`
|
||||
KimiKeyFile string `toml:"kimi_key_file"`
|
||||
// Per-call timeout in seconds. 0 = 180s default.
|
||||
TimeoutSecs int `toml:"timeout_secs"`
|
||||
}
|
||||
|
||||
// ObserverdConfig drives the observer service (cmd/observerd).
|
||||
// PersistPath: file path to the JSONL ops log; empty = in-memory
|
||||
// only (test/dev). Production sets a stable path under
|
||||
@@ -284,6 +316,7 @@ func DefaultConfig() Config {
|
||||
PathwaydURL: "http://127.0.0.1:3217",
|
||||
MatrixdURL: "http://127.0.0.1:3218",
|
||||
ObserverdURL: "http://127.0.0.1:3219",
|
||||
ChatdURL: "http://127.0.0.1:3220",
|
||||
},
|
||||
Storaged: ServiceConfig{Bind: "127.0.0.1:3211"},
|
||||
Catalogd: CatalogConfig{Bind: "127.0.0.1:3212", StoragedURL: "http://127.0.0.1:3211"},
|
||||
@@ -317,6 +350,19 @@ func DefaultConfig() Config {
|
||||
Bind: "127.0.0.1:3219",
|
||||
// PersistPath empty by default = in-memory only.
|
||||
},
|
||||
Chatd: ChatdConfig{
|
||||
Bind: "127.0.0.1:3220",
|
||||
OllamaURL: "http://localhost:11434",
|
||||
OllamaCloudKeyEnv: "OLLAMA_CLOUD_KEY",
|
||||
OpenRouterKeyEnv: "OPENROUTER_API_KEY",
|
||||
OpenCodeKeyEnv: "OPENCODE_API_KEY",
|
||||
KimiKeyEnv: "KIMI_API_KEY",
|
||||
OllamaCloudKeyFile: "/etc/lakehouse/ollama_cloud.env",
|
||||
OpenRouterKeyFile: "/etc/lakehouse/openrouter.env",
|
||||
OpenCodeKeyFile: "/etc/lakehouse/opencode.env",
|
||||
KimiKeyFile: "/etc/lakehouse/kimi.env",
|
||||
TimeoutSecs: 180,
|
||||
},
|
||||
Queryd: QuerydConfig{
|
||||
Bind: "127.0.0.1:3214",
|
||||
CatalogdURL: "http://127.0.0.1:3212",
|
||||
|
||||
Reference in New Issue
Block a user