profit/golangLAKEHOUSE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

G2: embedd — text → vector via Ollama · 2 scrum fixes

Browse Source 
Bridges the missing piece for the staffing co-pilot: text inputs to
vectord-shaped vectors. Standalone cmd/embedd on :3216 fronted by
gateway at /v1/embed. Pluggable embed.Provider interface (G2 ships
Ollama; OpenAI/Voyage swap in via the same interface in G3+).

Wire format:
  POST /v1/embed {"texts":[...], "model":"..."}  // model optional
  → 200 {"model","dimension","vectors":[[...]]}

Default model: nomic-embed-text (768-d). Ollama returns float64;
provider converts to float32 at the boundary so vectors flow through
vectord/HNSW without re-conversion.

Acceptance smoke 5/5 PASS — including the architectural payoff:
end-to-end embed → vectord add → search by re-embedded text returns
recall=1 at distance 5.96e-8 (float32 precision noise on identical
unit vectors). The staffing co-pilot pipeline (text → vector →
similarity search) is now functional end-to-end.

All 9 smokes (D1-D6 + G1 + G1P + G2) PASS deterministically.

Cross-lineage scrum on shipped code:
  - Opus 4.7 (opencode):                    0 BLOCK + 4 WARN + 3 INFO
  - Kimi K2-0905 (openrouter):              0 BLOCK + 2 WARN + 1 INFO
  - Qwen3-coder (openrouter):               "No BLOCKs" (3 tokens)

Fixed (2 — 1 convergent + 1 single-reviewer):
  C1 (Opus + Kimi convergent WARN): per-text 60s timeout × N-text
    batch was up to N×60s with no batch-level cap. One stuck Ollama
    call would stall the whole handler indefinitely. Fix:
    context.WithTimeout(r.Context(), 60s) wraps the entire batch.
  O-W3 (Opus WARN): empty strings in texts went to Ollama unchecked,
    producing version-dependent garbage. Fix: reject "" with 400 at
    the handler boundary so callers get a deterministic answer
    instead of an upstream-conditional 502.

Deferred (4): drainAndClose 64KiB cap (matches G0 pattern), no
concurrency limit on /embed (single-tenant G2), missing Accept
header (exotic-proxy concern), MaxBytesError string-match
redundancy (paranoia layer kept consistent across codebase).

Zero false positives this round — Qwen returned 3 tokens "No BLOCKs"
and the other two reviewers' findings were all real.

Setup confirmed: Ollama 0.21.0 on :11434 with nomic-embed-text loaded.
Per-text /api/embeddings used (forward-compat with 0.21+); newer
0.4+ /api/embed batch endpoint can swap in via the Provider interface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-29 01:42:27 -05:00
co-authored by Claude Opus 4.7
parent 8b92518d21
commit 9ee7fc5550
8 changed files with 624 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/embed/embed.go
+43
View File
@@ -0,0 +1,43 @@
// Package embed turns text into vectors. G2 ships an Ollama-backed
// Provider (local model on :11434, no network call to a hosted
// service); the Provider interface keeps the door open for OpenAI /
// Voyage / etc. without per-call switching at the HTTP boundary.
//
// Vectors are float32 throughout the system (matches vectord +
// coder/hnsw types). Ollama returns float64; we convert at this
// package's boundary so callers don't have to think about it.
package embed
import (
"context"
"errors"
)
// Provider is the embed boundary. Embed takes a list of input
// texts and returns the same number of vectors, in input order,
// using the named model. Empty model = use the Provider's default.
type Provider interface {
Embed(ctx context.Context, texts []string, model string) (Result, error)
}
// Result is the per-call response shape. Model echoes which model
// was actually used (after default-resolution); Dimension is the
// vector dimension reported by the model. The Provider guarantees
// every Vector in Vectors has Dimension components.
type Result struct {
Model string `json:"model"`
Dimension int `json:"dimension"`
Vectors [][]float32 `json:"vectors"`
}
// ErrEmptyTexts is returned when the caller passed no inputs.
// Empty-batch is an error because returning an empty result
// silently wastes the round trip and hides caller bugs.
var ErrEmptyTexts = errors.New("embed: empty texts")
// ErrModelMismatch is returned when one call to the upstream
// returns a different vector dimension than the previous call in
// the same batch — only possible if the model actually switched
// mid-batch (load swap on the server). Surfaced as a 502 by
// callers because the upstream's behavior was inconsistent.
var ErrModelMismatch = errors.New("embed: dimension changed mid-batch")
internal/embed/ollama.go
+127
View File
@@ -0,0 +1,127 @@
// ollama.go — Provider backed by an Ollama HTTP server. Compatible
// with Ollama 0.21+ via the per-text /api/embeddings endpoint.
// Newer Ollama (0.4+) exposes /api/embed for batched calls, but
// the per-text loop is forward-compatible with both.
package embed
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
)
// OllamaProvider hits an Ollama server at the configured base URL.
type OllamaProvider struct {
baseURL string
defaultModel string
hc *http.Client
}
// NewOllama builds a provider against baseURL (e.g.
// "http://localhost:11434"). defaultModel is what gets used when
// callers pass an empty model name.
func NewOllama(baseURL, defaultModel string) *OllamaProvider {
return &OllamaProvider{
baseURL: strings.TrimRight(baseURL, "/"),
defaultModel: defaultModel,
hc: &http.Client{
// Embeddings are CPU-bound on the server side; 60s gives
// plenty of headroom for a single-text call. Caller can
// add an outer ctx deadline for batch-level cap.
Timeout: 60 * time.Second,
},
}
}
// ollamaRequest is Ollama's /api/embeddings body shape.
type ollamaRequest struct {
Model string `json:"model"`
Prompt string `json:"prompt"`
}
// ollamaResponse mirrors the success body. Embedding is float64
// from Ollama; we convert to float32 at the boundary.
type ollamaResponse struct {
Embedding []float64 `json:"embedding"`
}
// Embed loops over texts, issuing one HTTP call per text. Errors
// short-circuit — if call N fails, we return the error and the
// caller sees no partial Result.
func (p *OllamaProvider) Embed(ctx context.Context, texts []string, model string) (Result, error) {
if len(texts) == 0 {
return Result{}, ErrEmptyTexts
}
if model == "" {
model = p.defaultModel
}
if model == "" {
return Result{}, fmt.Errorf("embed: no model (empty request, no default)")
}
out := Result{Model: model, Vectors: make([][]float32, 0, len(texts))}
for i, text := range texts {
vec, err := p.embedOne(ctx, model, text)
if err != nil {
return Result{}, fmt.Errorf("embed text[%d]: %w", i, err)
}
// Per-text vectors must agree on dimension. Lock on first.
if out.Dimension == 0 {
out.Dimension = len(vec)
} else if len(vec) != out.Dimension {
return Result{}, fmt.Errorf("%w: text[%d] returned %d, prior were %d",
ErrModelMismatch, i, len(vec), out.Dimension)
}
out.Vectors = append(out.Vectors, vec)
}
return out, nil
}
func (p *OllamaProvider) embedOne(ctx context.Context, model, text string) ([]float32, error) {
body, err := json.Marshal(ollamaRequest{Model: model, Prompt: text})
if err != nil {
return nil, fmt.Errorf("marshal: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
p.baseURL+"/api/embeddings", bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("req: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.ContentLength = int64(len(body))
resp, err := p.hc.Do(req)
if err != nil {
return nil, fmt.Errorf("do: %w", err)
}
defer drainAndClose(resp.Body)
if resp.StatusCode != http.StatusOK {
preview, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
return nil, fmt.Errorf("upstream status %d: %s", resp.StatusCode, string(preview))
}
var or ollamaResponse
if err := json.NewDecoder(resp.Body).Decode(&or); err != nil {
return nil, fmt.Errorf("decode: %w", err)
}
if len(or.Embedding) == 0 {
return nil, fmt.Errorf("upstream returned empty embedding")
}
// Float64 → Float32. Loss of precision is acceptable for HNSW
// search; float32 matches the rest of the system.
out := make([]float32, len(or.Embedding))
for i, v := range or.Embedding {
out[i] = float32(v)
}
return out, nil
}
func drainAndClose(body io.ReadCloser) {
_, _ = io.Copy(io.Discard, io.LimitReader(body, 64<<10))
_ = body.Close()
}
internal/embed/ollama_test.go
+115
View File
@@ -0,0 +1,115 @@
package embed
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
func TestOllama_EmbedBatch_PreservesOrder(t *testing.T) {
var mu sync.Mutex
var seenPrompts []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var req ollamaRequest
_ = json.NewDecoder(r.Body).Decode(&req)
mu.Lock()
seenPrompts = append(seenPrompts, req.Prompt)
mu.Unlock()
// Return a vector that encodes which prompt this was, so
// we can assert order at the caller. 4-d vector for cheap.
var vec [4]float64
switch req.Prompt {
case "alpha":
vec = [4]float64{1, 0, 0, 0}
case "beta":
vec = [4]float64{0, 1, 0, 0}
case "gamma":
vec = [4]float64{0, 0, 1, 0}
}
_ = json.NewEncoder(w).Encode(map[string]any{"embedding": vec[:]})
}))
defer srv.Close()
p := NewOllama(srv.URL, "test-model")
res, err := p.Embed(context.Background(), []string{"alpha", "beta", "gamma"}, "")
if err != nil {
t.Fatal(err)
}
if res.Model != "test-model" || res.Dimension != 4 || len(res.Vectors) != 3 {
t.Fatalf("Result: got %+v", res)
}
if res.Vectors[0][0] != 1 || res.Vectors[1][1] != 1 || res.Vectors[2][2] != 1 {
t.Errorf("vectors out of order: %v", res.Vectors)
}
// Sanity: all three prompts hit the server.
if len(seenPrompts) != 3 {
t.Errorf("expected 3 upstream calls, got %d", len(seenPrompts))
}
}
func TestOllama_EmptyTextsErrors(t *testing.T) {
p := NewOllama("http://nope:0", "x")
_, err := p.Embed(context.Background(), nil, "")
if !errors.Is(err, ErrEmptyTexts) {
t.Errorf("expected ErrEmptyTexts, got %v", err)
}
}
func TestOllama_NoModelNoDefault(t *testing.T) {
p := NewOllama("http://nope:0", "") // empty default
_, err := p.Embed(context.Background(), []string{"hi"}, "")
if err == nil || !strings.Contains(err.Error(), "no model") {
t.Errorf("expected no-model error, got %v", err)
}
}
func TestOllama_UpstreamErrorPropagates(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "model not loaded", http.StatusInternalServerError)
}))
defer srv.Close()
p := NewOllama(srv.URL, "x")
_, err := p.Embed(context.Background(), []string{"hi"}, "")
if err == nil || !strings.Contains(err.Error(), "upstream status 500") {
t.Errorf("expected wrapped 500 error, got %v", err)
}
}
func TestOllama_DimensionMismatchMidBatch(t *testing.T) {
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
calls++
// First call returns 4-d, second returns 8-d → server changed
// model under us. Provider should ErrModelMismatch.
var v []float64
if calls == 1 {
v = []float64{1, 0, 0, 0}
} else {
v = []float64{1, 0, 0, 0, 0, 0, 0, 0}
}
_ = json.NewEncoder(w).Encode(map[string]any{"embedding": v})
}))
defer srv.Close()
p := NewOllama(srv.URL, "x")
_, err := p.Embed(context.Background(), []string{"a", "b"}, "")
if !errors.Is(err, ErrModelMismatch) {
t.Errorf("expected ErrModelMismatch, got %v", err)
}
}
func TestOllama_EmptyEmbeddingErrors(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"embedding": []float64{}})
}))
defer srv.Close()
p := NewOllama(srv.URL, "x")
_, err := p.Embed(context.Background(), []string{"hi"}, "")
if err == nil || !strings.Contains(err.Error(), "empty embedding") {
t.Errorf("expected empty-embedding error, got %v", err)
}
}