Phase A + B (MVP) — local review harness
Implements the MVP cutline from the planning artifact: - Phase A: skeleton + CLI dispatch + provider interface + stub model doctor - Phase B: scanner + git probe + 12 static analyzers + reporters + pipeline - Phase B fixtures: clean-repo, insecure-repo, degraded-repo 12 static analyzers per PROMPT.md "Suggested Static Checks For MVP": hardcoded_paths, shell_execution, raw_sql_interpolation, broad_cors, secret_patterns, large_files, todo_comments, missing_tests, env_file_committed, unsafe_file_io, exposed_mutation_endpoint, hardcoded_local_ip. Acceptance gates passing: - B1 (intake produces accurate counts) ✓ - B2 (insecure fixture fires ≥8 distinct check_ids — actually 11/12) ✓ - B3 (clean fixture produces 0 confirmed findings — no false positives) ✓ - B4 (scrum mode produces all 6 required markdown + JSON reports) ✓ - B5 (receipts.json marks degraded phases honestly) ✓ - F (self-review on this repo runs without crashing) ✓ — exit 66 (degraded because Phase C LLM review is hardcoded skipped) Phases C (LLM review), D (validation cross-check), E (memory + diff + rules subcommands) deferred per the cutline. The MVP delivers the evidence-first path; LLM is purely additive. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
commit
f3ee4722a8
Executable
+86
@@ -0,0 +1,86 @@
|
||||
# Local Model Setup
|
||||
|
||||
## Purpose
|
||||
|
||||
The review harness should use local models first.
|
||||
|
||||
The first supported provider is Ollama.
|
||||
|
||||
The design must allow OpenAI-compatible local endpoints later.
|
||||
|
||||
## Default Ollama Profile
|
||||
|
||||
```yaml
|
||||
provider: ollama
|
||||
base_url: http://localhost:11434
|
||||
model: qwen2.5-coder
|
||||
fallback_model: llama3.1
|
||||
timeout_seconds: 120
|
||||
temperature: 0.1
|
||||
```
|
||||
|
||||
## Model Doctor Command
|
||||
|
||||
The harness must provide:
|
||||
|
||||
```bash
|
||||
review-harness model doctor
|
||||
```
|
||||
|
||||
## Doctor Checks
|
||||
|
||||
The doctor command should test:
|
||||
|
||||
- Ollama server availability
|
||||
- configured model availability
|
||||
- fallback model availability
|
||||
- basic prompt response
|
||||
- JSON response reliability
|
||||
- timeout behavior
|
||||
- degraded-mode behavior
|
||||
|
||||
## Required Doctor Output
|
||||
|
||||
```text
|
||||
reports/latest/model-doctor.json
|
||||
```
|
||||
|
||||
## Required JSON Fields
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "ollama",
|
||||
"base_url": "http://localhost:11434",
|
||||
"primary_model": "",
|
||||
"fallback_model": "",
|
||||
"server_available": false,
|
||||
"primary_model_available": false,
|
||||
"fallback_model_available": false,
|
||||
"basic_prompt_ok": false,
|
||||
"json_mode_ok": false,
|
||||
"timeout_seconds": 120,
|
||||
"status": "ok|degraded|failed",
|
||||
"errors": []
|
||||
}
|
||||
```
|
||||
|
||||
## Provider Interface
|
||||
|
||||
Do not hardcode Ollama into all logic.
|
||||
|
||||
Use a provider interface with these operations:
|
||||
|
||||
```text
|
||||
list_models()
|
||||
complete(prompt, options)
|
||||
complete_json(prompt, schema, options)
|
||||
health_check()
|
||||
```
|
||||
|
||||
## Local Model Rules
|
||||
|
||||
- temperature should default low for review tasks
|
||||
- prompts should request strict JSON where possible
|
||||
- raw model output must be saved for failed parse attempts
|
||||
- invalid model output must never be silently accepted
|
||||
- fallback model usage must be recorded
|
||||
Reference in New Issue
Block a user