Phase C — local-Ollama LLM review wired end-to-end

Implements PROMPT.md / docs/REVIEW_PIPELINE.md Phase 2:
- internal/llm/ollama.go — real Ollama provider:
  - HealthCheck probes /api/tags + a 1-token completion + a JSON-mode
    probe ({"ok": true} round-trip), populating the model-doctor.json
    schema documented in docs/LOCAL_MODEL_SETUP.md
  - Complete + CompleteJSON via /api/chat with stream=false
  - think=false set for ALL completions (qwen3.5:latest is reasoning-
    capable but the inner-loop hot path wants direct answers, not
    reasoning traces consuming the token budget — same finding as
    the Lakehouse-Go chatd 2026-04-30 wave)
- internal/llm/review.go — Reviewer wrapper:
  - 2-attempt flow: prompt → parse → repair-prompt → parse
  - Strict JSON shape enforced; markdown fences stripped before parse
  - Severity normalized to enum; out-of-range confidence clamped
  - Per-file chunking (file-level for v0; function-level Phase D+)
  - Bounded by review-profile max_file_bytes + max_llm_chunk_chars
- pipeline.go — Phase 2 wired between static scan + report gen:
  - --enable-llm flag opts in (off by default — static-only is
    cheaper and faster)
  - Raw output ALWAYS saved to llm-findings.raw.json (forensics)
  - Normalized findings → llm-findings.normalized.json
  - LLM findings merged into the report findings list (sourced
    "llm" so consumers can filter)
  - Receipts honestly mark phase status: "ok" | "degraded" | "skipped"
- cli model doctor — real probes replace the Phase A stub.

Verified:
- model doctor: status="ok" with qwen3.5:latest + qwen3:latest both
  loaded, basic_prompt_ok=true, json_mode_ok=true
- insecure-repo with --enable-llm: 9 LLM findings; qwen3.5 correctly
  flagged SQLi, RCE, hardcoded credentials as critical with verbatim
  evidence; 27s wall for 3 chunks
- clean-repo with --enable-llm: 0 LLM findings, 4 parsed chunks, 2.8s
- self-review with --enable-llm: 77 LLM findings + 83 static; 3 of
  ~30 chunks needed retry (PROMPT.md, REPORT_SCHEMA.md,
  SCRUM_TEST_TEMPLATE.md — all eventually parsed); 5min wall

go vet + go test -short clean. Fixture stray.go now `package fixture`
so go-tooling doesn't choke on the orphan.

Phase D (validator cross-check) + Phase E (memory + diff/rules
subcommands) remain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude (review-harness setup)
2026-04-30 01:13:39 -05:00
co-authored by Claude Opus 4.7
parent 70d68757f7
commit e346b54e0f
27 changed files with 1785 additions and 22 deletions
@@ -0,0 +1,8 @@
# Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
@@ -0,0 +1,8 @@
# Claim Coverage Table
Each row is a finding paired with whether existing tests cover the affected area.
Phase B emits this shape; LLM-side claim generation lands in Phase C.
| Claim | Code Location | Existing Test | Missing Test | Risk |
|---|---|---|---|---|
| _no claims yet_ | — | — | — | — |
@@ -0,0 +1,16 @@
{
"generated_at": "2026-04-30T06:06:56.669606679Z",
"findings": [],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
@@ -0,0 +1,30 @@
[
{
"chunk_id": "README.md",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "package.json",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/calc.ts",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "tests/calc.test.ts",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
}
]
+70
View File
@@ -0,0 +1,70 @@
{
"run_id": "20260430T060653-57461e2c",
"repo_path": "tests/fixtures/clean-repo",
"started_at": "2026-04-30T06:06:53.880883402Z",
"finished_at": "2026-04-30T06:06:56.669797363Z",
"phases": [
{
"name": "repo_intake",
"status": "ok",
"output_hash": "db312c5ce39315cd",
"output_files": [
"repo-intake.json"
]
},
{
"name": "static_scan",
"status": "ok",
"output_hash": "837b6a5d9dc11126",
"output_files": [
"static-findings.json"
]
},
{
"name": "llm_review",
"status": "ok",
"output_hash": "3939252dabe358b1",
"output_files": [
"llm-findings.raw.json",
"llm-findings.normalized.json"
]
},
{
"name": "validation",
"status": "skipped",
"errors": [
"Phase D not implemented in MVP — depends on Phase C"
]
},
{
"name": "report_generation",
"status": "ok",
"output_files": [
"scrum-test.md",
"risk-register.md",
"claim-coverage-table.md",
"sprint-backlog.md",
"acceptance-gates.md"
]
},
{
"name": "memory_update",
"status": "skipped",
"errors": [
"Phase E not implemented in MVP"
]
}
],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
@@ -0,0 +1,42 @@
{
"repo_path": "/home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo",
"current_branch": "main",
"latest_commit": "70d68757f78ea722bf24585c73120c09d82c4fea",
"git_status": "M ../../../internal/cli/cli.go\n M ../../../internal/cli/repo.go\n M ../../../internal/pipeline/pipeline.go\n?? ../../../internal/llm/ollama.go\n?? ../../../internal/llm/review.go\n?? ../insecure-repo/reports/",
"has_git": true,
"file_count": 4,
"language_breakdown": {
"JSON": 1,
"Markdown": 1,
"TypeScript": 2
},
"largest_files": [
{
"path": "src/calc.ts",
"size": 206,
"lines": 7
},
{
"path": "tests/calc.test.ts",
"size": 198,
"lines": 4
},
{
"path": "README.md",
"size": 80,
"lines": 2
},
{
"path": "package.json",
"size": 43,
"lines": 1
}
],
"dependency_manifests": [
"package.json"
],
"test_manifests": [
"tests/calc.test.ts"
],
"generated_at": "2026-04-30T06:06:53.895008668Z"
}
@@ -0,0 +1,5 @@
# Risk Register
Findings ranked by severity. `Suspected` rows haven't been validated yet (Phase D).
_No findings._
+69
View File
@@ -0,0 +1,69 @@
# Scrum Test — clean-repo
**Generated:** 2026-04-30T06:06:53.895008668Z
**Branch:** main · **Commit:** 70d68757f78ea722bf24585c73120c09d82c4fea
## Verdict
**production-ready** — static scan + LLM review found no issues. Re-validate after every wave.
## Evidence
- repo path: `/home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
- file count: 4
- languages: TypeScript (2), Markdown (1), JSON (1)
- dependency manifests: 1 (package.json)
- test files/dirs: 1
## Confirmed Risks
_No confirmed risks at static-scan level. (LLM review may surface more.)_
## Suspected Risks
_None._
## Blocked Checks
_None._
## Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
**Sprint 1 — Trust Boundary Gate**
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
## Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
## Next Commands
- Re-run after fixes: `review-harness repo /home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
- Generate the full Scrum bundle: `review-harness scrum /home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
@@ -0,0 +1,26 @@
# Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
**Sprint 1 — Trust Boundary Gate**
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
@@ -0,0 +1,16 @@
{
"generated_at": "2026-04-30T06:06:53.896533109Z",
"findings": [],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
+6
View File
@@ -1 +1,7 @@
// Fixture: orphan source file in a non-git directory. Exists so the
// scanner has something to walk. Marked package fixture so `go vet`
// over the harness's own module doesn't choke on the orphan; the
// harness itself reads this file as raw text and doesn't care.
package fixture
// just a stray file
@@ -0,0 +1,9 @@
# Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
5. **Critical findings block production deploy:** at least one critical finding is currently present; resolve before deploy.
@@ -0,0 +1,23 @@
# Claim Coverage Table
Each row is a finding paired with whether existing tests cover the affected area.
Phase B emits this shape; LLM-side claim generation lands in Phase C.
| Claim | Code Location | Existing Test | Missing Test | Risk |
|---|---|---|---|---|
| Environment file in source tree | `.env:?` | _unknown_ | _likely_ | high |
| Hardcoded absolute path | `src/handler.go:10` | _unknown_ | _likely_ | medium |
| Shell command execution | `src/handler.go:19` | _unknown_ | _likely_ | high |
| Raw SQL interpolation | `src/handler.go:14` | _unknown_ | _likely_ | high |
| Possible secret committed to source | `src/handler.go:23` | _unknown_ | _likely_ | critical |
| Possible secret committed to source | `src/handler.go:23` | _unknown_ | _likely_ | critical |
| TODO/FIXME comment | `src/handler.go:9` | _unknown_ | _likely_ | low |
| TODO/FIXME comment | `src/handler.go:22` | _unknown_ | _likely_ | low |
| Hardcoded private-network IP | `src/handler.go:11` | _unknown_ | _likely_ | medium |
| Large file | `src/huge.go:1-901` | _unknown_ | _likely_ | medium |
| Wildcard CORS | `src/server.js:2` | _unknown_ | _likely_ | high |
| Possible secret committed to source | `src/server.js:5` | _unknown_ | _likely_ | critical |
| TODO/FIXME comment | `src/server.js:1` | _unknown_ | _likely_ | low |
| Mutation route in file with no visible auth | `src/server.js:7` | _unknown_ | _likely_ | medium |
| Mutation route in file with no visible auth | `src/server.js:8` | _unknown_ | _likely_ | medium |
| No tests found | `.:?` | _unknown_ | _likely_ | medium |
@@ -0,0 +1,147 @@
{
"generated_at": "2026-04-30T06:06:33.240219171Z",
"findings": [
{
"id": "",
"title": "Hardcoded file path for secrets",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Hardcoding a file path for a private key exposes secrets and prevents proper secret management.",
"suggested_fix": "Move the path to an environment variable or a configuration file outside the source code.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded server IP address",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "Hardcoding an IP address reduces portability and may leak internal network topology.",
"suggested_fix": "Read the server IP from an environment variable.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "SQL Injection vulnerability",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "15-16",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)\ndb.Query(q)",
"reason": "Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.",
"suggested_fix": "Use parameterized queries with placeholders instead of string formatting.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Unsafe shell command execution",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19-20",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Executing arbitrary shell commands without validation allows for remote code execution.",
"suggested_fix": "Validate and sanitize the input command strictly, or avoid using shell execution entirely.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded API key",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.",
"suggested_fix": "Store the API key in a secure environment variable or secrets manager.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "CORS misconfiguration allows cross-origin attacks",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.",
"suggested_fix": "Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded AWS access key in source code",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.",
"suggested_fix": "Use environment variables or a secure secrets manager to store AWS credentials.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on user creation endpoint",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.",
"suggested_fix": "Implement authentication middleware to verify user identity before allowing POST requests.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on admin deletion endpoint",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.",
"suggested_fix": "Implement strict authentication and authorization checks for all admin endpoints.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"summary": {
"total": 9,
"confirmed": 0,
"suspected": 9,
"rejected": 0,
"critical": 5,
"high": 3,
"medium": 1,
"low": 0,
"by_source": {
"llm": 9
},
"by_check": {
"llm.review": 9
}
}
}
@@ -0,0 +1,151 @@
[
{
"chunk_id": "src/handler.go",
"findings": [
{
"id": "",
"title": "Hardcoded file path for secrets",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Hardcoding a file path for a private key exposes secrets and prevents proper secret management.",
"suggested_fix": "Move the path to an environment variable or a configuration file outside the source code.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded server IP address",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "Hardcoding an IP address reduces portability and may leak internal network topology.",
"suggested_fix": "Read the server IP from an environment variable.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "SQL Injection vulnerability",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "15-16",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)\ndb.Query(q)",
"reason": "Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.",
"suggested_fix": "Use parameterized queries with placeholders instead of string formatting.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Unsafe shell command execution",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19-20",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Executing arbitrary shell commands without validation allows for remote code execution.",
"suggested_fix": "Validate and sanitize the input command strictly, or avoid using shell execution entirely.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded API key",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.",
"suggested_fix": "Store the API key in a secure environment variable or secrets manager.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"raw_content": "{\n \"findings\": [\n {\n \"title\": \"Hardcoded file path for secrets\",\n \"severity\": \"high\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"10\",\n \"evidence\": \"const HARDCODED_PATH = \\\"/home/profit/secrets/key.pem\\\"\",\n \"reason\": \"Hardcoding a file path for a private key exposes secrets and prevents proper secret management.\",\n \"suggested_fix\": \"Move the path to an environment variable or a configuration file outside the source code.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded server IP address\",\n \"severity\": \"medium\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"11\",\n \"evidence\": \"const SERVER_IP = \\\"192.168.1.176\\\"\",\n \"reason\": \"Hardcoding an IP address reduces portability and may leak internal network topology.\",\n \"suggested_fix\": \"Read the server IP from an environment variable.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"SQL Injection vulnerability\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"15-16\",\n \"evidence\": \"q := fmt.Sprintf(\\\"SELECT * FROM users WHERE name = '%s'\\\", name)\\ndb.Query(q)\",\n \"reason\": \"Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.\",\n \"suggested_fix\": \"Use parameterized queries with placeholders instead of string formatting.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Unsafe shell command execution\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"19-20\",\n \"evidence\": \"exec.Command(\\\"bash\\\", \\\"-c\\\", cmd).Run()\",\n \"reason\": \"Executing arbitrary shell commands without validation allows for remote code execution.\",\n \"suggested_fix\": \"Validate and sanitize the input command strictly, or avoid using shell execution entirely.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded API key\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"23\",\n \"evidence\": \"const API_KEY = \\\"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\\\"\",\n \"reason\": \"Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.\",\n \"suggested_fix\": \"Store the API key in a secure environment variable or secrets manager.\",\n \"confidence\": 1.0\n }\n ]\n}",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/huge.go",
"findings": [],
"raw_content": "```json\n{\n \"error\": \"No valid content found. The input appears to be a list of generated line markers without any actual text or data to process.\",\n \"status\": \"empty_input\"\n}\n```",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/server.js",
"findings": [
{
"id": "",
"title": "CORS misconfiguration allows cross-origin attacks",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.",
"suggested_fix": "Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded AWS access key in source code",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.",
"suggested_fix": "Use environment variables or a secure secrets manager to store AWS credentials.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on user creation endpoint",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.",
"suggested_fix": "Implement authentication middleware to verify user identity before allowing POST requests.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on admin deletion endpoint",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.",
"suggested_fix": "Implement strict authentication and authorization checks for all admin endpoints.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"raw_content": "{\n \"findings\": [\n {\n \"title\": \"CORS misconfiguration allows cross-origin attacks\",\n \"severity\": \"high\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"2\",\n \"evidence\": \"res.setHeader(\\\"Access-Control-Allow-Origin\\\", \\\"*\\\");\",\n \"reason\": \"Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.\",\n \"suggested_fix\": \"Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded AWS access key in source code\",\n \"severity\": \"critical\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"5\",\n \"evidence\": \"const AWS_KEY = \\\"AKIAIOSFODNN7EXAMPLE\\\";\",\n \"reason\": \"Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.\",\n \"suggested_fix\": \"Use environment variables or a secure secrets manager to store AWS credentials.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Missing authentication on user creation endpoint\",\n \"severity\": \"high\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"7\",\n \"evidence\": \"app.post(\\\"/api/users\\\", function(req, res) { /* no auth */ });\",\n \"reason\": \"The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.\",\n \"suggested_fix\": \"Implement authentication middleware to verify user identity before allowing POST requests.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Missing authentication on admin deletion endpoint\",\n \"severity\": \"critical\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"8\",\n \"evidence\": \"app.delete(\\\"/api/admin\\\", function(req, res) { /* no auth */ });\",\n \"reason\": \"The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.\",\n \"suggested_fix\": \"Implement strict authentication and authorization checks for all admin endpoints.\",\n \"confidence\": 1.0\n }\n ]\n}",
"parsed": true,
"retried": false
}
]
@@ -0,0 +1,82 @@
{
"run_id": "20260430T060713-f513f6dc",
"repo_path": "tests/fixtures/insecure-repo",
"started_at": "2026-04-30T06:07:13.917781613Z",
"finished_at": "2026-04-30T06:07:13.953011207Z",
"phases": [
{
"name": "repo_intake",
"status": "ok",
"output_hash": "540f222456204a27",
"output_files": [
"repo-intake.json"
]
},
{
"name": "static_scan",
"status": "ok",
"output_hash": "a7aeccbda6841c1e",
"output_files": [
"static-findings.json"
]
},
{
"name": "llm_review",
"status": "skipped",
"errors": [
"LLM review not requested (pass --enable-llm to opt in)"
]
},
{
"name": "validation",
"status": "skipped",
"errors": [
"Phase D not implemented in MVP — depends on Phase C"
]
},
{
"name": "report_generation",
"status": "ok",
"output_files": [
"scrum-test.md",
"risk-register.md",
"claim-coverage-table.md",
"sprint-backlog.md",
"acceptance-gates.md"
]
},
{
"name": "memory_update",
"status": "skipped",
"errors": [
"Phase E not implemented in MVP"
]
}
],
"summary": {
"total": 16,
"confirmed": 2,
"suspected": 14,
"rejected": 0,
"critical": 3,
"high": 4,
"medium": 6,
"low": 3,
"by_source": {
"static": 16
},
"by_check": {
"static.broad_cors": 1,
"static.env_file_committed": 1,
"static.exposed_mutation_endpoint": 2,
"static.hardcoded_local_ip": 1,
"static.hardcoded_paths": 1,
"static.large_files": 1,
"static.missing_tests": 1,
"static.raw_sql_interpolation": 1,
"static.secret_patterns": 3,
"static.shell_execution": 1,
"static.todo_comments": 3
}
}
}
@@ -0,0 +1,37 @@
{
"repo_path": "/home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo",
"current_branch": "main",
"latest_commit": "70d68757f78ea722bf24585c73120c09d82c4fea",
"git_status": "M ../../../internal/cli/cli.go\n M ../../../internal/cli/repo.go\n M ../../../internal/pipeline/pipeline.go\n?? ../../../internal/llm/ollama.go\n?? ../../../internal/llm/review.go\n?? ../clean-repo/reports/\n?? reports/",
"has_git": true,
"file_count": 4,
"language_breakdown": {
"Go": 2,
"JavaScript": 1
},
"largest_files": [
{
"path": "src/huge.go",
"size": 19705,
"lines": 901
},
{
"path": "src/handler.go",
"size": 462,
"lines": 23
},
{
"path": "src/server.js",
"size": 286,
"lines": 8
},
{
"path": ".env",
"size": 59,
"lines": 2
}
],
"dependency_manifests": null,
"test_manifests": null,
"generated_at": "2026-04-30T06:07:13.931830669Z"
}
@@ -0,0 +1,22 @@
# Risk Register
Findings ranked by severity. `Suspected` rows haven't been validated yet (Phase D).
| ID | Severity | Status | File | Line | Title |
|---|---|---|---|---|---|
| `9bc97c579efc` | critical | suspected | `src/handler.go` | 23 | Possible secret committed to source |
| `9bc97c579efc` | critical | suspected | `src/handler.go` | 23 | Possible secret committed to source |
| `d3c2c5606e1d` | critical | suspected | `src/server.js` | 5 | Possible secret committed to source |
| `750676119e4a` | high | confirmed | `.env` | — | Environment file in source tree |
| `3a198539c923` | high | suspected | `src/handler.go` | 14 | Raw SQL interpolation |
| `5bf85ae888a0` | high | suspected | `src/handler.go` | 19 | Shell command execution |
| `ef8bb39704d3` | high | suspected | `src/server.js` | 2 | Wildcard CORS |
| `4d59806aeb57` | medium | confirmed | `.` | — | No tests found |
| `eb3c41b3a186` | medium | suspected | `src/handler.go` | 10 | Hardcoded absolute path |
| `bb70e8e262d6` | medium | suspected | `src/handler.go` | 11 | Hardcoded private-network IP |
| `512b795dc551` | medium | suspected | `src/huge.go` | 1-901 | Large file |
| `7ed1cab08825` | medium | suspected | `src/server.js` | 7 | Mutation route in file with no visible auth |
| `2b765c240c96` | medium | suspected | `src/server.js` | 8 | Mutation route in file with no visible auth |
| `f99cd5bb5f2c` | low | suspected | `src/handler.go` | 22 | TODO/FIXME comment |
| `f3e510b70ec9` | low | suspected | `src/handler.go` | 9 | TODO/FIXME comment |
| `4a631055edd1` | low | suspected | `src/server.js` | 1 | TODO/FIXME comment |
@@ -0,0 +1,96 @@
# Scrum Test — insecure-repo
**Generated:** 2026-04-30T06:07:13.931830669Z
**Branch:** main · **Commit:** 70d68757f78ea722bf24585c73120c09d82c4fea
## Verdict
**blocked** — critical-severity finding present. See Confirmed Risks; rotate any leaked credentials, then re-run.
## Evidence
- repo path: `/home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
- file count: 4
- languages: Go (2), JavaScript (1)
- dependency manifests: 0 ()
- test files/dirs: 0
- LLM review: **skipped** (Phase C not implemented OR provider unavailable; see model-doctor.json)
## Confirmed Risks
| Severity | File:Line | Title | Evidence |
|---|---|---|---|
| high | `.env` | Environment file in source tree | `filename=.env` |
| medium | `.` | No tests found | `No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)` |
## Suspected Risks
Each entry is a static-scan regex hit awaiting validation (Phase D / LLM cross-check).
| Severity | File:Line | Title | Evidence |
|---|---|---|---|
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
| critical | `src/server.js:5` | Possible secret committed to source | `const AWS_KEY = "AKIAIOSFODNN7EXAMPLE";` |
| high | `src/handler.go:14` | Raw SQL interpolation | `q := fmt.Sprintf("SELECT * FROM users WHERE name = '%s'", name)` |
| high | `src/handler.go:19` | Shell command execution | `exec.Command("bash", "-c", cmd).Run()` |
| high | `src/server.js:2` | Wildcard CORS | `res.setHeader("Access-Control-Allow-Origin", "*");` |
| medium | `src/handler.go:10` | Hardcoded absolute path | `const HARDCODED_PATH = "/home/profit/secrets/key.pem"` |
| medium | `src/handler.go:11` | Hardcoded private-network IP | `const SERVER_IP = "192.168.1.176"` |
| medium | `src/huge.go:1-901` | Large file | `901 lines (limit: 800)` |
| medium | `src/server.js:7` | Mutation route in file with no visible auth | `app.post("/api/users", function(req, res) { /* no auth */ });` |
| medium | `src/server.js:8` | Mutation route in file with no visible auth | `app.delete("/api/admin", function(req, res) { /* no auth */ });` |
| low | `src/handler.go:22` | TODO/FIXME comment | `// FIXME: hardcoded creds` |
| low | `src/handler.go:9` | TODO/FIXME comment | `// TODO: rotate this and move to env` |
| low | `src/server.js:1` | TODO/FIXME comment | `// HACK: open CORS for now` |
## Blocked Checks
- LLM review (Phase 2 in REVIEW_PIPELINE.md). Reason: provider unavailable or stub. Next command: `review-harness model doctor`
## Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
- Triage the 16 findings emitted by this run; mark each as accepted / blocking / dismiss-with-reason.
**Sprint 1 — Trust Boundary Gate**
- Resolve every `critical` and `high` finding before non-loopback deploy.
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
## Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
5. **Critical findings block production deploy:** at least one critical finding is currently present; resolve before deploy.
## Next Commands
1. Open the risk register: `cat reports/latest/risk-register.md`
2. Triage every `critical` finding; rotate any leaked credentials immediately.
- Probe the model provider: `review-harness model doctor`
- Re-run after fixes: `review-harness repo /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
- Generate the full Scrum bundle: `review-harness scrum /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
@@ -0,0 +1,28 @@
# Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
- Triage the 16 findings emitted by this run; mark each as accepted / blocking / dismiss-with-reason.
**Sprint 1 — Trust Boundary Gate**
- Resolve every `critical` and `high` finding before non-loopback deploy.
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
@@ -0,0 +1,242 @@
{
"generated_at": "2026-04-30T06:07:13.951970576Z",
"findings": [
{
"id": "750676119e4a",
"title": "Environment file in source tree",
"severity": "high",
"status": "confirmed",
"file": ".env",
"evidence": "filename=.env",
"reason": ".env files commonly hold real secrets and should not be tracked. If this is a sample, rename to .env.example with placeholder values.",
"suggested_fix": "Rename to .env.example with placeholders; add .env to .gitignore; rotate any committed secrets.",
"source": "static",
"confidence": 0.9,
"check_id": "static.env_file_committed"
},
{
"id": "eb3c41b3a186",
"title": "Hardcoded absolute path",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Absolute path encoded in source — couples the binary to one filesystem layout. Move to config or env var.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_paths"
},
{
"id": "5bf85ae888a0",
"title": "Shell command execution",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Direct subprocess/shell invocation. Confirm inputs are sanitized; prefer typed APIs over string-built commands.",
"source": "static",
"confidence": 0.6,
"check_id": "static.shell_execution"
},
{
"id": "3a198539c923",
"title": "Raw SQL interpolation",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "14",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)",
"reason": "SQL assembled via string formatting/concatenation rather than parameterized query. Verify inputs aren't user-controlled.",
"suggested_fix": "Use parameterized queries / prepared statements; pass values via driver placeholders, not string interpolation.",
"source": "static",
"confidence": 0.6,
"check_id": "static.raw_sql_interpolation"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "OpenAI/OpenRouter-shaped key detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoded credential pattern detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "f3e510b70ec9",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "9",
"evidence": "// TODO: rotate this and move to env",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "f99cd5bb5f2c",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "22",
"evidence": "// FIXME: hardcoded creds",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "bb70e8e262d6",
"title": "Hardcoded private-network IP",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "RFC 1918 / link-local IP literal in source. Move to config so the binary isn't tied to one network.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_local_ip"
},
{
"id": "512b795dc551",
"title": "Large file",
"severity": "medium",
"status": "suspected",
"file": "src/huge.go",
"line_hint": "1-901",
"evidence": "901 lines (limit: 800)",
"reason": "File exceeds the configured size threshold. Long files are a refactor target — split by responsibility.",
"source": "static",
"confidence": 1,
"check_id": "static.large_files"
},
{
"id": "ef8bb39704d3",
"title": "Wildcard CORS",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Access-Control-Allow-Origin: * permits cross-origin reads from any domain. Narrow to an explicit allowlist unless this endpoint is intentionally public.",
"source": "static",
"confidence": 0.85,
"check_id": "static.broad_cors"
},
{
"id": "d3c2c5606e1d",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "AWS access key ID detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "4a631055edd1",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/server.js",
"line_hint": "1",
"evidence": "// HACK: open CORS for now",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "7ed1cab08825",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "2b765c240c96",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "4d59806aeb57",
"title": "No tests found",
"severity": "medium",
"status": "confirmed",
"file": ".",
"evidence": "No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)",
"reason": "Repository has source code but no test surface. Refactoring or extending without test cover is high-risk.",
"source": "static",
"confidence": 0.95,
"check_id": "static.missing_tests"
}
],
"summary": {
"total": 16,
"confirmed": 2,
"suspected": 14,
"rejected": 0,
"critical": 3,
"high": 4,
"medium": 6,
"low": 3,
"by_source": {
"static": 16
},
"by_check": {
"static.broad_cors": 1,
"static.env_file_committed": 1,
"static.exposed_mutation_endpoint": 2,
"static.hardcoded_local_ip": 1,
"static.hardcoded_paths": 1,
"static.large_files": 1,
"static.missing_tests": 1,
"static.raw_sql_interpolation": 1,
"static.secret_patterns": 3,
"static.shell_execution": 1,
"static.todo_comments": 3
}
}
}