profit/local-review-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase D — validator cross-checks LLM findings + 2 close-out fixes

Browse Source 
Implements PROMPT.md / docs/REVIEW_PIPELINE.md Phase 3:
"AI may suggest. Code validates."

internal/validators/validate.go — 3 hard checks per the
"Reject A Finding If" list:
- file does not exist (with path-traversal guard against the LLM
  hallucinating ../../../etc/passwd)
- cited evidence does not appear in the file (verbatim or
  trim-line-by-line — models often re-indent quotes when quoting code)
- line hint exceeds file line count

3 soft checks documented as open (claim semantics, suggested-fix
relevance, invented tests/commands — all need another LLM pass).

internal/validators/validate_test.go — 9 tests including:
- TestValidate_RejectsNonexistentFile (gate D1)
- TestValidate_RejectsEvidenceNotInFile
- TestValidate_RejectsLineHintBeyondFile
- TestValidate_AcceptsRealFinding
- TestValidate_AcceptsEvidenceWithDifferentLeadingWhitespace
- TestValidate_RejectsEmptyEvidence
- TestValidate_PassesThroughStaticFindings
- TestValidate_RejectsPathEscapingRepo (path-traversal protection)
- TestValidate_AcceptsRelativeRepoPath (the regression — see below)

Pipeline phase 3 wired between LLM review (Phase C) and report gen
(Phase 4). validated-findings.json contains the confirmed set;
rejected-findings.json contains rejects with per-finding reason +
detail. Receipt phase entry honest about output files + status.

=== Bug J caught ===

First Phase D run rejected EVERY real LLM finding as file_not_found
because the path-traversal check compared a relative joined path
(`tests/fixtures/insecure-repo/src/handler.go`) against an absolute
repoAbs (`/home/profit/share/.../insecure-repo`), so HasPrefix
always returned false. Both sides now resolved via filepath.Abs
before comparison. Regression test
TestValidate_AcceptsRelativeRepoPath locks this in — runs the
validator against a relative repo path AND a relative chdir, the
exact shape that hit the bug.

J's framing was honest: "I don't know what the problem is, but you
know what we're trying to accomplish." The fix-it-yourself signal
let me trace through the rejection details + see the smoking gun
in the detail string ("escapes repo root"). Without that prompt the
9 false rejections might have looked like real LLM bugs.

=== 2 close-out fixes ===

1. .gitignore: changed `/reports/latest/` → `**/reports/latest/`
   (and same for `run-*`). Phase C committed 22 generated files
   from `tests/fixtures/*/reports/latest/` because the original
   pattern was anchored at the harness root only. Existing tracked
   files removed via git rm --cached; new pattern keeps fixture
   reports out of version control going forward.

2. pipeline.cleanOutputDir: pipeline now deletes the bounded list
   of known per-run files at the start of each run. Before this,
   a prior run's rejected-findings.json could linger when the
   current run had no rejections — confused J during the bug hunt
   above. cleanOutputDir is bounded (deletes only files we emit)
   so operator-owned adjacent files stay.

Verified end-to-end: insecure-repo + --enable-llm → 25 confirmed
findings (16 static + 9 LLM), 0 rejected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude (review-harness setup)
2026-04-30 01:24:02 -05:00
co-authored by Claude Opus 4.7
parent e346b54e0f
commit 4dc53c5798
24 changed files with 546 additions and 1135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/fixtures/clean-repo/reports/latest/acceptance-gates.md
Vendored
-8
View File
@@ -1,8 +0,0 @@
# Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
tests/fixtures/clean-repo/reports/latest/claim-coverage-table.md
Vendored
-8
View File
@@ -1,8 +0,0 @@
# Claim Coverage Table
Each row is a finding paired with whether existing tests cover the affected area.
Phase B emits this shape; LLM-side claim generation lands in Phase C.
| Claim | Code Location | Existing Test | Missing Test | Risk |
|---|---|---|---|---|
| _no claims yet_ | — | — | — | — |
tests/fixtures/clean-repo/reports/latest/llm-findings.normalized.json
Vendored
-16
View File
@@ -1,16 +0,0 @@
{
"generated_at": "2026-04-30T06:06:56.669606679Z",
"findings": [],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
tests/fixtures/clean-repo/reports/latest/llm-findings.raw.json
Vendored
-30
View File
@@ -1,30 +0,0 @@
[
{
"chunk_id": "README.md",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "package.json",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/calc.ts",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
},
{
"chunk_id": "tests/calc.test.ts",
"findings": [],
"raw_content": "{\"findings\": []}",
"parsed": true,
"retried": false
}
]
tests/fixtures/clean-repo/reports/latest/receipts.json
Vendored
-70
View File
@@ -1,70 +0,0 @@
{
"run_id": "20260430T060653-57461e2c",
"repo_path": "tests/fixtures/clean-repo",
"started_at": "2026-04-30T06:06:53.880883402Z",
"finished_at": "2026-04-30T06:06:56.669797363Z",
"phases": [
{
"name": "repo_intake",
"status": "ok",
"output_hash": "db312c5ce39315cd",
"output_files": [
"repo-intake.json"
]
},
{
"name": "static_scan",
"status": "ok",
"output_hash": "837b6a5d9dc11126",
"output_files": [
"static-findings.json"
]
},
{
"name": "llm_review",
"status": "ok",
"output_hash": "3939252dabe358b1",
"output_files": [
"llm-findings.raw.json",
"llm-findings.normalized.json"
]
},
{
"name": "validation",
"status": "skipped",
"errors": [
"Phase D not implemented in MVP — depends on Phase C"
]
},
{
"name": "report_generation",
"status": "ok",
"output_files": [
"scrum-test.md",
"risk-register.md",
"claim-coverage-table.md",
"sprint-backlog.md",
"acceptance-gates.md"
]
},
{
"name": "memory_update",
"status": "skipped",
"errors": [
"Phase E not implemented in MVP"
]
}
],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
tests/fixtures/clean-repo/reports/latest/repo-intake.json
Vendored
-42
View File
@@ -1,42 +0,0 @@
{
"repo_path": "/home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo",
"current_branch": "main",
"latest_commit": "70d68757f78ea722bf24585c73120c09d82c4fea",
"git_status": "M ../../../internal/cli/cli.go\n M ../../../internal/cli/repo.go\n M ../../../internal/pipeline/pipeline.go\n?? ../../../internal/llm/ollama.go\n?? ../../../internal/llm/review.go\n?? ../insecure-repo/reports/",
"has_git": true,
"file_count": 4,
"language_breakdown": {
"JSON": 1,
"Markdown": 1,
"TypeScript": 2
},
"largest_files": [
{
"path": "src/calc.ts",
"size": 206,
"lines": 7
},
{
"path": "tests/calc.test.ts",
"size": 198,
"lines": 4
},
{
"path": "README.md",
"size": 80,
"lines": 2
},
{
"path": "package.json",
"size": 43,
"lines": 1
}
],
"dependency_manifests": [
"package.json"
],
"test_manifests": [
"tests/calc.test.ts"
],
"generated_at": "2026-04-30T06:06:53.895008668Z"
}
tests/fixtures/clean-repo/reports/latest/risk-register.md
Vendored
-5
View File
@@ -1,5 +0,0 @@
# Risk Register
Findings ranked by severity. `Suspected` rows haven't been validated yet (Phase D).
_No findings._
tests/fixtures/clean-repo/reports/latest/scrum-test.md
Vendored
-69
View File
@@ -1,69 +0,0 @@
# Scrum Test — clean-repo
**Generated:** 2026-04-30T06:06:53.895008668Z
**Branch:** main · **Commit:** 70d68757f78ea722bf24585c73120c09d82c4fea
## Verdict
**production-ready** — static scan + LLM review found no issues. Re-validate after every wave.
## Evidence
- repo path: `/home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
- file count: 4
- languages: TypeScript (2), Markdown (1), JSON (1)
- dependency manifests: 1 (package.json)
- test files/dirs: 1
## Confirmed Risks
_No confirmed risks at static-scan level. (LLM review may surface more.)_
## Suspected Risks
_None._
## Blocked Checks
_None._
## Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
**Sprint 1 — Trust Boundary Gate**
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
## Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
## Next Commands
- Re-run after fixes: `review-harness repo /home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
- Generate the full Scrum bundle: `review-harness scrum /home/profit/share/local-review-harness-full-md/tests/fixtures/clean-repo`
tests/fixtures/clean-repo/reports/latest/sprint-backlog.md
Vendored
-26
View File
@@ -1,26 +0,0 @@
# Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
**Sprint 1 — Trust Boundary Gate**
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
tests/fixtures/clean-repo/reports/latest/static-findings.json
Vendored
-16
View File
@@ -1,16 +0,0 @@
{
"generated_at": "2026-04-30T06:06:53.896533109Z",
"findings": [],
"summary": {
"total": 0,
"confirmed": 0,
"suspected": 0,
"rejected": 0,
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"by_source": {},
"by_check": {}
}
}
tests/fixtures/insecure-repo/reports/latest/acceptance-gates.md
Vendored
-9
View File
@@ -1,9 +0,0 @@
# Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
5. **Critical findings block production deploy:** at least one critical finding is currently present; resolve before deploy.
tests/fixtures/insecure-repo/reports/latest/claim-coverage-table.md
Vendored
-23
View File
@@ -1,23 +0,0 @@
# Claim Coverage Table
Each row is a finding paired with whether existing tests cover the affected area.
Phase B emits this shape; LLM-side claim generation lands in Phase C.
| Claim | Code Location | Existing Test | Missing Test | Risk |
|---|---|---|---|---|
| Environment file in source tree | `.env:?` | _unknown_ | _likely_ | high |
| Hardcoded absolute path | `src/handler.go:10` | _unknown_ | _likely_ | medium |
| Shell command execution | `src/handler.go:19` | _unknown_ | _likely_ | high |
| Raw SQL interpolation | `src/handler.go:14` | _unknown_ | _likely_ | high |
| Possible secret committed to source | `src/handler.go:23` | _unknown_ | _likely_ | critical |
| Possible secret committed to source | `src/handler.go:23` | _unknown_ | _likely_ | critical |
| TODO/FIXME comment | `src/handler.go:9` | _unknown_ | _likely_ | low |
| TODO/FIXME comment | `src/handler.go:22` | _unknown_ | _likely_ | low |
| Hardcoded private-network IP | `src/handler.go:11` | _unknown_ | _likely_ | medium |
| Large file | `src/huge.go:1-901` | _unknown_ | _likely_ | medium |
| Wildcard CORS | `src/server.js:2` | _unknown_ | _likely_ | high |
| Possible secret committed to source | `src/server.js:5` | _unknown_ | _likely_ | critical |
| TODO/FIXME comment | `src/server.js:1` | _unknown_ | _likely_ | low |
| Mutation route in file with no visible auth | `src/server.js:7` | _unknown_ | _likely_ | medium |
| Mutation route in file with no visible auth | `src/server.js:8` | _unknown_ | _likely_ | medium |
| No tests found | `.:?` | _unknown_ | _likely_ | medium |
tests/fixtures/insecure-repo/reports/latest/llm-findings.normalized.json
Vendored
-147
View File
@@ -1,147 +0,0 @@
{
"generated_at": "2026-04-30T06:06:33.240219171Z",
"findings": [
{
"id": "",
"title": "Hardcoded file path for secrets",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Hardcoding a file path for a private key exposes secrets and prevents proper secret management.",
"suggested_fix": "Move the path to an environment variable or a configuration file outside the source code.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded server IP address",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "Hardcoding an IP address reduces portability and may leak internal network topology.",
"suggested_fix": "Read the server IP from an environment variable.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "SQL Injection vulnerability",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "15-16",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)\ndb.Query(q)",
"reason": "Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.",
"suggested_fix": "Use parameterized queries with placeholders instead of string formatting.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Unsafe shell command execution",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19-20",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Executing arbitrary shell commands without validation allows for remote code execution.",
"suggested_fix": "Validate and sanitize the input command strictly, or avoid using shell execution entirely.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded API key",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.",
"suggested_fix": "Store the API key in a secure environment variable or secrets manager.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "CORS misconfiguration allows cross-origin attacks",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.",
"suggested_fix": "Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded AWS access key in source code",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.",
"suggested_fix": "Use environment variables or a secure secrets manager to store AWS credentials.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on user creation endpoint",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.",
"suggested_fix": "Implement authentication middleware to verify user identity before allowing POST requests.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on admin deletion endpoint",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.",
"suggested_fix": "Implement strict authentication and authorization checks for all admin endpoints.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"summary": {
"total": 9,
"confirmed": 0,
"suspected": 9,
"rejected": 0,
"critical": 5,
"high": 3,
"medium": 1,
"low": 0,
"by_source": {
"llm": 9
},
"by_check": {
"llm.review": 9
}
}
}
tests/fixtures/insecure-repo/reports/latest/llm-findings.raw.json
Vendored
-151
View File
@@ -1,151 +0,0 @@
[
{
"chunk_id": "src/handler.go",
"findings": [
{
"id": "",
"title": "Hardcoded file path for secrets",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Hardcoding a file path for a private key exposes secrets and prevents proper secret management.",
"suggested_fix": "Move the path to an environment variable or a configuration file outside the source code.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded server IP address",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "Hardcoding an IP address reduces portability and may leak internal network topology.",
"suggested_fix": "Read the server IP from an environment variable.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "SQL Injection vulnerability",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "15-16",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)\ndb.Query(q)",
"reason": "Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.",
"suggested_fix": "Use parameterized queries with placeholders instead of string formatting.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Unsafe shell command execution",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19-20",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Executing arbitrary shell commands without validation allows for remote code execution.",
"suggested_fix": "Validate and sanitize the input command strictly, or avoid using shell execution entirely.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded API key",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.",
"suggested_fix": "Store the API key in a secure environment variable or secrets manager.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"raw_content": "{\n \"findings\": [\n {\n \"title\": \"Hardcoded file path for secrets\",\n \"severity\": \"high\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"10\",\n \"evidence\": \"const HARDCODED_PATH = \\\"/home/profit/secrets/key.pem\\\"\",\n \"reason\": \"Hardcoding a file path for a private key exposes secrets and prevents proper secret management.\",\n \"suggested_fix\": \"Move the path to an environment variable or a configuration file outside the source code.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded server IP address\",\n \"severity\": \"medium\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"11\",\n \"evidence\": \"const SERVER_IP = \\\"192.168.1.176\\\"\",\n \"reason\": \"Hardcoding an IP address reduces portability and may leak internal network topology.\",\n \"suggested_fix\": \"Read the server IP from an environment variable.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"SQL Injection vulnerability\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"15-16\",\n \"evidence\": \"q := fmt.Sprintf(\\\"SELECT * FROM users WHERE name = '%s'\\\", name)\\ndb.Query(q)\",\n \"reason\": \"Using string formatting to construct SQL queries directly exposes the application to SQL injection attacks.\",\n \"suggested_fix\": \"Use parameterized queries with placeholders instead of string formatting.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Unsafe shell command execution\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"19-20\",\n \"evidence\": \"exec.Command(\\\"bash\\\", \\\"-c\\\", cmd).Run()\",\n \"reason\": \"Executing arbitrary shell commands without validation allows for remote code execution.\",\n \"suggested_fix\": \"Validate and sanitize the input command strictly, or avoid using shell execution entirely.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded API key\",\n \"severity\": \"critical\",\n \"file\": \"src/handler.go\",\n \"line_hint\": \"23\",\n \"evidence\": \"const API_KEY = \\\"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\\\"\",\n \"reason\": \"Hardcoding an API key in source code exposes sensitive credentials to anyone with access to the repository.\",\n \"suggested_fix\": \"Store the API key in a secure environment variable or secrets manager.\",\n \"confidence\": 1.0\n }\n ]\n}",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/huge.go",
"findings": [],
"raw_content": "```json\n{\n \"error\": \"No valid content found. The input appears to be a list of generated line markers without any actual text or data to process.\",\n \"status\": \"empty_input\"\n}\n```",
"parsed": true,
"retried": false
},
{
"chunk_id": "src/server.js",
"findings": [
{
"id": "",
"title": "CORS misconfiguration allows cross-origin attacks",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.",
"suggested_fix": "Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Hardcoded AWS access key in source code",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.",
"suggested_fix": "Use environment variables or a secure secrets manager to store AWS credentials.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on user creation endpoint",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.",
"suggested_fix": "Implement authentication middleware to verify user identity before allowing POST requests.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
},
{
"id": "",
"title": "Missing authentication on admin deletion endpoint",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.",
"suggested_fix": "Implement strict authentication and authorization checks for all admin endpoints.",
"source": "llm",
"confidence": 1,
"check_id": "llm.review"
}
],
"raw_content": "{\n \"findings\": [\n {\n \"title\": \"CORS misconfiguration allows cross-origin attacks\",\n \"severity\": \"high\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"2\",\n \"evidence\": \"res.setHeader(\\\"Access-Control-Allow-Origin\\\", \\\"*\\\");\",\n \"reason\": \"Allowing all origins (*) exposes the API to cross-site request forgery and data theft from any website.\",\n \"suggested_fix\": \"Restrict Access-Control-Allow-Origin to specific trusted domains or use credentials with a specific origin.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Hardcoded AWS access key in source code\",\n \"severity\": \"critical\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"5\",\n \"evidence\": \"const AWS_KEY = \\\"AKIAIOSFODNN7EXAMPLE\\\";\",\n \"reason\": \"Hardcoded credentials in source code pose a severe security risk as they can be easily leaked and misused.\",\n \"suggested_fix\": \"Use environment variables or a secure secrets manager to store AWS credentials.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Missing authentication on user creation endpoint\",\n \"severity\": \"high\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"7\",\n \"evidence\": \"app.post(\\\"/api/users\\\", function(req, res) { /* no auth */ });\",\n \"reason\": \"The /api/users endpoint lacks authentication, allowing anyone to create or modify user accounts.\",\n \"suggested_fix\": \"Implement authentication middleware to verify user identity before allowing POST requests.\",\n \"confidence\": 1.0\n },\n {\n \"title\": \"Missing authentication on admin deletion endpoint\",\n \"severity\": \"critical\",\n \"file\": \"src/server.js\",\n \"line_hint\": \"8\",\n \"evidence\": \"app.delete(\\\"/api/admin\\\", function(req, res) { /* no auth */ });\",\n \"reason\": \"The /api/admin endpoint lacks authentication, allowing unauthenticated users to delete administrative resources.\",\n \"suggested_fix\": \"Implement strict authentication and authorization checks for all admin endpoints.\",\n \"confidence\": 1.0\n }\n ]\n}",
"parsed": true,
"retried": false
}
]
tests/fixtures/insecure-repo/reports/latest/receipts.json
Vendored
-82
View File
@@ -1,82 +0,0 @@
{
"run_id": "20260430T060713-f513f6dc",
"repo_path": "tests/fixtures/insecure-repo",
"started_at": "2026-04-30T06:07:13.917781613Z",
"finished_at": "2026-04-30T06:07:13.953011207Z",
"phases": [
{
"name": "repo_intake",
"status": "ok",
"output_hash": "540f222456204a27",
"output_files": [
"repo-intake.json"
]
},
{
"name": "static_scan",
"status": "ok",
"output_hash": "a7aeccbda6841c1e",
"output_files": [
"static-findings.json"
]
},
{
"name": "llm_review",
"status": "skipped",
"errors": [
"LLM review not requested (pass --enable-llm to opt in)"
]
},
{
"name": "validation",
"status": "skipped",
"errors": [
"Phase D not implemented in MVP — depends on Phase C"
]
},
{
"name": "report_generation",
"status": "ok",
"output_files": [
"scrum-test.md",
"risk-register.md",
"claim-coverage-table.md",
"sprint-backlog.md",
"acceptance-gates.md"
]
},
{
"name": "memory_update",
"status": "skipped",
"errors": [
"Phase E not implemented in MVP"
]
}
],
"summary": {
"total": 16,
"confirmed": 2,
"suspected": 14,
"rejected": 0,
"critical": 3,
"high": 4,
"medium": 6,
"low": 3,
"by_source": {
"static": 16
},
"by_check": {
"static.broad_cors": 1,
"static.env_file_committed": 1,
"static.exposed_mutation_endpoint": 2,
"static.hardcoded_local_ip": 1,
"static.hardcoded_paths": 1,
"static.large_files": 1,
"static.missing_tests": 1,
"static.raw_sql_interpolation": 1,
"static.secret_patterns": 3,
"static.shell_execution": 1,
"static.todo_comments": 3
}
}
}
tests/fixtures/insecure-repo/reports/latest/repo-intake.json
Vendored
-37
View File
@@ -1,37 +0,0 @@
{
"repo_path": "/home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo",
"current_branch": "main",
"latest_commit": "70d68757f78ea722bf24585c73120c09d82c4fea",
"git_status": "M ../../../internal/cli/cli.go\n M ../../../internal/cli/repo.go\n M ../../../internal/pipeline/pipeline.go\n?? ../../../internal/llm/ollama.go\n?? ../../../internal/llm/review.go\n?? ../clean-repo/reports/\n?? reports/",
"has_git": true,
"file_count": 4,
"language_breakdown": {
"Go": 2,
"JavaScript": 1
},
"largest_files": [
{
"path": "src/huge.go",
"size": 19705,
"lines": 901
},
{
"path": "src/handler.go",
"size": 462,
"lines": 23
},
{
"path": "src/server.js",
"size": 286,
"lines": 8
},
{
"path": ".env",
"size": 59,
"lines": 2
}
],
"dependency_manifests": null,
"test_manifests": null,
"generated_at": "2026-04-30T06:07:13.931830669Z"
}
tests/fixtures/insecure-repo/reports/latest/risk-register.md
Vendored
-22
View File
@@ -1,22 +0,0 @@
# Risk Register
Findings ranked by severity. `Suspected` rows haven't been validated yet (Phase D).
| ID | Severity | Status | File | Line | Title |
|---|---|---|---|---|---|
| `9bc97c579efc` | critical | suspected | `src/handler.go` | 23 | Possible secret committed to source |
| `9bc97c579efc` | critical | suspected | `src/handler.go` | 23 | Possible secret committed to source |
| `d3c2c5606e1d` | critical | suspected | `src/server.js` | 5 | Possible secret committed to source |
| `750676119e4a` | high | confirmed | `.env` | — | Environment file in source tree |
| `3a198539c923` | high | suspected | `src/handler.go` | 14 | Raw SQL interpolation |
| `5bf85ae888a0` | high | suspected | `src/handler.go` | 19 | Shell command execution |
| `ef8bb39704d3` | high | suspected | `src/server.js` | 2 | Wildcard CORS |
| `4d59806aeb57` | medium | confirmed | `.` | — | No tests found |
| `eb3c41b3a186` | medium | suspected | `src/handler.go` | 10 | Hardcoded absolute path |
| `bb70e8e262d6` | medium | suspected | `src/handler.go` | 11 | Hardcoded private-network IP |
| `512b795dc551` | medium | suspected | `src/huge.go` | 1-901 | Large file |
| `7ed1cab08825` | medium | suspected | `src/server.js` | 7 | Mutation route in file with no visible auth |
| `2b765c240c96` | medium | suspected | `src/server.js` | 8 | Mutation route in file with no visible auth |
| `f99cd5bb5f2c` | low | suspected | `src/handler.go` | 22 | TODO/FIXME comment |
| `f3e510b70ec9` | low | suspected | `src/handler.go` | 9 | TODO/FIXME comment |
| `4a631055edd1` | low | suspected | `src/server.js` | 1 | TODO/FIXME comment |
tests/fixtures/insecure-repo/reports/latest/scrum-test.md
Vendored
-96
View File
@@ -1,96 +0,0 @@
# Scrum Test — insecure-repo
**Generated:** 2026-04-30T06:07:13.931830669Z
**Branch:** main · **Commit:** 70d68757f78ea722bf24585c73120c09d82c4fea
## Verdict
**blocked** — critical-severity finding present. See Confirmed Risks; rotate any leaked credentials, then re-run.
## Evidence
- repo path: `/home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
- file count: 4
- languages: Go (2), JavaScript (1)
- dependency manifests: 0 ()
- test files/dirs: 0
- LLM review: **skipped** (Phase C not implemented OR provider unavailable; see model-doctor.json)
## Confirmed Risks
| Severity | File:Line | Title | Evidence |
|---|---|---|---|
| high | `.env` | Environment file in source tree | `filename=.env` |
| medium | `.` | No tests found | `No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)` |
## Suspected Risks
Each entry is a static-scan regex hit awaiting validation (Phase D / LLM cross-check).
| Severity | File:Line | Title | Evidence |
|---|---|---|---|
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
| critical | `src/server.js:5` | Possible secret committed to source | `const AWS_KEY = "AKIAIOSFODNN7EXAMPLE";` |
| high | `src/handler.go:14` | Raw SQL interpolation | `q := fmt.Sprintf("SELECT * FROM users WHERE name = '%s'", name)` |
| high | `src/handler.go:19` | Shell command execution | `exec.Command("bash", "-c", cmd).Run()` |
| high | `src/server.js:2` | Wildcard CORS | `res.setHeader("Access-Control-Allow-Origin", "*");` |
| medium | `src/handler.go:10` | Hardcoded absolute path | `const HARDCODED_PATH = "/home/profit/secrets/key.pem"` |
| medium | `src/handler.go:11` | Hardcoded private-network IP | `const SERVER_IP = "192.168.1.176"` |
| medium | `src/huge.go:1-901` | Large file | `901 lines (limit: 800)` |
| medium | `src/server.js:7` | Mutation route in file with no visible auth | `app.post("/api/users", function(req, res) { /* no auth */ });` |
| medium | `src/server.js:8` | Mutation route in file with no visible auth | `app.delete("/api/admin", function(req, res) { /* no auth */ });` |
| low | `src/handler.go:22` | TODO/FIXME comment | `// FIXME: hardcoded creds` |
| low | `src/handler.go:9` | TODO/FIXME comment | `// TODO: rotate this and move to env` |
| low | `src/server.js:1` | TODO/FIXME comment | `// HACK: open CORS for now` |
## Blocked Checks
- LLM review (Phase 2 in REVIEW_PIPELINE.md). Reason: provider unavailable or stub. Next command: `review-harness model doctor`
## Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
- Triage the 16 findings emitted by this run; mark each as accepted / blocking / dismiss-with-reason.
**Sprint 1 — Trust Boundary Gate**
- Resolve every `critical` and `high` finding before non-loopback deploy.
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
## Acceptance Gates
Each gate must be testable. Format: command + verifiable post-condition.
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
5. **Critical findings block production deploy:** at least one critical finding is currently present; resolve before deploy.
## Next Commands
1. Open the risk register: `cat reports/latest/risk-register.md`
2. Triage every `critical` finding; rotate any leaked credentials immediately.
- Probe the model provider: `review-harness model doctor`
- Re-run after fixes: `review-harness repo /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
- Generate the full Scrum bundle: `review-harness scrum /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
tests/fixtures/insecure-repo/reports/latest/sprint-backlog.md
Vendored
-28
View File
@@ -1,28 +0,0 @@
# Sprint Backlog
**Sprint 0 — Reproducibility Gate**
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
- Add a CI step that fails on `critical` findings.
- Triage the 16 findings emitted by this run; mark each as accepted / blocking / dismiss-with-reason.
**Sprint 1 — Trust Boundary Gate**
- Resolve every `critical` and `high` finding before non-loopback deploy.
- Confirm auth posture for any mutation endpoint flagged as exposed.
- Replace raw SQL interpolation with parameterized queries.
**Sprint 2 — Memory Correctness Gate**
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
**Sprint 3 — Agent Loop Reality Gate**
- (Phase C) Wire local-Ollama LLM review.
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
**Sprint 4 — Deployment Gate**
- Ship the harness as a single static binary (`go build -o review-harness`).
- Document operator runbook (model setup, profile editing, output retention).
tests/fixtures/insecure-repo/reports/latest/static-findings.json
Vendored
-242
View File
@@ -1,242 +0,0 @@
{
"generated_at": "2026-04-30T06:07:13.951970576Z",
"findings": [
{
"id": "750676119e4a",
"title": "Environment file in source tree",
"severity": "high",
"status": "confirmed",
"file": ".env",
"evidence": "filename=.env",
"reason": ".env files commonly hold real secrets and should not be tracked. If this is a sample, rename to .env.example with placeholder values.",
"suggested_fix": "Rename to .env.example with placeholders; add .env to .gitignore; rotate any committed secrets.",
"source": "static",
"confidence": 0.9,
"check_id": "static.env_file_committed"
},
{
"id": "eb3c41b3a186",
"title": "Hardcoded absolute path",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Absolute path encoded in source — couples the binary to one filesystem layout. Move to config or env var.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_paths"
},
{
"id": "5bf85ae888a0",
"title": "Shell command execution",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Direct subprocess/shell invocation. Confirm inputs are sanitized; prefer typed APIs over string-built commands.",
"source": "static",
"confidence": 0.6,
"check_id": "static.shell_execution"
},
{
"id": "3a198539c923",
"title": "Raw SQL interpolation",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "14",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)",
"reason": "SQL assembled via string formatting/concatenation rather than parameterized query. Verify inputs aren't user-controlled.",
"suggested_fix": "Use parameterized queries / prepared statements; pass values via driver placeholders, not string interpolation.",
"source": "static",
"confidence": 0.6,
"check_id": "static.raw_sql_interpolation"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "OpenAI/OpenRouter-shaped key detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoded credential pattern detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "f3e510b70ec9",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "9",
"evidence": "// TODO: rotate this and move to env",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "f99cd5bb5f2c",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "22",
"evidence": "// FIXME: hardcoded creds",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "bb70e8e262d6",
"title": "Hardcoded private-network IP",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "RFC 1918 / link-local IP literal in source. Move to config so the binary isn't tied to one network.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_local_ip"
},
{
"id": "512b795dc551",
"title": "Large file",
"severity": "medium",
"status": "suspected",
"file": "src/huge.go",
"line_hint": "1-901",
"evidence": "901 lines (limit: 800)",
"reason": "File exceeds the configured size threshold. Long files are a refactor target — split by responsibility.",
"source": "static",
"confidence": 1,
"check_id": "static.large_files"
},
{
"id": "ef8bb39704d3",
"title": "Wildcard CORS",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Access-Control-Allow-Origin: * permits cross-origin reads from any domain. Narrow to an explicit allowlist unless this endpoint is intentionally public.",
"source": "static",
"confidence": 0.85,
"check_id": "static.broad_cors"
},
{
"id": "d3c2c5606e1d",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "AWS access key ID detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "4a631055edd1",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/server.js",
"line_hint": "1",
"evidence": "// HACK: open CORS for now",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "7ed1cab08825",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "2b765c240c96",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "4d59806aeb57",
"title": "No tests found",
"severity": "medium",
"status": "confirmed",
"file": ".",
"evidence": "No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)",
"reason": "Repository has source code but no test surface. Refactoring or extending without test cover is high-risk.",
"source": "static",
"confidence": 0.95,
"check_id": "static.missing_tests"
}
],
"summary": {
"total": 16,
"confirmed": 2,
"suspected": 14,
"rejected": 0,
"critical": 3,
"high": 4,
"medium": 6,
"low": 3,
"by_source": {
"static": 16
},
"by_check": {
"static.broad_cors": 1,
"static.env_file_committed": 1,
"static.exposed_mutation_endpoint": 2,
"static.hardcoded_local_ip": 1,
"static.hardcoded_paths": 1,
"static.large_files": 1,
"static.missing_tests": 1,
"static.raw_sql_interpolation": 1,
"static.secret_patterns": 3,
"static.shell_execution": 1,
"static.todo_comments": 3
}
}
}