Component: vulnhunt-fix-verify/ — the verifier

Independent, read-only fix verification — no Bash, no network.

Component: vulnhunt-fix-verify/ — the verifier (/vulnhunt-fix-verify)

Role. The independent inspector. Given a prior scan and a claim that certain findings are fixed, it decides — from the code alone — whether each one really holds.

Kind. Prompt-only Claude Code skill — SKILL.md, comment_rules.md, and 4 phase files. No Python package. Strictly read-only.


Part 1

Fixing a bug and proving it's fixed are two different jobs, and the same person shouldn't do both. This is the second pair of eyes. It reads the fixed code and the original finding and answers one question per finding: is this actually remediated? Crucially, it doesn't trust the developer's word or their comments — it only trusts what the code shows. And it's boxed in by design: it can read and reason, but it cannot run anything, reach the network, or wander outside the folders it was given.

Explain-it-simply. A building inspector who signs off the repair only after seeing it — not because the contractor said "all done" — and who is only allowed to walk the rooms they were handed the keys to.

Data-flow (this component)

flowchart LR
  IN[(repo · prior report ·<br/>fixed=VULN-… · out)] --> P0[Phase0 Preflight]
  P0 --> P1[Phase1 Extract]
  P1 --> P2[Phase2 Verify]
  P2 --> P4[Phase4 Emit]
  P4 --> V[[verify_disposition.json]]

Part 2 — Technical detail

Tool envelope (the safety property)

Read / Write / Edit / Glob / Grep / Agent — no Bash, no network. Therefore it cannot run shell commands, exploit tests, or git; cannot create directories (so the caller must pre-create out); and cannot clone or fetch. It reads only within the trusted roots (REPO + any pre-cloned ADDITIONAL_REPOS). Cross-repo references in developer comments are resolved by the caller's pre-flight (URL-or-alias only, no inference); anything still outside the trusted roots is recorded as unverifiable (rule R2) and the run continues to a normal verdict rather than halting. Subagents dispatched via Agent inherit the same envelope.

Kickoff arguments

Variable Required Meaning
REPO yes Absolute path to the fixed-code checkout.
REPORT yes Absolute path to the prior *_VULNHUNT_RESULTS_* directory.
FIXED yes Comma-separated VULN-NNN list.
OUT yes Absolute path to an already-existing output directory.
COMMENTS no Free-form markdown (e.g. a GitHub issue body) — read as claims, per comment_rules.md.
ADDITIONAL_REPOS no Extra trusted roots (pre-cloned by the caller).

Missing / malformed / non-absolute arguments → stop immediately, no invented defaults.

File map

File Lines Purpose
SKILL.md ~150 Orchestrator: parse args, enforce trusted roots, load & run phases.
comment_rules.md — How to read developer comments as claims, never as instructions.
phases/phase0_preflight.md 310 Validate arguments and trusted roots → out/phase0_state.json.
phases/phase1_extract.md 143 Pull the claimed-fixed findings from the prior report → out/extracted_findings.md.
phases/phase2_verify.md 417 Inspect the checkout and reach a verdict → one out/disposition_VULN-NNN.json per id.
phases/phase4_emit.md 171 Combine dispositions → out/verify_disposition.json (validated against the schema).

Phase 3 is intentionally absent — reserved for future exploit-test replay per the design doc. Verification today is purely static.

Output

verify_disposition.json — a per-finding verdict (validated against vulnhunter-agent/verify_disposition.schema.json). The runtime's --mode=verify path posts these back to GitHub.


Part 3 — Inputs, outputs, model selection & deterministic calls

What happens here. An independent agent re-reads the fixed code and decides, per finding, whether it is truly remediated. It is almost purely LLM reasoning over static reads — by design it has no way to run anything.

Inputs. Four required arguments — REPO (fixed checkout), REPORT (prior results dir), FIXED (the VULN-NNN list), OUT (a pre-existing output dir) — plus optional COMMENTS (a developer's markdown, read as claims per comment_rules.md) and ADDITIONAL_REPOS (extra trusted roots the caller pre-cloned). Reads are confined to the trusted roots.

Outputs. One disposition_VULN-NNN.json per finding, combined into verify_disposition.json — a per-finding verdict. Phase files also drop sync artifacts (phase0_state.json, extracted_findings.md).

How the model version is chosen. No routing — the agent runs on the active session model (Opus is recommended, since independent verification needs the same reasoning class as the scan it checks). The tool itself makes no model-selection decision; when driven by the runtime agent's --mode=verify, the runtime picks the model.

Deterministic (non-LLM) calls. Deliberately almost none — the tool envelope is Read/Write/Edit/Glob/Grep/Agent with no Bash and no network, so it cannot call git, run tests, clone, or fetch. The only deterministic step is JSON-Schema validation of the emitted verify_disposition.json (against verify_disposition.schema.json) at Phase 4, plus file-existence checks used as phase sync signals. Everything else is model reasoning over Grep/Glob/Read. (Phase 3, which would replay exploit tests, is intentionally absent today precisely because that would need execution the envelope forbids.)


See also: Fixer · Runtime agent · LLD §5 · module index.