Deterministic Tools vs. LLM Reasoning

Exactly which parts of each component are model calls vs ordinary code — and why there's no LangChain/LangGraph/DeepAgents here.

Deterministic Tools vs. LLM Reasoning

VulnHunter mixes two kinds of work: LLM reasoning (a model reads code and decides something) and deterministic tool calls (ordinary code that always produces the same output for the same input — parsers, schema validators, git, subprocess calls). This document lays out exactly which is which, per component, and answers a specific question: does VulnHunter use a LangChain/LangGraph-style agent framework anywhere? It does not.

No LangChain / LangGraph / DeepAgents

A full-repository search (grep -rniIE "langchain|langgraph|deepagent" across every .py, .md, .toml, .json, .txt file) returns zero matches. None of the five components declare langchain, langgraph, deepagents, crewai, autogen, or any comparable agent-orchestration package as a dependency. This is a real architectural choice, not an oversight — VulnHunter's own docs describe the reasoning explicitly (see vulnhunter-fix/vulnhunter_fix/graph/build.py): graph.build runs Graphify's AST extraction in-process, with a hard check (check_backend_isolation()) that refuses to proceed if any cloud-LLM environment variable is set — the code-graph step is required to stay non-agentic.

VulnHunter's only two agentic mechanisms are:

  1. Native Claude Code subagent dispatch — the Agent tool, used by all three skills to spawn phase/worker subagents. This is a Claude Code primitive, not a third-party framework.
  2. The Claude Agent SDK (claude-agent-sdk) — the single orchestration dependency of vulnhunter-agent/pyproject.toml, used only to drive /vulnhunt headlessly.

No component builds a state graph, a DAG of LLM calls, or a multi-agent framework of its own; the "agentic" behavior is entirely Claude Code's native subagent model.

Per-component breakdown

vulnhunt/ (scanner) — almost entirely LLM reasoning

There is no static-analysis engine in this component at all — no tree-sitter pass, no AST parse of its own. Every phase is a subagent that reasons over what Grep/Glob/Read surface. The only deterministic pieces are the navigation tools themselves and, in bash-enabled mode, two mechanical steps: dependency install (npm install / pip install -r / go mod download / mvn dependency:resolve / sbt update — chosen deterministically by which manifest file is present) and exploit-test execution (the target project's own test runner). /cost after each phase is a deterministic spend query, not reasoning. In read-only mode, there are no external process calls at all.

vulnhunter-fix/ (fixer) — a genuine hybrid

This is the component where deterministic code brackets LLM reasoning most heavily:

Deterministic (always same output for same input) LLM reasoning
scripts/detect_mode.sh — in-place vs fork, from env/CWD Plan the fix approach per cluster
scripts/language-detect.py — manifest files, then suffix majority-vote Write the exploit demo
scripts/compute-completeness-tier.py — walks fixed signals conservative-first (WORKAROUND → MITIGATION → FULL), only escalates to a bounded LLM prompt when no signal matches Write the RED (failing) security test
scripts/parse-tier-judgment.py — re-validates that bounded LLM output against a JSON Schema Implement the GREEN fix
vulnhunter_fix/graph/build.py — Graphify AST-only extraction, content-hash cached, refuses cloud-LLM env vars —
git worktree create/reset/commit (setup_worktree.sh, worktree-reset.py) —
gh CLI — PR creation —
pytest / the project's test runner — RED/GREEN execution —
jsonschema (Draft 2020-12) validation of finding/fix-plan/result/triage artifacts —
The six delivery gates (run-gates.py → check-*.py/validate-*.py/anti-merge-check.py) — severity-mask, body-completeness, scope, idempotency, anti-merge, verification-table; each is a stateless pass/fail script —
Per-finding model routing — config.json → models.routing picks opus/sonnet/haiku by severity + CWE prefix + multi-file flag, a deterministic lookup (the routed model then reasons about the fix)

The design intent is explicit: LLM output is only ever trusted after it passes a deterministic check. The completeness-tier classifier is the clearest example — it is documented to never silently return FULL; only a matched deterministic signal, or an escalation to a schema-validated bounded LLM call, produces a result.

vulnhunt-fix-verify/ (verifier) — almost entirely LLM reasoning, by design

The tool envelope is Read/Write/Edit/Glob/Grep/Agent — no Bash, no network, so it structurally cannot run deterministic external tools like git, a test runner, or a linter. The only deterministic step is JSON-Schema validation of the emitted verify_disposition.json at Phase 4, plus file-existence checks used as phase synchronization signals. Everything else — reading the fixed code and the original finding, deciding whether the fix holds — is model reasoning over static reads. This is intentional: verification is supposed to re-derive the answer independently, not re-run the fixer's own deterministic machinery.

vulnhunter-agent/ (headless runtime) — a deterministic shell around one LLM call

The runtime is software-engineering plumbing wrapped around a single LLM invocation (the scan itself, which it does not reason inside — it just launches and streams it):

harness/ (batch & benchmark, dev-only) — deterministic orchestration + one LLM judge

git clone, subprocess.run(["claude", "-p", "/vulnhunt", ...]), a ThreadPoolExecutor with retry/backoff and per-scan timeouts, JSON parsing of ground truth and results, tally/report rendering, and atomic state persistence after every operation are all deterministic. The judge (judge.py) is the one LLM call in this component — it compares the scanner's findings against a known-vulnerable corpus using the single MODEL constant; the scoring that follows the judge's output is deterministic arithmetic.

The pattern across all five components

The two prompt-only skills (vulnhunt, vulnhunt-fix-verify) are almost pure LLM reasoning — by design, since their job is judgment, not mechanical processing. The three Python components (vulnhunter-fix, vulnhunter-agent, harness) each follow the same shape: deterministic code handles configuration, credentials, version control, network calls, schema validation, and pass/fail gating; LLM calls are reserved for the parts that require judgment (writing a fix, extracting findings from prose, judging accuracy against ground truth) and are always followed by a deterministic check before their output is trusted — a schema validation, a gate script, or a human Approve/Pause. No component delegates orchestration itself to an LLM framework; the control flow is ordinary Python (or Claude Code's native subagent dispatch), and the LLM is called as a function, not as the thing driving the loop.


See also: HLD · LLD · module index (each module doc's Part 3 has the same inputs/outputs/model-selection/deterministic-calls breakdown, per component).