Inside Project Glasswing: How Five Independent Teams Built the Same Machine
An architecture review of the AI vulnerability-discovery-and-remediation harnesses built around Project Glasswing, and what happens when the model finds bugs faster than anyone can fix them.
What is Project Glasswing?
On April 7, 2026, Anthropic announced Project Glasswing: an initiative bringing together Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks around a single observation — a new, unreleased frontier model, internally named Claude Mythos Preview, had crossed a threshold in offensive security capability [7]. Working mostly autonomously, it found a 27-year-old remote crash in OpenBSD, a 16-year-old bug in FFmpeg that automated fuzzers had hit five million times without catching, and a chained privilege-escalation exploit in the Linux kernel [7][8]. On a reproduction benchmark called CyberGym, it scored 83.1% against 66.6% for Anthropic’s next-best model [7][10].
Glasswing is the response to that finding: give the model to defenders — launch partners plus more than 40 additional organizations that build or maintain critical infrastructure — before its capabilities proliferate to people who aren’t trying to be careful with them [7]. Anthropic committed up to $100M in usage credits across the program, plus $4M in direct donations to open-source security organizations, and promised a public report on lessons learned within 90 days [7].
This piece isn’t about Mythos Preview itself. It’s about what happens next: what a partner actually has to build around a model like that to turn “the model found a bug” into “the bug got fixed, safely, at scale.” That thing is called a harness, and at least five organizations have independently built one.
Why was it created?
The case for Glasswing rests on a specific asymmetry: AI has gotten dramatically better at finding vulnerabilities, faster than the world’s supply of people who can triage and fix them has grown. Anthropic’s own framing is blunt — “the window between a vulnerability being discovered and being exploited by an adversary has collapsed… what once took months now happens in minutes with AI,” in the words of CrowdStrike’s CTO, quoted on the Glasswing announcement page [7]. Cisco’s SVP of Security put it similarly: “the old ways of hardening systems are no longer sufficient” [7].
Underneath the announcement is a workforce reality that makes the timing legible. There were roughly 4.8 million unfilled cybersecurity roles globally in 2025, over 750,000 of them in the US alone [12]. In ISC2’s 2025 workforce survey, 59% of security professionals said they were considering a career change, and about half reported feeling exhausted just trying to keep current with the threat landscape [11]. Proofpoint found 63% of CISOs personally witnessed or experienced burnout in the past year [14]; Splunk’s 2025 State of Security put 52% of security teams as overworked, with a third already planning to close the gap with AI and automation [13]. Organizations that lean into security AI and automation save an average of $2.2M in breach costs and detect and contain incidents roughly 100 days faster than peers who don’t [15].
Put those two facts together and Glasswing’s premise is simple: a model that can find thousands of zero-days across every major OS and browser in a few weeks [7] is arriving on top of a workforce that was already stretched before AI made discovery this cheap. Discovery was never really the bottleneck. Triage and remediation were — and still are.
What code harness is needed?
“Point a frontier model at a repo and ask it to find bugs” does not scale, and every team reviewed here converged on the same diagnosis before building anything. A single chat-style pass with a general coding agent holds one hypothesis at a time, fills its context window after covering a sliver of a real codebase, and loses information the moment it has to compact its context [6]. Cloudflare’s engineering team, writing about their own build, put the failure mode plainly: an hour into an unstructured run, “the context window fills up and the model will cannibalize its own memory, instantly forgetting the bugs it spent all morning tracking down” [6].
A harness is the answer to that failure mode: persistent state outside the model, work broken into narrow single-purpose agent calls, and a pipeline that can pause, resume, and be audited stage by stage. It’s the difference between asking one person to review 50,000 lines of code from memory and running a structured chain of specialists who each own one narrow question, write their answer to a database, and hand off. None of the five teams below built this because it was elegant. Every one of them built it after hitting the same wall a lighter-weight approach couldn’t get past.
Who has developed them?
Five implementations of this pattern were reviewed for this piece, four of them named, purpose-built harnesses and one a large-scale internal system described in detail but not open-sourced:
| Harness | Built by | Status | Core mechanism |
|---|---|---|---|
| VVAH — Visa Vulnerability Agentic Harness | Visa | Open source (Apache License 2.0), config-driven, model-agnostic | 11-stage pipeline (S0–S11), swappable AI “skills,” 4 backend transports |
| VulnHunter | Capital One | Open source (Apache-2.0) [2] | Closed loop of 3 Claude Code skills: Hunt → Fix → Verify |
| STRIX | Independent open-source project | Open source [3] | Root/child agent tree in a shared Docker sandbox, real pentest tooling |
| DCRH — Defending Code Reference Harness | Anthropic | Open source, unmaintained reference implementation [4] | Find → fresh-container grade → patch/re-attack, direct Claude Code CLI |
| VDH / VVS — Vulnerability Discovery Harness / Validation System | Cloudflare | Internal, described in detail in two public posts [5][6] | 8-stage discovery loop feeding a shared, multi-harness triage system |
A sixth reference point, Alibaba’s Open Code Review (OCR) [9], sits outside Project Glasswing entirely — it’s a code-review tool, not a vulnerability harness, built independently by Alibaba and open-sourced after two years of internal use. It’s included in this review as a control: does the same architectural pattern show up in a completely unrelated lineage, solving a different problem? It does, and that turns out to matter for the argument later in this piece.
What I reviewed
The review covered, for each implementation: the architecture and design documentation (high-level and low-level design, data-flow diagrams, module-by-module breakdowns); the source code structure itself, cross-checked with static code-graph tooling rather than taken from READMEs; the public writeups where they exist (Cloudflare’s two-part blog series [5][6], Anthropic’s DCRH repository and system documentation [4], VulnHunter’s public repository [2]); and the primary Project Glasswing announcement and its underlying model card claims [7][8]. Internal facts — trust boundaries, tool permissions, model-routing logic, sandbox configuration — were verified against source where source was available, not paraphrased from marketing copy.
What was the purpose of the review?
Three questions drove this review. First: does “point a frontier model at a codebase” actually require this much scaffolding, or is the harness pattern overkill? Second: when five teams solve the same problem independently — a bank, a card network, an open-source pentest project, a frontier lab, and a CDN — where do they converge, and where do they genuinely diverge? Third, and least answered by any of the five: what happens once the finding leaves the harness and lands on code the operator doesn’t control? None of VVAH, VulnHunter, STRIX, DCRH, or Cloudflare’s system is built to answer that third question, and the gap is instructive.
What framework am I reviewing them against?
Each harness was compared along six axes, chosen because they recur across every design document reviewed rather than imposed from outside:
- Discovery mechanism — how candidates are generated, and how false positives are suppressed at the source.
- Validation / adversarial review — whether a separate process tries to disprove findings before they’re trusted, and how isolated that process is from the finder.
- Remediation posture — whether the harness edits code at all, and if so, under what constraints.
- Model / backend routing — whether the system is pinned to one model or built to swap models, and whether discovery and validation deliberately use different ones.
- Trust boundaries and sandboxing — what the harness assumes about the code it’s scanning, and what a compromised or adversarial target could and couldn’t do to it.
- Quality mechanisms and metrics — what, concretely, each team measures to know the system is working.
What is the difference in their architecture?
At the level of packages and stage names, the five look unrelated. VVAH is an 11-stage Python CLI pipeline (S0 through S11) with four pluggable backend transports (cli, sdk, openai, deepagents) selected per-role by a config file [1]. VulnHunter is three separate Claude Code skills — /vulnhunt, /vulnhunter-fix, /vulnhunt-fix-verify — plus two Python runtimes that drive them at scale, with zero dependency on any multi-agent framework — no LangChain, LangGraph, or DeepAgents anywhere; the sole agent-related dependency is the Claude Agent SDK, confined to a dev-only batch runner [2]. STRIX is a tree of agents coordinated through a mailbox model inside one shared Docker sandbox, built on top of the OpenAI Agents SDK as a hard dependency [3]. DCRH invokes the Claude Code CLI directly as a subprocess rather than through any SDK, reasoning in its own source comments that “the Agent SDK is itself a subprocess wrapper around the same CLI; going direct keeps the argv shape under our control” [4]. Cloudflare’s system is the largest by scope — a fleet scanner spanning 128–145 repositories across Rust, Go, C, Lua, TypeScript, and Python, feeding a single shared triage system that other harnesses can also feed into [5][6].
Underneath those surface differences, the diagram below is what actually recurs.
(Diagram note: this second figure previews the open-source remediation problem discussed later in this piece — but the pattern in the first figure, discovery → adversarial validation → tightly-leashed remediation, is what every one of the five teams built independently.)
What architecture design decisions have they taken?
1. The finder and the grader are never the same process, and often not even the same model. VVAH runs majority-vote filtering across repeated independent analysis passes and, in its default configuration, routes its S10 (remediate) and S11 (validate) stages through a DeepAgents/LangGraph runtime with deliberately different tool permissions for each [1]. VulnHunter’s falsification engine actively tries to disprove roughly half of its own candidates before they’re ever written up [2]. DCRH’s grading agent runs in a fresh container that never sees the finder’s reasoning — only the proof-of-concept bytes cross the boundary [4]. Cloudflare goes furthest: their Vulnerability Discovery Harness and Vulnerability Validation System deliberately run on two different models, so that “Model B is forced to judge the output of Model A… an unbiased, adversarial third party whose sole job is to ruthlessly stress-test Model A’s assumptions” [6].
2. Findings require executable proof, not a plausible-sounding narrative. All five teams independently arrived at the same countermeasure against a model confidently reporting a vulnerability that isn’t real. VulnHunter requires a working proof-of-concept for every finding [2]. STRIX will not accept a report without a demonstrated exploit script and captured evidence [3]. DCRH requires a crash to reproduce three times out of three against an ASAN-instrumented build before it counts [4]. Cloudflare’s Hunters must compile and run code, not just read it, and their Validator “cannot log findings of its own; its sole job is to aggressively disprove the Hunter’s theory” [6] — because, as their own postmortem states, “if a Hunter is allowed to grade its own homework, it will confidently validate everything it outputs” [6].
3. Remediation gets a shorter leash than discovery, everywhere. No implementation reviewed grants its remediation agent an unrestricted shell. VVAH’s post-scan agents get Read/Glob/Grep(/Edit/Write) but never Bash, “because a prompt-injected agent with a host shell would be RCE on the scanner” [1]. VulnHunter’s fixer edits only on a git branch or worktree, with an operator approving every phase boundary [2]. DCRH wraps every piece of target-derived text reaching its patch agent in an <untrusted_data> block, explicitly calling this a mitigation, not a guarantee [4]. Every one of the five stops short of auto-merge; a human is the last gate before code moves.
4. Model routing is a genuine fault line, not a footnote. Here the five actually diverge, and the divergence is informative. VVAH is deliberately model-agnostic via a config-selectable backend router, including an optional agent-framework runtime [1]. VulnHunter and DCRH both independently chose to build on zero multi-agent framework — no LangChain, no LangGraph, no DeepAgents; DCRH goes further and bypasses even the vendor’s Agent SDK — reasoning that direct control over the transport layer beats framework convenience [2][4]. STRIX sits at the opposite extreme: it hard-depends on the OpenAI Agents SDK for its agent-loop primitives, even while keeping the underlying LLM itself fully pluggable via LiteLLM [3]. Cloudflare treats model choice as the one thing that should never be load-bearing: “the future of agentic workflows will not be found in standalone models, prompts, or single-agent sessions… the harness is the bit that lasts” [6].
5. Trust boundaries assume the target is hostile, not just buggy. Every harness treats the scanned repository as untrusted input capable of attempting prompt injection, not merely as a source of code to read. VVAH refuses a config.yaml found inside the scanned repo itself, to prevent a committed file from redirecting model credentials [1]. VulnHunter’s verifier runs with no Bash and no network at all [2]. DCRH’s autonomous pipelines execute entirely inside gVisor on an egress-restricted network with an allowlist proxy [4].
Component-by-component: where the same job gets a different machine
The stage-level view above hides a second layer of comparison: each harness is assembled from the same seven functional components, and the component-level choices diverge more than the stage diagrams do. This is where implementation teams should actually compare notes.
| Component | VVAH (Visa) | VulnHunter (Capital One) | STRIX | DCRH (Anthropic) | VDH/VVS (Cloudflare) |
|---|---|---|---|---|---|
| State store / resume | run_manifest.json + SQLite outside the scanned repo [1] | Git branches/worktrees as state [2] | Shared sandbox state, agent mailboxes [3] | Per-task result.json terminal status [4] | Store keyed by (run_id, repo, stage) [5] |
| Recon / context builder | Early inventory stages of the S0–S11 pipeline [1] | Folded into the hunt skill [2] | Dynamic — agents probe the running target [3] | Minimal by design (reference scope) [4] | Dedicated Recon stage — its context quality drove the 40%→11% rejection-rate gain [6] |
| Finder | Majority-vote across repeated independent passes [1] | Hunt skill, over-reports by design [2] | Child agents per attack surface [3] | Finder agent, PoC-oriented [4] | Hunters that must compile and run code [6] |
| Validator | Separate S11 stage, distinct tool permissions [1] | Falsification engine — kills ~half its own candidates [2] | Exploit script + captured evidence required [3] | Fresh container, PoC bytes only [4] | Different model entirely; cannot log its own findings [6] |
| Deduplication | Per-run | Per-run | Per-engagement | Per-run | Fleet-level shared triage pool — folded 5,442 of 13,841 findings [6] |
| Fixer | No-Bash toolset, policy file-driven [1] | Branch-only edits, RED→GREEN gate [2] | Out of scope (reports, doesn’t fix) [3] | <untrusted_data>-wrapped inputs, human review [4] | Automated Fixer, ~5 min/bug, PR output [6] |
| Human-interaction surface | validator_hints.yaml, remediation_policy.yaml [1] | Approval gate at every phase boundary [2] | Operator watches live | Merge review | Wishlist queue — 25,472 writes [6] |
| Sandbox layer | Tool-permission gating per role [1] | No-Bash/no-network verifier [2] | Shared Docker sandbox [3] | gVisor + egress allowlist proxy [4] | Isolated build/run environments [6] |
Two component-level findings stand out. Deduplication is the least-developed component everywhere except Cloudflare — four of five implementations dedup only within a single run, which is fine for one repo and collapses at fleet scale; Cloudflare’s shared triage pool is the only fleet-grade dedup component in the group, and it’s also the piece that makes multi-harness operation possible at all [6]. Recon is the highest-leverage component and the least standardized — Cloudflare attributes its largest measured accuracy improvement not to a better model or a better prompt but to feeding the finder better context [6], yet recon ranges from a dedicated stage (Cloudflare, VVAH) to nothing at all (DCRH). A team looking for the cheapest quality win should look at its recon component before touching anything else.
The security risks inside each component — what replicators must address
A harness is itself an attack surface, and a peculiarly attractive one: it runs with model credentials, executes attacker-influenceable code on purpose, and accumulates a database of validated, unpatched vulnerabilities with working proof-of-concept exploits. The five teams’ controls exist because each of these components has a specific failure mode. Anyone replicating this pattern inside their own organization should treat the following as a minimum risk register, component by component:
State store. The risk is twofold: the store holds the
The five decisions above describe what was built. It’s worth separating that from why — because the same three-stage architecture emerged from noticeably different first principles, and the difference predicts where each harness will bend as requirements change.
| Harness | Architecture choice | Underlying design principle |
|---|---|---|
| VVAH (Visa) | 11 config-selected stages, 4 swappable backend transports, per-role model keys [1] | Policy separated from mechanism. Governance decisions (which model, which tools, which framework) live in config, not code — the pipeline outlives every one of those choices. |
| VulnHunter (Capital One) | 3 human-invocable Claude Code skills, zero frameworks, RED→GREEN test gates [2] | Auditability over abstraction. Every phase is a thing a human can run, read, and approve by hand; nothing hides behind a framework’s event loop. |
| STRIX | Root/child agent tree in one shared sandbox, hard OpenAI Agents SDK dependency [3] | Emulate the human red team. The org chart of a pentest engagement, reproduced in agents — buys capability breadth at the price of transport control. |
| DCRH (Anthropic) | Direct CLI subprocess, gVisor isolation, fresh-container grading [4] | Minimize the trusted computing base. Every layer that could hold hidden state or hidden behavior is removed or isolated; clarity of the reference is itself a feature. |
| VDH/VVS (Cloudflare) | Producer-consumer fleet loop, cross-model validation, shared triage pool [5][6] | The harness outlives the model. Explicitly anti-model-loyalty: any component that would break on a model swap is treated as a design defect. |
| OCR (Alibaba, control) | Deterministic scaffold around one narrow AI judgment call [9] | Minimize the AI surface area. Tokens are the expensive part; everything that can be deterministic should be — reporting ~9× lower token cost than a general agent [9]. |
Read down the right-hand column and a cleaner split appears than the architecture diagrams suggest. Two teams reasoned from security first (DCRH’s minimal TCB, VVAH’s least-privilege roles). Two reasoned from economics first (OCR’s token minimization, Cloudflare’s per-repo budgeting and model portability). Two reasoned from workflow first (VulnHunter’s human-auditable phases, STRIX’s red-team emulation). All six converged on finder → adversarial validator → leashed fixer anyway. That’s the strongest form of the convergence argument: the same machine falls out of three different starting axioms, which suggests the shape is forced by the problem, not by fashion.
The practical use of this mapping: when evaluating or extending a harness, its founding principle tells you what it will resist. VulnHunter will resist automation that removes the human-readable phase boundary. STRIX will resist decoupling from its SDK. Cloudflare’s system will resist anything that pins it to one model. Extensions that fight the founding principle cost more than the architecture diagram implies.
What is the general outcome?
Cloudflare is the only one of the five that has published a hard funnel with real numbers, and it’s worth reading closely because it shows what “it’s working” actually looks like at scale, not in theory. Across their fleet: 20,799 raw candidates were generated, of which 12,057 survived independent validation; those, joined with findings from another harness, brought a shared triage pool to 13,841; deduplication folded away 5,442 of those as duplicates; 1,154 more were routed out as wrong-repo or low-risk; leaving 7,245 actionable findings for engineering teams [6]. Their initial validation rejection rate dropped from 40% to 11% as their Recon stage’s context improved, while the share of high-integrity findings climbed from 35% to 58% [6]. For a standard ~30,000-line repository, a full run produces around 100 initial findings in 3–4 hours, compresses to roughly 80 distinct high-fidelity bugs within another 3 hours, and the automated Fixer processes those at about 5 minutes per bug — discovery through open pull request in roughly 14 hours [6].
The honest caveat, stated by Cloudflare itself, is that none of this produces a real recall number: “there’s no labeled set of every real bug in a codebase, so any claimed recall number is entirely speculative” [6]. What every team instead tracks is the shape of the funnel — how sharply raw noise compresses into trusted findings — because that’s the only thing that’s actually measurable.
What does a finding actually cost?
The most common first reaction to these systems — “it burns too many tokens, it’s too expensive” — is real but imprecisely aimed. No team reviewed publishes a dollar figure per finding, but the design documents and Cloudflare’s published numbers let the cost structure be reconstructed, and it’s more interesting than a single number would be.
Discovery dominates spend, and its cost is wildly variable. Cloudflare budgets per-repository rather than per-run precisely because Hunt-stage cost varies so much between codebases [6] — a deep C codebase with a complex build costs a different order of magnitude to hunt than a small TypeScript service. That variability is why “what does a run cost” is the wrong question; “what does this repo cost to keep scanned” is the one operators actually budget against.
Validation is cheap relative to discovery, because falsification is targeted. A validator doesn’t explore a codebase; it attacks one specific claim with one specific proof-of-concept. This is part of why the adversarial-validation pattern won everywhere: it buys a large precision improvement for a small share of total token spend. The expensive stage and the accuracy-critical stage are not the same stage.
Deduplication is the highest-leverage cost control in the whole pipeline. In Cloudflare’s funnel, 5,442 of 13,841 pooled findings — roughly 39% — folded away as duplicates before any engineer saw them [6]. Every duplicate that survives dedup costs the most expensive resource in the system: human triage attention. The same logic explains Alibaba’s result from the control case — OCR’s deterministic scaffolding, which reserves the model for one narrow judgment call, reports roughly 9× lower token cost than a general-purpose agent doing the same review [9]. The cheapest token is the one a deterministic component made unnecessary.
The right benchmark for the total is not zero — it’s the human alternative. For scale: a full run on a ~30,000-line repository lands at roughly 14 hours from discovery to open PR, producing on the order of 80 distinct high-fidelity bugs [6]. Compare any plausible model bill for that run against the same work bought as human labor — weeks of senior review time, or per-bug bounty payouts that routinely run four to five figures for the classes of bug these systems surface — and the token complaint inverts. The genuine cost risk isn’t token spend; it’s spending tokens on findings that turn out to be false positives, because every false positive that reaches an engineer spends trust as well as time. This is why every mature team measures cost per validated finding, not cost per finding, and treats the false-positive rate as a cost metric, not just a quality metric.
The metrics a harness program should track
Across the five implementations, and independent of the model debate, the same measurement stack recurs. Written out as what a program should actually instrument, it has four layers and one honest gap:
Funnel metrics — raw candidates generated; validation rejection rate, tracked as a trend (Cloudflare’s fell from 40% to 11% as their Recon stage’s context improved [6]); deduplication rate; actionable findings delivered. The funnel’s shape is the closest thing to a system-health dashboard this domain has.
Economic metrics — token spend by stage; cost per raw candidate; cost per validated finding; cost per merged fix; and human-minutes per finding, which is the number most teams forget to instrument and the one most likely to be the real bottleneck. Budget per-repository, not per-run [6].
Quality metrics — false-positive rate at each gate, separately, so regressions localize; proof-of-concept reproduction rate (DCRH’s 3-of-3 crash standard is the strictest published [4]); detection accuracy against a held-out, known-vulnerable corpus (VulnHunter’s dev-only benchmark harness exists specifically for this: clone → scan → LLM-judge → tally [2]); and fix acceptance rate — the share of machine-drafted patches humans actually merge, which is the single best proxy for whether the remediation side earns its cost.
Operational metrics — time from discovery to open PR; pipeline resume/recovery rate after crashes and rate-limits; and human-intervention count. Cloudflare’s Wishlist counter is the model here: agents wrote requests for missing infrastructure 25,472 times across their fleet [6], a direct measurement of how far from autonomous the system actually is, and a number you’d want to see falling quarter over quarter.
The honest gap: false-negative rate is unmeasurable and every credible team says so. “There’s no labeled set of every real bug in a codebase, so any claimed recall number is entirely speculative” [6]. A benchmark corpus of known vulnerabilities gives a recall proxy; it can never say what the system missed in the wild. Any vendor or internal team claiming a real-world recall or false-negative number should be asked how they could possibly know — and the answer will be a version of the benchmark proxy, or hand-waving.
Security risks in the modules themselves — read this before you replicate one
The reviewed harnesses are built by security teams, but the harness is itself a piece of software that ingests hostile input, holds secrets, executes untrusted code, and stores exploitable findings. An organization copying these modules inherits their attack surface, and several of the risks are not obvious from the architecture diagrams. Mapped to the shared component model, these are the module-level risks worth an explicit threat-modeling pass before anything runs against real code.
The finder / discovery module — prompt injection from the target is the load-bearing threat. The scanned repository is attacker-controllable input, and a committed file, a crafted comment, or a poisoned README can attempt to steer the model. This is why VVAH refuses a config.yaml found inside the target [1] and DCRH wraps all target-derived text in <untrusted_data> markers [4] — but note DCRH’s own docs call that wrapping a mitigation, not a guarantee [4]. If you replicate this module, treat every byte from the target as adversarial: never let target content reach a system-prompt position, never let it select a model or endpoint, and assume the wrapping can leak. The risk that’s easy to miss: the finder often has the widest tool access of any stage (it needs to read broadly and often compile/run code), so a successful injection here has the largest blast radius.
The validation / sandbox module — this is where untrusted code executes, and containment is the whole game. Every harness runs attacker-relevant code to prove findings; that means arbitrary code execution is a feature of this module, not a bug. DCRH’s answer is gVisor plus an egress-restricted network whose only exit is an allowlist proxy [4]; STRIX runs children in a shared Docker sandbox [3]. The replication risks: (1) Docker alone is a weaker boundary than gVisor/microVMs for genuinely hostile code — a container escape is a host compromise; (2) a shared sandbox (STRIX’s model) means one poisoned target can affect sibling agents’ work, so isolation-per-target matters if you scan multiple repos; (3) egress is the exfiltration path — without an allowlist proxy, a malicious build script can phone home, and your scanner becomes the leak. Default-deny egress and per-run disposable sandboxes are the controls to copy first.
The remediation / fixer module — a prompt-injected fixer with write access is the highest-severity failure. Every reviewed harness deliberately gives this module the shortest leash: VVAH grants Read/Glob/Grep/Edit/Write but never Bash, on the explicit reasoning that “a prompt-injected agent with a host shell would be RCE on the scanner” [1]; VulnHunter’s fixer edits only on a branch/worktree with a human gate [2]. If you replicate this, the non-negotiables are: no shell for the fixer, branch-only writes (never direct-to-main), and a human merge gate — all five kept the last one. The subtle risk: the fixer is the module most tempting to over-automate for throughput, and it’s precisely the one where over-automation converts a false finding or an injection into merged, attacker-influenced code.
The state / findings store — a database of unpatched vulnerabilities with working exploits is a crown-jewel target. This is the risk least discussed in the source material and arguably the most serious for a replicating org. VVAH keeps SQLite state outside the scanned repo [1]; the store holds proof-of-concept exploits for real, unfixed bugs in your own systems. If that store is breached, you’ve handed an attacker a pre-validated target list. Controls to add that none of the reviewed docs fully specify: encrypt findings at rest, tightly scope who and what can read the store, put findings under the same (or stricter) access control as your most sensitive incident data, and set retention limits so validated-and-fixed findings don’t linger as an exploit library.
The credential / model-routing module — the harness holds API keys and often cloud credentials. Model API keys, and frequently repo-read and CI credentials, live in this module’s config. The target-config-refusal controls [1] exist partly to stop a committed file from redirecting those credentials to an attacker endpoint. Replication risks: secrets in environment variables or config files that a compromised stage can read; over-broad repo tokens (read-only, least-privilege, short-lived is the target); and model-endpoint pinning, so an injection can’t silently reroute inference (and your prompts, which may contain source code) to a hostile URL.
The orchestration / customization module — operator hooks are a supply-chain surface. The validator_hints.yaml / remediation_policy.yaml-style customization files [1], playbooks, and Wishlist-style human-fulfillment mechanisms [6] are powerful precisely because they change agent behavior without code review. That makes them an insider- and supply-chain-risk surface: a malicious or careless policy file can widen tool permissions, disable a validation gate, or approve infrastructure requests that shouldn’t be granted. Version-control and review these files like code, and keep the human who fulfills Wishlist requests trained to treat them as privileged changes, not chores.
The through-line: these systems are safe as their authors run them because the authors are security teams who threat-modeled their own tool. A replicating organization gets the architecture for free but not the threat model. Before pointing a copy at production code, run the harness through the same review you’d give any system that executes untrusted input, holds secrets, and stores exploit material — because it does all three.
What are the next set of challenges?
Is it the model, or the harness?
There’s a live disagreement inside the industry about where the value actually sits. Cloudflare’s own framing states it as their core thesis: different models surface a different share of the same bugs when pointed at the same target, and no single model should be load-bearing for a defensive pipeline built to last [6]. That’s a direct argument for the “design and execution matter more than model tier” position some teams hold — the harness (retry logic, context discipline, adversarial validation, deduplication) is portable across a model swap; a prompt tuned to one model’s quirks is not. None of the five teams claims a cheaper model performs as well as a frontier one on raw discovery — but several structurally hedge against needing to bet on any one model staying best.
Autonomy is being dialed up incrementally, not switched on
Every implementation gives operators a way to customize scope and behavior without touching the harness’s core code: VVAH’s per-role models.<role> config keys and operator-supplied validator_hints.yaml / remediation_policy.yaml [1]; VulnHunter’s /customize-equivalent playbooks; DCRH’s dedicated /customize skill; Cloudflare’s “Wishlist” mechanism, where an agent that needs infrastructure it doesn’t have — a FreeBSD VM, a specific build environment — writes a request that a human fulfills once, after which the system resumes automatically (written to 25,472 times across their fleet, by their own count) [6]. That’s the practical shape autonomy takes in production: not a single flag from manual to autonomous, but dozens of small, reversible delegations of judgment, each backed by a human who can still say no.
The open-source remediation problem
This is the challenge none of the five harnesses is built to solve, and it’s worth treating as its own section because the gap is structural, not an oversight. Every harness reviewed here assumes the operator controls the merge button. That assumption holds for a bank’s own codebase, a CDN’s own fleet, or a reference implementation’s own demo targets. It does not hold for the dependency an operator’s code pulls in three hops away.
The embargo model was built for one merge button, not several. Coordinated Vulnerability Disclosure — privately notifying a maintainer, holding details back, disclosing only once a fix ships — is the industry’s standard mechanism, but its timelines vary sharply with how much trust and infrastructure already exists around a project. The Linux distros mailing list, coordinating dozens of downstream distributions that all need to ship simultaneously, holds embargoes to under 7 days, 14 at the absolute maximum [17]. CERT/CC discloses 45 days after a report regardless of whether a fix exists [17]. Google Project Zero uses 90 days plus a 14-day grace period, and reports 95.5%–96.9% of issues fixed under deadline over the program’s lifetime [16]. Ninety days is treated as the general OSS norm absent a project-specific policy [17]. None of these windows was designed with a second variable in mind: how long it takes a fix to propagate once it exists.
Transitive dependencies turn one merge button into an unknown number of them. In the npm ecosystem, roughly 24.78% of all library versions are transitively affected by a known vulnerability, originating from only about 19.96% of libraries — a small set of root bugs fanning out through the dependency graph far beyond where they started [19]. Even after a fix ships upstream, an estimated one in five active npm packages can still introduce the vulnerability through a stale direct or transitive pin [19]. And the trend is getting worse, not better: one study found average remediation time rising from roughly 25 days in 2017 to more than 300 days in 2024 [20], plausibly because dependency graphs have simply gotten deeper over that period.
Log4Shell is the clearest illustration available. Over 12,000 Maven artifacts were affected; for more than 80% of them, the vulnerability sat more than one dependency level deep — meaning the affected project couldn’t fix itself even if it wanted to, because it had to wait on an intermediate library to fix itself first [21]. Three years after disclosure, Contrast Security found 12% of Java applications still running a vulnerable Log4j version, and Sonatype found roughly 13% of 2025 Log4j downloads were still on a vulnerable version [22]. An AI harness pointed at a downstream consumer’s code today would correctly flag this class of bug — and then have nowhere to send the fix, because the party who can actually ship one may be several hops and several separate maintainers away.
The abandoned tail makes “time to fix” meaningless for a meaningful share of the graph. Synopsys/Black Duck’s 2026 OSSRA report found 49% of codebases contain components with no development activity in the past two years, and 91% contain components ten or more versions out of date [23]. HeroDevs estimates at least 81,000 package versions across five major registries carry known CVEs on end-of-life lines that will never be officially patched — potentially over 400,000 across all registries, accounting for roughly 10% of Maven Central and PyPI and a striking 25% of npm [24]. For that slice of the dependency graph, a harness can find the bug and even draft a correct patch, but there is no maintainer left to review a pull request, no embargo to coordinate, and no merge button to press.
Funding is currently weighted almost entirely toward finding more bugs, not fixing the ones already found. Project Glasswing’s own numbers make this concrete: up to $100M in model usage credits for discovery-side work, against $4M in direct donations earmarked for open-source maintainer capacity — $2.5M to Alpha-Omega and OpenSSF via the Linux Foundation, $1.5M to the Apache Software Foundation [7]. The Linux Foundation’s CEO, a launch partner, names the imbalance directly in his own quote on the announcement: open-source maintainers have “historically been left to figure out security on their own,” and the goal of the smaller fund is to make AI-augmented security “a trusted sidekick for every maintainer, not just those who can afford expensive security teams” [7]. Notably, Anthropic’s own roadmap for the program lists “triage scaling and automation” and “patching automation” as open questions still to be worked out in the 90-day report, not solved problems [7] — an implicit admission that the remediation side of this equation is unfinished, even for the launch partners with the largest budgets.
Should a new harness be built?
The question every team arriving at this space now faces: five-plus implementations exist, several of them open source — is building a sixth justified? The review suggests a three-part answer.
Adopt first if your problem matches an existing shape. VulnHunter is production-grade, Apache-2.0, and purpose-built for scanning repositories you own with Claude Code in the loop [2]. STRIX covers the black-box, running-application pentest case [3]. DCRH is explicitly a reference — unmaintained by design, but the cleanest expression of the pattern to learn from before writing anything [4]. VVAH demonstrates what the config-driven, multi-backend version of the pattern looks like at a regulated enterprise [1]. If your codebase is first-party and your model access is mainstream, adopting and customizing one of these is weeks of work; rebuilding their six-week convergence journey from scratch is a known, documented waste.
Build when a constraint genuinely breaks the existing shapes. The legitimate triggers that show up across the design docs: regulatory or data-residency constraints that prohibit the existing tools’ model routing; an existing internal triage system the harness must feed (Cloudflare’s shared triage pool is the template — the harness is replaceable, the triage system is the investment [6]); fleet scale, where per-repo orchestration becomes its own engineering problem; or a target class none of the five handle, like firmware, mobile, or infrastructure-as-code.
What’s commoditized and what isn’t. The three-stage shape, PoC gating, sandbox posture, and no-shell-for-fixers are now settled patterns — six independent teams converging is as close to a standard as this field gets without a standards body. Where the five still genuinely differ, and where a new entrant could add value, is upstream and downstream of the pattern: the quality of recon context fed to the finder (Cloudflare’s biggest measured accuracy win came from improving Recon, not swapping models — 40%→11% rejection rate [6]), integration into real triage and ticketing workflows, and above all the piece nobody has built — the coordination layer for findings in code the operator doesn’t control. A sixth generic harness adds little. The missing product is the remediation-coordination layer for the open-source gap described below, and that’s where new engineering effort would actually move the industry.
What this review still leaves open
Several topics surfaced during this review that none of the design documents adequately address, flagged here as open questions rather than findings. The harness as a target and finding-data governance — the fact that a validated-findings store is itself crown-jewel exploit material — are treated at module level in the security-risk section above; what remains open is that none of the reviewed docs describe red-teaming the harness’s own store or specify its retention, access-control, and disclosure-clock policy. Model-version drift: providers change inference behavior beneath a pinned model name [6], but no team publishes a regression-testing protocol for model updates. Two further gaps are large enough to warrant their own treatment.
The missing product: an open-source remediation-coordination layer
The open-source remediation problem described earlier in this piece is not just a policy gap — it’s a missing product. Every harness reviewed here, VVAH, VulnHunter, STRIX, DCRH, and Cloudflare’s VDH/VVS alike, is built around the assumption that the operator who finds a bug is also the operator who can merge the fix. That assumption fails precisely where the numbers say it matters most: in the npm ecosystem, roughly 24.78% of all library versions are transitively affected by a known vulnerability, originating from only about 19.96% of libraries [19]; Log4Shell affected more than 12,000 Maven artifacts, over 80% of them more than one dependency hop deep [21]; three years on, 12–13% of Java deployments were still running a vulnerable version [22]; 49% of codebases carry components with no development activity in the past two years [23]; and at least 81,000 package versions across five major registries carry known CVEs on end-of-life lines that will never be officially patched [24].
None of the five teams reviewed here has built, or claims to be building, the thing that gap actually calls for: a shared, cross-project coordination layer that tracks a validated finding across every downstream consumer it affects, runs embargo-aware notification to multiple maintainers at once instead of one, tracks fix propagation hop-by-hop through the dependency graph rather than assuming a single merge closes the issue, and manages the fork-or-replace decision when a maintainer never shows up at all. This is the strongest candidate for follow-up engineering work to come out of this review — not because the five harnesses did anything wrong, but because none of them was scoped to solve a problem that sits one layer above where they operate.
A SARIF for agentic findings does not exist yet
Traditional static-analysis tooling settled its interchange problem years ago. SARIF (Static Analysis Results Interchange Format) is the OASIS-standardized JSON format that lets static-analysis tools exchange results with CI pipelines, ticketing systems, and each other — GitHub’s code-scanning feature, for instance, ingests SARIF natively [25]. A scanner that emits SARIF can be swapped for another SARIF emitter without the downstream tooling caring which one produced the data.
No equivalent exists for AI-agent-generated findings, and the five harnesses reviewed here show why that’s already a problem rather than a future one: each has invented its own internal evidence package, incompatible with the others. DCRH’s unit of evidence is proof-of-concept bytes plus a 3-of-3 crash reproduction [4]; VulnHunter’s is a proof-of-concept plus a RED→GREEN test pair [2]; STRIX’s is an exploit script plus captured evidence [3]; Cloudflare’s is a validation verdict sitting in a shared, multi-harness triage pool [6]. A “SARIF for agentic findings” would need, at minimum, attached PoC artifacts, model-provenance metadata (which model or version found the issue, which model or version validated it), a falsification/validation history, and dedup keys stable enough to merge findings across tools and organizations. Without it, findings can’t be pooled or exchanged the way Cloudflare’s internal triage pool already pools them internally — the pattern that works inside one company’s fleet has no path to working across companies.
Closing
Five teams — a card network, a bank, an open-source pentest project, a frontier lab, and a CDN — solved the same problem independently and landed on the same shape: separate the finder from the grader, demand executable proof, keep remediation on the shortest leash of the three, and never let the target dictate its own verdict. A sixth team, building a code-review tool for an unrelated reason inside Alibaba, landed on a compatible shape too — deterministic scaffolding wrapped tightly around one narrow AI-driven judgment call, reporting roughly nine times lower token cost than a general-purpose coding agent doing the same review [9]. That convergence, across five vulnerability harnesses and one unrelated code-review tool, is reasonably strong evidence that this pattern isn’t specific to security work — it’s what “make an LLM’s output trustworthy enough to act on automatically” looks like in general.
What none of them have solved is what happens once the fix leaves the operator’s own repository. The harness pattern was built for a world with one merge button. A meaningful share of the software supply chain doesn’t have one — it has several, several hops away, some of them unattended entirely. That’s not a reason to slow down on the finder side. It’s the next thing that has to get built.
References
- Visa, Visa Vulnerability Agentic Harness (VVAH), github.com/visa/visa-vulnerability-agentic-harness — open-sourced June 10, 2026 under the Apache License 2.0 (© 2026 Visa, Inc.). Reviewed design documentation.
- Capital One, VulnHunter, github.com/capitalone/VulnHunter. Reviewed design documentation.
- STRIX, open-source autonomous penetration-testing agent — architecture and design documentation reviewed for this piece. Reviewed design documentation.
- Anthropic, Defending Code Reference Harness (DCRH), github.com/anthropics/defending-code-reference-harness. Reviewed design documentation.
- Cloudflare, “Project Glasswing: what Mythos showed us,” The Cloudflare Blog, May 18, 2026.
- Cloudflare, “Build your own vulnerability harness,” The Cloudflare Blog, June 18, 2026.
- Anthropic, “Project Glasswing: Securing critical software for the AI era,” April 7, 2026.
- Anthropic Frontier Red Team, “Mythos Preview,” red.anthropic.com/2026/mythos-preview, cited via [7].
- Alibaba, Open Code Review (OCR), github.com/alibaba/open-code-review. Reviewed design documentation.
- CyberGym cybersecurity vulnerability-reproduction benchmark result, cited via [7].
- ISC2, 2025 Cybersecurity Workforce Study.
- DeepStrike, “Cybersecurity Skills Gap Statistics for 2025: Record 4.8M Roles Unfilled.”
- Splunk, 2025 State of Security, cited via industry workforce-automation roundups.
- Proofpoint, Voice of the CISO 2025, cited via industry burnout roundups.
- SANS, 2026 Cybersecurity Skills Report, via Industrial Cyber.
- Google Project Zero, “Policy and Disclosure: 2025 Edition.”
- GitHub Blog, “Coordinated vulnerability disclosure (CVD) for open source projects.”
- kernel.org, “How we coordinate embargoed releases.”
- “Original Sin of npm: A Study on Vulnerability Propagation in JavaScript Dependency Networks,” arXiv.
- Empirical study on open-source vulnerability remediation-time trends (2017–2024), cited via secondary industry summaries.
- Open Source Insights (deps.dev), “Understanding the Impact of Apache Log4j Vulnerability.”
- Vulert, “Log4Shell 2026: Is Your Application Still Vulnerable?”
- Black Duck / Synopsys, 2026 OSSRA Report: Open Source Security & Risk Analysis.
- HeroDevs, “81,000 Open Source Package Versions Have Known CVEs and No Patch.”
- OASIS, Static Analysis Results Interchange Format (SARIF) v2.1.0, docs.oasis-open.org/sarif/sarif/v2.1.0.