Process Document: Operate Visa Vulnerability Agentic Harness (VVAH) Against a Target Repository

docs / process / process-operate-vvah

← all docs

Process Document: Operate Visa Vulnerability Agentic Harness (VVAH) Against a Target Repository

Owner: AppSec / Security Engineering (tool owner) — execution shared with development teams Last Updated: 2026-08-14 | Review Cadence: Quarterly, or on any VVAH minor-version bump

Purpose

Give any analyst, developer, or platform operator a single, unambiguous process for taking a codebase from "unscanned" to "findings triaged, fixes validated" using VVAH, without needing to read the source. It exists so that (a) scans are run consistently across teams, (b) the ⚠️ code-editing and data-egress steps are never run by accident, and (c) findings have a clear, repeatable path to a verdict instead of dying in someone's inbox.

Scope

In scope: running detection scans (S0–S9), remediation (S10), validation (S11), reading and routing the outputs, and the credential/profile setup that gates all of the above.

Out of scope: VVAH's internal design (see ../design/HLD.md and ../design/LLD.md); modifying VVAH's own source code; CI/CD wiring (not shipped by the harness — see Exceptions).

RACI Matrix

Step Responsible Accountable Consulted Informed
Install & credential setup Operator (platform/DevOps) AppSec tool owner Security engineering Requesting team
Choose profile & config AppSec tool owner AppSec tool owner Operator —
Run detection scan (S0–S9) Analyst or CI job owner AppSec tool owner — Development team lead
Triage report.md findings Analyst AppSec tool owner Development team Development team
Run remediation (S10) Analyst or developer (on their own repo) AppSec tool owner Development team lead Development team
Run validation (S11) Analyst or developer AppSec tool owner — Development team, AppSec
Act on verdict (merge / escalate) Development team Development team lead AppSec AppSec
Prune old runs (gc) Operator AppSec tool owner — —

Process Flow

 ┌──────────┐   ┌───────────────┐   ┌────────────────┐   ┌──────────────┐   ┌───────────┐
 │  Setup   │──▶│ Detection scan │──▶│ Triage report   │──▶│ Remediate    │──▶│ Validate  │
 │ (once)   │   │ S0–S9 (⚠read- │   │  .md / .sarif   │   │ S10 (⚠edits │   │ S11       │
 │          │   │  only, egress) │   │  → pick issues  │   │  code)       │   │ (verdict) │
 └──────────┘   └───────────────┘   └────────┬────────┘   └──────────────┘   └─────┬─────┘
                                              │  no issues chosen                    │
                                              ▼                                      ▼
                                         ┌─────────┐                          ┌─────────────┐
                                         │  Close  │◀─────────────────────────│ validated /  │
                                         │  (no    │  needs_review / failed → │ needs_review │
                                         │  action)│  route back to developer │ / failed     │
                                         └─────────┘                          └─────────────┘

Detailed Steps

Step 1: Setup & readiness check (once per environment)

Step 2: Scope check

Step 3: Run the detection scan (S0–S9)

Step 4: Triage findings

Step 5: Remediate (S10) — ⚠️ edits code

Step 6: Validate (S11)

Step 7: Act on the verdict

Step 8: Housekeeping

Exceptions and Edge Cases

Scenario What to Do
Repo is too large / cost is a concern Run estimate first; use --stop-after s3 for an exact scope without full analysis cost; consider the taint.yaml profile (S0 seeding narrows S1–S9 to source→sink paths).
No SDK/CLI credentials available yet, but you need a shape-check Use --stop-after s1 to validate repo access and config resolution without any model spend.
A scan is interrupted (network, timeout, Ctrl-C) Re-run with --resume — completed stages are skipped via checkpoints in the state DB.
config.yaml exists inside the target repo VVAH refuses it (treats the target as untrusted input) and falls back to the packaged default — this is expected, not a bug. Keep your config outside the scanned repo.
Remediation touches a file outside the allowed policy paths The policy gate blocks the edit and reverts it; this is by design (see remediation_policy.yaml), not an error to work around.
Validation returns needs_review because consensus couldn't be persisted Treat as fail-closed by design — always route to manual review, never treat as validated.
You only want fixes proposed, not applied Use vvaharness remediate --mode report-only.
Batch scanning many repos Use --repo-file <csv> instead of --repo; see runbooks/run-a-detection-scan.md §Batch mode.
CI/CD integration Not shipped out of the box — wrap the CLI commands above in your pipeline's own job step; treat scan --stop-after s9 as the CI-safe (read-only) invocation.

Metrics

Metric Target How to Measure
Mean Time to Adapt (MTTA) Trending down quarter over quarter Time from scan completion (Step 3) to a validated verdict (Step 6) on a given finding, from report.md timestamps and DTO validation blocks.
False-positive rate at triage < upstream baseline for the same repo class Findings marked FALSE_POSITIVE at Step 4 ÷ total findings in report.md.
Verdict distribution validated majority; needs_review reviewed within 2 business days Count DTOs by verdict under security-remediation/.
Scan cost per repo Within estimate's projection ±20% Compare estimate output to actual token/cost accounting (util/tokens.py, util/metrics.py — see per-stage cost caps in config.yaml).