Process Document: Operate Visa Vulnerability Agentic Harness (VVAH) Against a Target Repository
Owner: AppSec / Security Engineering (tool owner) — execution shared with development teams Last Updated: 2026-08-14 | Review Cadence: Quarterly, or on any VVAH minor-version bump
Purpose
Give any analyst, developer, or platform operator a single, unambiguous process for taking a codebase from "unscanned" to "findings triaged, fixes validated" using VVAH, without needing to read the source. It exists so that (a) scans are run consistently across teams, (b) the ⚠️ code-editing and data-egress steps are never run by accident, and (c) findings have a clear, repeatable path to a verdict instead of dying in someone's inbox.
Scope
In scope: running detection scans (S0–S9), remediation (S10), validation (S11), reading and routing the outputs, and the credential/profile setup that gates all of the above.
Out of scope: VVAH's internal design (see ../design/HLD.md and
../design/LLD.md); modifying VVAH's own source code; CI/CD wiring
(not shipped by the harness — see Exceptions).
RACI Matrix
| Step | Responsible | Accountable | Consulted | Informed |
|---|---|---|---|---|
| Install & credential setup | Operator (platform/DevOps) | AppSec tool owner | Security engineering | Requesting team |
| Choose profile & config | AppSec tool owner | AppSec tool owner | Operator | — |
| Run detection scan (S0–S9) | Analyst or CI job owner | AppSec tool owner | — | Development team lead |
Triage report.md findings |
Analyst | AppSec tool owner | Development team | Development team |
| Run remediation (S10) | Analyst or developer (on their own repo) | AppSec tool owner | Development team lead | Development team |
| Run validation (S11) | Analyst or developer | AppSec tool owner | — | Development team, AppSec |
| Act on verdict (merge / escalate) | Development team | Development team lead | AppSec | AppSec |
Prune old runs (gc) |
Operator | AppSec tool owner | — | — |
Process Flow
┌──────────┐ ┌───────────────┐ ┌────────────────┐ ┌──────────────┐ ┌───────────┐
│ Setup │──▶│ Detection scan │──▶│ Triage report │──▶│ Remediate │──▶│ Validate │
│ (once) │ │ S0–S9 (⚠read- │ │ .md / .sarif │ │ S10 (⚠edits │ │ S11 │
│ │ │ only, egress) │ │ → pick issues │ │ code) │ │ (verdict) │
└──────────┘ └───────────────┘ └────────┬────────┘ └──────────────┘ └─────┬─────┘
│ no issues chosen │
▼ ▼
┌─────────┐ ┌─────────────┐
│ Close │◀─────────────────────────│ validated / │
│ (no │ needs_review / failed → │ needs_review │
│ action)│ route back to developer │ / failed │
└─────────┘ └─────────────┘
Detailed Steps
Step 1: Setup & readiness check (once per environment)
- Who: Operator
- When: Before the first scan in a new environment, or after a credential rotation
- How:
vvaharness setup(guided) thenvvaharness doctor(live probe). Full procedure inrunbooks/setup-and-credentials.md. - Output: A working profile (
config.yaml) and a cleandoctorrun (0 blocking issues).
Step 2: Scope check
- Who: Analyst
- When: Before scanning a new or unfamiliar repo
- How:
vvaharness estimate --repo <path>— no API spend. - Output: File count / token estimate, used to sanity-check cost before committing to a full scan.
Step 3: Run the detection scan (S0–S9)
- Who: Analyst or CI job owner
- When: On demand, or on a schedule per repo/application
- How:
vvaharness scan --repo <path> --application-id <id> --stop-after s9for detection-only. Full command reference inrunbooks/run-a-detection-scan.md. - Output:
<repo>/security-scan/report.md,report.sarif,errors.jsonl.
Step 4: Triage findings
- Who: Analyst
- When: Immediately after each scan
- How: Read
report.md(ranked by CVSS); cross-check any SARIF import into your tracker of record. Decide which findings warrant remediation now vs. backlog. - Output: A short list of finding IDs to remediate, or a decision that no action is needed.
Step 5: Remediate (S10) — ⚠️ edits code
- Who: Analyst (proposing) or developer (owning the repo)
- When: After triage identifies findings worth fixing
- How:
vvaharness remediate --repo <path> --top N(or--interactiveto pick from a menu). Full procedure, including the policy gate and kill-switch, inrunbooks/remediate-and-validate.md. - Output: Per-finding DTOs under
<repo>/security-remediation/, a diff per fix, and a working-tree change in<repo>— review before committing.
Step 6: Validate (S11)
- Who: Analyst or developer
- When: Immediately after remediation, before the fix is trusted or merged
- How:
vvaharness validate --repo <path> --all(or--finding <id>for one). - Output: A verdict per finding (
validated/validation_failed/needs_review) written back into the DTO.
Step 7: Act on the verdict
- Who: Development team
- When: After Step 6
- How:
validated→ proceed to code review / merge as normal.needs_revieworvalidation_failed→ a human re-examines the diff and either fixes manually or escalates to AppSec. Never auto-merge onneeds_review. - Output: Finding closed, or routed back for manual work.
Step 8: Housekeeping
- Who: Operator
- When: Periodically (weekly/monthly), or when disk/state DB grows large
- How:
vvaharness gc --max-age-days 30 --dry-runfirst, then without--dry-run. - Output: Pruned checkpoint rows in the state DB;
report.md/report.sarifundersecurity-scan/are never touched bygc.
Exceptions and Edge Cases
| Scenario | What to Do |
|---|---|
| Repo is too large / cost is a concern | Run estimate first; use --stop-after s3 for an exact scope without full analysis cost; consider the taint.yaml profile (S0 seeding narrows S1–S9 to source→sink paths). |
| No SDK/CLI credentials available yet, but you need a shape-check | Use --stop-after s1 to validate repo access and config resolution without any model spend. |
| A scan is interrupted (network, timeout, Ctrl-C) | Re-run with --resume — completed stages are skipped via checkpoints in the state DB. |
config.yaml exists inside the target repo |
VVAH refuses it (treats the target as untrusted input) and falls back to the packaged default — this is expected, not a bug. Keep your config outside the scanned repo. |
| Remediation touches a file outside the allowed policy paths | The policy gate blocks the edit and reverts it; this is by design (see remediation_policy.yaml), not an error to work around. |
Validation returns needs_review because consensus couldn't be persisted |
Treat as fail-closed by design — always route to manual review, never treat as validated. |
| You only want fixes proposed, not applied | Use vvaharness remediate --mode report-only. |
| Batch scanning many repos | Use --repo-file <csv> instead of --repo; see runbooks/run-a-detection-scan.md §Batch mode. |
| CI/CD integration | Not shipped out of the box — wrap the CLI commands above in your pipeline's own job step; treat scan --stop-after s9 as the CI-safe (read-only) invocation. |
Metrics
| Metric | Target | How to Measure |
|---|---|---|
| Mean Time to Adapt (MTTA) | Trending down quarter over quarter | Time from scan completion (Step 3) to a validated verdict (Step 6) on a given finding, from report.md timestamps and DTO validation blocks. |
| False-positive rate at triage | < upstream baseline for the same repo class | Findings marked FALSE_POSITIVE at Step 4 ÷ total findings in report.md. |
| Verdict distribution | validated majority; needs_review reviewed within 2 business days |
Count DTOs by verdict under security-remediation/. |
| Scan cost per repo | Within estimate's projection ±20% |
Compare estimate output to actual token/cost accounting (util/tokens.py, util/metrics.py — see per-stage cost caps in config.yaml). |