Runbook: Remediate (S10) & Validate (S11)
Owner: Analyst / developer (repo owner) | Frequency: After each scan that surfaces findings worth fixing Last Updated: 2026-08-14 | Last Run: —
⚠️ This runbook edits code. Step 1 below writes changes into the target repo's working tree. Run it on a clean git state, on a branch, never on an uncommitted working tree you care about.
Purpose
Turn triaged findings from a prior scan into a proposed fix (S10), then adversarially
grade that fix (S11) before anyone trusts it. Produces a verdict —
validated / validation_failed / needs_review — per finding.
Prerequisites
- [ ] A completed scan for the target repo (
security-scan/report.mdexists — seerun-a-detection-scan.md) - [ ] Target repo's git working tree is clean (
git statusshows nothing to commit) — remediation will write to this working tree - [ ]
pip install -e ".[validation]"done (S11 requires the Claude Agent SDK extra) - [ ] You know which findings to act on (finding IDs from
report.md, or use--top N)
Procedure
Step 1: Remediate — interactive selection (recommended first time)
vvaharness remediate --repo /path/to/target --config config.yaml --interactive
Expected result: a menu of findings from the prior scan; select the ones to remediate.
Per-finding: <repo>/security-remediation/<finding-id>/remediate_report.json + a diff.
If it fails: usage: vvaharness remediate --repo <path> means --repo was omitted or
mistyped — this is a required argument, not optional.
Step 1 (alternative): Remediate the top-N by CVSS, non-interactively
vvaharness remediate --repo /path/to/target --config config.yaml --top 5
Use --top all (or --top *) to remediate every remaining finding instead of a fixed count.
Expected result: same DTOs/diffs as above, chosen automatically by descending CVSS.
Step 1 (alternative): Report-only — propose fixes without applying them
vvaharness remediate --repo /path/to/target --config config.yaml --mode report-only --top 5
Expected result: DTOs describe the proposed fix but the working tree is not modified — use this to review fixes before anyone commits to applying them.
Step 2: Inspect what changed before trusting it
cd /path/to/target && git status && git diff
Expected result: a diff scoped to the finding(s) you selected — nothing outside the files implicated by those findings. This is a human checkpoint, not optional. If it fails: if the diff touches files you didn't expect, do not proceed to commit — go straight to Rollback below and escalate.
Step 3: Validate the applied fix(es)
vvaharness validate --repo /path/to/target --config config.yaml --all
or validate a single finding:
vvaharness validate --repo /path/to/target --config config.yaml --finding <finding-id>
Expected result: for each finding, a verdict written back into its DTO's validation
block: validated, validation_failed, or needs_review. A staged, pre-patched copy of
the repo is used for comparison and is removed on completion.
If it fails: --repo is required here too; check --scan-report if the combined
report to enrich isn't the one you expect (defaults to the newest under security-remediation/).
Step 4: Resume a partial validation run
vvaharness validate --repo /path/to/target --config config.yaml --all --resume
Expected result: findings already validated in a prior run print their cached verdict instead of re-running the (budget-spending) adversarial panel.
Step 5: Act on the verdict
validated→ proceed to normal code review / commit / PR.needs_review→ always route to a human; this status means consensus could not be reliably established (fail-closed by design) — never treat it as equivalent tovalidated.validation_failed→ discard the fix, return the finding to backlog, or have a developer fix it manually; re-runremediateonly after understanding why it failed.
Verification
- [ ] Every selected finding's DTO under
security-remediation/has a non-emptyvalidationblock - [ ]
git diffin the target matches onlyvalidatedfixes before you commit - [ ] No
needs_reviewfinding is merged without a human sign-off recorded somewhere (ticket, PR review)
Troubleshooting
| Symptom | Likely Cause | Fix |
|---|---|---|
| Remediation edit was reverted automatically | Policy gate blocked a forbidden path (by design) | Check remediation_policy.yaml / remediation_playbook.yaml in inputs/ — this is expected behavior, not a bug to bypass. |
remediate refuses to run, cites a stale git SHA |
Target repo changed since the scan that produced the findings | Re-scan, or pass --force only if you've confirmed the drift is safe (⚠️ overrides a safety check for a reason). |
validate fails to find a --scan-report |
No combined report under security-remediation/ yet |
Run remediate first — validation enriches remediation's own report, not the raw scan output. |
Validation --workspace refuses to run |
Path already exists and is non-empty | Pass a new or empty path — VVAH treats the staged workspace as ephemeral and refuses to reuse a dirty one. |
Verdict is needs_review every time |
Consensus synthesis couldn't be persisted (host-gate issue) | This fails closed by design — do not work around it; escalate if it happens on every run, not just occasionally. |
pip/import error mentioning the Claude Agent SDK |
validation extra not installed |
pip install -e ".[validation]" (see setup-and-credentials.md). |
Rollback
If a remediation diff is wrong or unwanted, discard it before committing:
cd /path/to/target
git checkout -- . # discard all uncommitted changes
# or, to discard only specific files:
git checkout -- path/to/file
Then re-run remediate with --mode report-only first next time to review before applying.
To evict a remediation/validation run's checkpoints and start clean:
vvaharness gc --run /path/to/target
Escalation
| Situation | Contact | Method |
|---|---|---|
| A remediation diff looks unsafe or unexpectedly broad | AppSec tool owner + repo owner | Do not commit; share the diff and finding ID |
needs_review verdicts happening on nearly every finding |
AppSec tool owner | Likely a config/backend issue with the validation panel, not individual findings |
| Kill-switch needed (stop all remediation immediately) | Operator / AppSec tool owner | Disable step_remediate.enabled in config.yaml, or don't pass --remediate/leave it out of the profile |
History
| Date | Run By | Notes |
|---|---|---|
| 2026-08-14 | — | Runbook authored against harness commit 3d972f67; flags verified against remediation_agent/options.py and validation/cli/_parser.py. |