Runbook: Remediate (S10) & Validate (S11)

docs / process / runbooks / remediate-and-validate

← all docs

Runbook: Remediate (S10) & Validate (S11)

Owner: Analyst / developer (repo owner) | Frequency: After each scan that surfaces findings worth fixing Last Updated: 2026-08-14 | Last Run: —

⚠️ This runbook edits code. Step 1 below writes changes into the target repo's working tree. Run it on a clean git state, on a branch, never on an uncommitted working tree you care about.

Purpose

Turn triaged findings from a prior scan into a proposed fix (S10), then adversarially grade that fix (S11) before anyone trusts it. Produces a verdict — validated / validation_failed / needs_review — per finding.

Prerequisites

Procedure

vvaharness remediate --repo /path/to/target --config config.yaml --interactive

Expected result: a menu of findings from the prior scan; select the ones to remediate. Per-finding: <repo>/security-remediation/<finding-id>/remediate_report.json + a diff. If it fails: usage: vvaharness remediate --repo <path> means --repo was omitted or mistyped — this is a required argument, not optional.

Step 1 (alternative): Remediate the top-N by CVSS, non-interactively

vvaharness remediate --repo /path/to/target --config config.yaml --top 5

Use --top all (or --top *) to remediate every remaining finding instead of a fixed count. Expected result: same DTOs/diffs as above, chosen automatically by descending CVSS.

Step 1 (alternative): Report-only — propose fixes without applying them

vvaharness remediate --repo /path/to/target --config config.yaml --mode report-only --top 5

Expected result: DTOs describe the proposed fix but the working tree is not modified — use this to review fixes before anyone commits to applying them.

Step 2: Inspect what changed before trusting it

cd /path/to/target && git status && git diff

Expected result: a diff scoped to the finding(s) you selected — nothing outside the files implicated by those findings. This is a human checkpoint, not optional. If it fails: if the diff touches files you didn't expect, do not proceed to commit — go straight to Rollback below and escalate.

Step 3: Validate the applied fix(es)

vvaharness validate --repo /path/to/target --config config.yaml --all

or validate a single finding:

vvaharness validate --repo /path/to/target --config config.yaml --finding <finding-id>

Expected result: for each finding, a verdict written back into its DTO's validation block: validated, validation_failed, or needs_review. A staged, pre-patched copy of the repo is used for comparison and is removed on completion. If it fails: --repo is required here too; check --scan-report if the combined report to enrich isn't the one you expect (defaults to the newest under security-remediation/).

Step 4: Resume a partial validation run

vvaharness validate --repo /path/to/target --config config.yaml --all --resume

Expected result: findings already validated in a prior run print their cached verdict instead of re-running the (budget-spending) adversarial panel.

Step 5: Act on the verdict

Verification

Troubleshooting

Symptom Likely Cause Fix
Remediation edit was reverted automatically Policy gate blocked a forbidden path (by design) Check remediation_policy.yaml / remediation_playbook.yaml in inputs/ — this is expected behavior, not a bug to bypass.
remediate refuses to run, cites a stale git SHA Target repo changed since the scan that produced the findings Re-scan, or pass --force only if you've confirmed the drift is safe (⚠️ overrides a safety check for a reason).
validate fails to find a --scan-report No combined report under security-remediation/ yet Run remediate first — validation enriches remediation's own report, not the raw scan output.
Validation --workspace refuses to run Path already exists and is non-empty Pass a new or empty path — VVAH treats the staged workspace as ephemeral and refuses to reuse a dirty one.
Verdict is needs_review every time Consensus synthesis couldn't be persisted (host-gate issue) This fails closed by design — do not work around it; escalate if it happens on every run, not just occasionally.
pip/import error mentioning the Claude Agent SDK validation extra not installed pip install -e ".[validation]" (see setup-and-credentials.md).

Rollback

If a remediation diff is wrong or unwanted, discard it before committing:

cd /path/to/target
git checkout -- .        # discard all uncommitted changes
# or, to discard only specific files:
git checkout -- path/to/file

Then re-run remediate with --mode report-only first next time to review before applying.

To evict a remediation/validation run's checkpoints and start clean:

vvaharness gc --run /path/to/target

Escalation

Situation Contact Method
A remediation diff looks unsafe or unexpectedly broad AppSec tool owner + repo owner Do not commit; share the diff and finding ID
needs_review verdicts happening on nearly every finding AppSec tool owner Likely a config/backend issue with the validation panel, not individual findings
Kill-switch needed (stop all remediation immediately) Operator / AppSec tool owner Disable step_remediate.enabled in config.yaml, or don't pass --remediate/leave it out of the profile

History

Date Run By Notes
2026-08-14 — Runbook authored against harness commit 3d972f67; flags verified against remediation_agent/options.py and validation/cli/_parser.py.