Runbook: Patch & Verify ⚠️ (this runbook edits code)
Owner: Security engineer | Frequency: After Step 3 (Triage) of the SOP produces ranked findings | Last Updated: 2026-08-14 | Last Run: —
Verified against ../../../samples/defending-code-reference-harness/
(.claude/skills/patch/, harness/cli.py's patch subparser, harness/patch_grade.py).
⚠️ This is the runbook that writes to disk.
/patchandvuln-pipeline patchgenerate diffs; in fix mode they can apply them. Review every generated diff before it reaches a real codebase — target-derived text is wrapped in<untrusted_data>at generation time, which is a mitigation against prompt injection, not a guarantee the diff is correct or safe.
Purpose
Draft a candidate fix for a verified finding (static TRIAGE.json or a pipeline
results/ directory) and put it through an adversarial grading loop — rebuild, re-run the
original PoC, run the test suite, and (unless skipped) a fresh find-agent re-attack —
before it's presented as trustworthy. This is Step 4 of
process-operate-dcrh.md.
Prerequisites
- [ ] A ranked
TRIAGE.json(static path) or aresults/<target>/<timestamp>/directory with at least one accepted bug inreports/(pipeline path) - [ ] For the pipeline path: sandbox set up per setup-and-sandbox.md
- [ ] A human available to review the resulting diff before it's applied upstream
Procedure
Step 1: Patch from static triage
claude
> /patch ./TRIAGE.json --repo targets/canary
Expected result: PATCHES/ directory with a candidate fix per verified finding.
If it fails: N/A — read/write-file only, safe to run unsandboxed; review the plan
before approving any file writes if running interactively.
Step 2: Patch from a pipeline run — all bugs
cd samples/defending-code-reference-harness
bin/vp-sandboxed patch results/drlibs/<timestamp>/ --model <model-id>
Expected result: For each unique bug in reports/, a fix is drafted and run through
the grade loop (default cap: 5 fix↔grade iterations, --max-iterations to change). Passing
patches land as a verified diff under the run's results directory.
If it fails: See Troubleshooting.
Step 3 (faster iteration): Patch one bug, skip the re-attack tier
bin/vp-sandboxed patch results/drlibs/<timestamp>/ --model <model-id> --bug 3 --no-reattack
Use this for quick iteration while developing a fix strategy — not for the fix you
intend to ship. --no-reattack skips the tier that tries to find a way around the patch,
which is the strongest signal that a fix actually closes the hole.
Expected result: Faster turnaround, T0–T2 verification only (build + original PoC +
test suite — no fresh find-agent re-attack).
If it fails: See Troubleshooting.
Step 4: Add the advisory style judge (optional)
bin/vp-sandboxed patch results/drlibs/<timestamp>/ --model <model-id> --style
Expected result: An additional, advisory judgment on whether the fix matches the target's existing code style — informational, does not affect the pass/fail verdict. If it fails: N/A.
Step 5: Human review — before applying anything upstream
# read the diff directly
cat results/drlibs/<timestamp>/reports/bug_NN/patch.diff # or wherever the diff lands — check step 2's output path
Expected result: You have personally read the diff, understand what it changes, and confirmed it does not touch anything outside the intended fix (no unrelated files, no suspicious additions). If it fails: Do not apply the diff. Escalate per below.
Verification
- [ ]
PatchVerdict(or the pipeline's equivalent per-bug record) shows a pass, not just "fix drafted" - [ ] The build succeeds with the patch applied
- [ ] The original PoC/crashing input no longer reproduces
- [ ] The target's existing test suite still passes
- [ ] (Unless
--no-reattackwas used) a fresh find agent could not route around the fix - [ ] A human has read the diff end to end
Troubleshooting
| Symptom | Likely Cause | Fix |
|---|---|---|
Grade loop exhausts --max-iterations without passing |
The bug may need a more structural fix than the playbook-style approach can draft in N tries | Escalate to the target's engineering owner for a manual fix; keep the failed attempts as documentation of what didn't work |
| Patch builds but the test suite fails | Fix introduced a regression | Do not apply; the grade loop should have caught and iterated on this — if it passed anyway, treat as a harness bug and report upstream |
| Re-attack tier finds a new crash after the "fix" | The original fix was incomplete | Expected outcome of the tier working correctly — treat the new crash as a fresh finding, back to Step 2 (scan) on the narrowed area |
| Diff touches files outside the expected scope | Possible prompt-injection influence from target-derived text, or an overly broad fix | Do not apply. Re-run with --no-reattack removed and review prompts/logs; escalate if the cause isn't obvious |
Rollback
If a patch was applied to the target's own repo (not just written to PATCHES/ or the
pipeline's results directory) and needs to be undone: standard git revert on the commit
that applied it, or git checkout -- <file> before committing. DCRH itself never commits
on your behalf — applying and committing a patch is always a separate, human-initiated step.
Escalation
| Situation | Contact | Method |
|---|---|---|
| Diff looks suspicious (scope creep, unexplained changes) | Security engineering lead | Immediate — do not apply, do not merge |
| Fix can't pass the grade loop after reasonable iteration | Target's engineering owner | Standard handoff — treat as a manually-fixed finding |
| Suspected prompt injection via target-derived content | Security engineering lead | Immediate — preserve the transcript for review |
History
| Date | Run By | Notes |
|---|---|---|
| — | — | — |