Module: vvaharness/orchestrator

docs / modules / orchestrator

← all docs

Module: vvaharness/orchestrator

Role. The shift manager: drives single-repo and batch runs, checks backends before spending money, checkpoints progress, and owns the SQLite state store.

Size. 11 files · ~3,341 LOC.


flowchart LR IN["Input\nCLI command · resolved Config"] --> ACT["Action\nRun each pipeline stage in sequence, checkpointing as it goes"] ACT --> VAL["Validation\nPreflight backend/credential probe — before any budget is spent"] VAL --> OUT["Output\nSequenced stages · SQLite checkpoints · final reports"]

The orchestrator runs the show. It prepares the workspace, verifies your model backends are reachable before any tokens are spent (preflight), then walks the pipeline stage by stage, saving a checkpoint after each so an interrupted run can resume. For batch jobs it clones many repos from a CSV and produces one report per application. It also loads CMDB data and persists the call graph so later stages don't recompute it.

Explain-it-simply. Think of it as the factory shift manager: clock everyone in, check the machines work, run the line, and keep a logbook so work can pause and resume.

Data-flow (this module)

flowchart LR CLI[cli.py] --> ENTRY[entry.py main] ENTRY --> PRE[preflight.py<br/>configure + probe backends] PRE --> SCAN[scan.py scan_repo] BATCH[batch.py] -->|per repo| SCAN SCAN -->|per stage| CKPT[checkpoints.py] SCAN <--> STORE[(store.py<br/>SQLite state + callgraph)] SCAN --> REM[remediation_agent] SCAN --> VAL[validation]

Technical details

flowchart LR IN["Input\nCLI command · resolved Config"] --> ACT["Action\nRun each pipeline stage in sequence, checkpointing as it goes"] ACT --> VAL["Validation\nPreflight backend/credential probe — before any budget is spent"] VAL --> OUT["Output\nSequenced stages · SQLite checkpoints · final reports"]

entry.py is the argparse main. preflight.py configures backends, probes agentic roles, and reports credential gaps (a missing post-scan key is a WARN, not fatal). scan.py (scan_repo) is the single-repo driver that sequences S0–S9 and then _run_remediation / _run_validation with their preflights. batch.py clones + groups by AppId, assigns per-finding slugs, and writes batch_summary.md. store.py is the SQLite state store (call-graph persistence via save_callgraph/load_callgraph); checkpoints.py handles per-stage save/load/prune with schema validation. Supporting: cmdb.py (app-profile load), cleanup.py (clone purge), config_paths.py (path confinement + role iteration), enrich_findings.py (path-prefixing for batch).

File map (quick scan)

File LOC Purpose
orchestrator/__init__.py 93 —
orchestrator/batch.py 689 —
orchestrator/checkpoints.py 255 —
orchestrator/cleanup.py 80 —
orchestrator/cmdb.py 69 —
orchestrator/config_paths.py 89 —
orchestrator/enrich_findings.py 60 —
orchestrator/entry.py 273 —
orchestrator/preflight.py 477 —
orchestrator/scan.py 707 —
orchestrator/store.py 549 orchestrator.store — SQLite state store.

Files & functions, explained

Every class and function below has a one-line plain-English note — readable whether you write code or not. _name marks an internal helper.

orchestrator/__init__.py

93 LOC. Utility / support module.

No top-level classes or functions (package init / constants).

orchestrator/batch.py

689 LOC. Utility / support module.

Functions - _module_name_from() — (internal helper) Provides the “module name from” logic. - _is_remote() — (internal helper) checks / decides the remote value. - _scrub_url_secrets() — (internal helper) hides sensitive data in the url secrets step. - _md_cell() — (internal helper) Provides the “Markdown report cell” logic. - _stash_url_for() — (internal helper) Provides the “temporary store url for” logic. - _state_root() — (internal helper) Provides the “state root” logic. - _ref_id() — (internal helper) Provides the “ref identifier” logic. - _stage_marker_path() — (internal helper) builds / writes the marker path step. - _write_stage_marker() — (internal helper) builds / writes the pipeline step marker step. - _stage_dir_bound() — (internal helper) builds / writes the dir bound step. - _assign_slugs() — (internal helper) Provides the “assign slugs” logic. - _check_manifest_size() — (internal helper) checks / decides the run record size step. - _parse_repo_csv() — (internal helper) reads and cleans up the code repository csv step. - _parse_repo_file() — (internal helper) reads and cleans up the code repository file step. - _with_token() — (internal helper) Provides the “with token” logic. - _acquire_repo() — (internal helper) Provides the “acquire code repository” logic. - run_batch() — Runs the batch value. - _stage_repo() — (internal helper) builds / writes the code repository value. - _app_module_name() — (internal helper) Provides the “app module name” logic. - _run_batch_grouped() — (internal helper) runs the batch grouped step. - _render_batch_summary() — (internal helper) builds / writes the batch summary step.

orchestrator/checkpoints.py

255 LOC. Utility / support module.

Classes - _S4Ckpt — (internal) Represents the “S4Ckpt” data/behaviour used in this file.

Functions - _validate_schema() — (internal helper) checks / decides the schema value. - _schema_for() — (internal helper) Provides the “schema for” logic. - save_ckpt() — Builds / writes the saved checkpoint value. - load_ckpt() — Looks up / reads the saved checkpoint value. - run_id_for() — Runs the identifier for step. - prune_checkpoints() — Sets up / maintains the checkpoints value.

orchestrator/cleanup.py

80 LOC. Utility / support module.

Functions - _rmtree_rw() — (internal helper) Provides the “rmtree rw” logic. - _preserve_set() — (internal helper) Provides the “preserve set” logic. - _purge_clone() — (internal helper) sets up / maintains the clone value.

orchestrator/cmdb.py

69 LOC. Utility / support module.

Functions - _cmdb_path() — (internal helper) Provides the “asset inventory path” logic. - _set_cmdb_path() — (internal helper) Provides the “set asset inventory path” logic. - _load_app_profile() — (internal helper) looks up / reads the app profile step.

orchestrator/config_paths.py

89 LOC. Utility / support module.

Functions - _app_root() — (internal helper) Provides the “app root” logic. - _default_config() — (internal helper) Provides the “default config” logic. - _packaged_default() — (internal helper) Provides the “packaged default” logic. - _path_within() — (internal helper) Provides the “path within” logic. - _resolve_against() — (internal helper) checks / decides the against value. - _iter_model_roles() — (internal helper) looks up / reads the model roles step.

orchestrator/enrich_findings.py

60 LOC. Utility / support module.

Functions - _prefix_path() — (internal helper) Provides the “prefix path” logic. - _prefix_ref() — (internal helper) Provides the “prefix ref” logic. - _enrich_findings() — (internal helper) adds detail to the candidate issues value.

orchestrator/entry.py

273 LOC. Utility / support module.

Functions - _top_arg() — (internal helper) Provides the “highest-priority arg” logic. - _resolve_auto_step1() — (internal helper) checks / decides the auto step1 step. - main() — Runs the main value.

orchestrator/preflight.py

477 LOC. Utility / support module.

Functions - _mask() — (internal helper) hides sensitive data in the mask value. - configure_backends() — Sets up / maintains the backends value. - _reachable_despite_token_cap() — (internal helper) Provides the “reachable despite token cap” logic. - _post_scan_only() — (internal helper) Provides the “post scan only” logic. - _report_credential_gap() — (internal helper) builds / writes the credential gap step. - check_backends() — Checks / decides the backends value. - _agentic_role_tools() — (internal helper) Provides the “agentic role tools” logic. - _probe_agentic_roles() — (internal helper) runs the agentic roles step. - probe_backends() — Runs the backends value.

orchestrator/scan.py

707 LOC. Utility / support module.

Functions - _head_sha() — (internal helper) Provides the “head content hash” logic. - _hydrate_ctx_callgraph_from_store() — (internal helper) reads and cleans up the repo context map of which code calls what from store step. - scan_repo() — Runs the code repository value. - _ranked_to_ra_finding() — (internal helper) checks / decides the to ra candidate issue step. - _run_remediation() — (internal helper) runs the fix value. - _run_validation() — (internal helper) runs the fix-check value. - _remediate_preflight() — (internal helper) runs the pre-run check value. - _validate_preflight() — (internal helper) checks / decides the pre-run check value.

orchestrator/store.py

549 LOC. orchestrator.store — SQLite state store.

Functions - db_path() — Provides the “db path” logic. - connect() — Sets up / maintains the connect value. - _migrate() — (internal helper) sets up / maintains the migrate value. - register_run() — Sets up / maintains the run value. - _qnode() — (internal helper) Provides the “a node in the code graph” logic. - _split_qnode() — (internal helper) reads and cleans up the a node in the code graph value. - _line_from_site() — (internal helper) Provides the “line from site” logic. - _graph_rows() — (internal helper) Provides the “code relationship map rows” logic. - save_callgraph() — Builds / writes the map of which code calls what value. - load_callgraph() — Looks up / reads the map of which code calls what value. - reset_run() — Sets up / maintains the run value. - delete_run() — Sets up / maintains the run value.


See also: HLD · LLD · Data Flow · module index.