Operational CLI Scripts
Relevant source files
The following files were used as context for generating this wiki page:
Operational CLI Scripts serve as the primary interface for managing data provenance, engine performance profiling, and model validation within the regime detection system. These scripts facilitate critical maintenance tasks such as materializing market data from manifests, calibrating V2 models (HMM/GMM), and executing shadow A/B gates to ensure parity between engine versions.
The operational layer acts as a bridge between raw data acquisition and the runtime classifier, providing tools for technical audits and performance monitoring. Key functionalities include canonicalizing Parquet datasets to ensure deterministic SHA256 signatures and generating human-readable verification reports for economic regime mappings.
Sources: scripts/profile_engine.py, scripts/publish_canonical_snapshot.py, scripts/run_v2_calibration.py
Data Lifecycle and Provenance Management
The project utilizes a manifest-driven architecture where scripts interact with ArtifactManifest objects to ensure data integrity across environments.
Canonical Snapshot Publishing
The publish_canonical_snapshot.py script manages the manifest-sha-drift gap. It ensures that any Parquet file modified under data/raw/ is rewritten into a canonical form—sorted rows, stripped metadata, and pinned compression settings—before updating the manifest's SHA256 and uploading to S3.
The canonicalization process is a fixed-point operation, ensuring that canonicalize(canonicalize(x)) == canonicalize(x). It specifically targets stability by pinning snappy compression and us timestamp coercion to avoid drift caused by different pyarrow versions.
Sources: scripts/publish_canonical_snapshot.py:20-40, scripts/publish_canonical_snapshot.py:117-140
Audit Harness and Provenance Bundling
Technical audits are supported by audit_step1_harness.py, which captures the materialization provenance of a runner invocation. It records whether a manifest was used, if CLI overrides were applied, and computes SHA256 hashes of tracked source directories (src/regime_detection and src/regime_data_fetch) to detect cross-worktree drift.
Sources: scripts/audit_step1_harness.py:48-60, scripts/audit_step1_harness.py:94-110
Model Calibration and Validation
Scripts in this category focus on the transition from V1 to V2 engine logic, specifically involving unsupervised learning models and text-based sentiment analysis.
V2 Calibration Runner
run_v2_calibration.py fits HMM (§6.1) and GMM (§6.2) clustering models on historical data. Because these models are unsupervised, the script generates "CANDIDATE" artifacts (.candidate.yaml) that require an operator to manually review centroids and assign economic labels (e.g., calm_bull, stress_crash).
| Component | Output Artifact | Purpose |
|---|---|---|
| HMM | hmm_state_label_map.candidate.yaml | Maps hidden states to economic labels based on posterior means. |
| GMM | cluster_label_map.candidate.yaml | Maps feature clusters to labels based on centroids. |
| Summary | v2_calibration_summary.md | Rollup of training windows, dimensions, and bias warnings. |
Sources: scripts/run_v2_calibration.py:12-25, scripts/run_v2_calibration.py:460-500
Central Bank Text Lexicon Validation
The validate_central_bank_text_lexicon.py script benchmarks the deterministic hawkish/dovish lexicon against the fomc_communication labeled corpus. It uses a decision matrix to determine if the bag-of-keywords approach is sufficient or if an upgrade to a transformer model (e.g., FOMC-RoBERTa) is required based on document-level accuracy thresholds.
Sources: scripts/validate_central_bank_text_lexicon.py:200-240
Engine Profiling and Shadow Testing
Operational scripts provide deep visibility into the performance and logic of the RegimeEngine.
Engine Profiling Architecture
profile_engine.py measures wall-clock time for various stages of the classify_window() method. It utilizes a StageTimer context manager to instrument internal engine calls, such as building the feature store and axis series.
The profiler is equipped with a SIGALRM budget (default 300s) to prevent runaway processes during full-history runs.
Sources: scripts/profile_engine.py:90-110, scripts/profile_engine.py:338-370
V2 Shadow A/B Gate
run_v2_shadow_ab_gate.py implements a 60-session shadow test to verify V2 logic (§9.3). It classifies the most recent sessions twice—once in V1 mode and once in V2 mode.
Gate Criteria:
- V1 Fields:
trend_direction,volatility_state, etc., MUST have zero disagreements. - V2 Fields:
change_point,cluster,credit_funding_state, etc., are EXPECTED to differ as these represent new enrichments.
Sources: scripts/run_v2_shadow_ab_gate.py:15-35, scripts/run_v2_shadow_ab_gate.py:400-430
Common Configuration and Argument Handling
Most scripts share a unified argument structure through _v2_calibration_helpers.py, ensuring consistent paths for market data, macro series, and constituent OHLCV.
| Argument | Description | Default Source |
|---|---|---|
--manifest | Path to the YAML artifact manifest. | manifests/runs/ |
--data-root | Local directory for raw Parquet storage. | data/raw |
--lookback-days | Number of sessions to process. | 30 |
--config-path | Path to the engine YAML configuration. | src/regime_detection/configs/ |
Sources: scripts/profile_engine.py:465-510, scripts/audit_layer2_30d.py:225-245
Summary
The Operational CLI Scripts are essential for maintaining the deterministic nature of the regime detection engine. By enforcing manifest-based data integrity, providing semi-automated model calibration, and implementing strict shadow gates, these tools ensure that V2 engine enhancements do not introduce regressions into the established V1 wire contract.