V1 vs V2 Engine Evolution
Relevant source files
The following files were used as context for generating this wiki page:
The transition from the V1 to the V2 regime detection engine represents a significant architectural shift from a deterministic, rule-based system to a hybrid model incorporating probabilistic machine learning and expanded macroeconomic data seams. While V1 focused on core market technicals like trend and volatility, V2 introduces layers for Hidden Markov Models (HMM), Gaussian Mixture Model (GMM) clustering, and Bayesian Online Change Point Detection (BOCPD) to provide deeper structural insight into market environments.
The evolution is managed through strict "Shadow A/B" gates to ensure that V1 core outputs (the "V1 Wire") remain identical during the transition, preventing regressions in existing trading strategies while enabling new V2-specific enrichments.
Sources: scripts/run_v2_shadow_ab_gate.py:11-20, TECH_DEBT_AUDIT.md:38-42
Core Architectural Differences
The engine evolution follows a layered approach where V2 components are "lit" (activated) as additional inputs become available, while maintaining a fallback mechanism to V1 behavior.
Data Flow and Component Activation
V1 architecture is primarily driven by OHLCV (Open, High, Low, Close, Volume) data for major indices like SPY. V2 expands this to include sector ETFs, cross-asset symbols, central bank text analysis, and point-in-time (PIT) constituent data.
The diagram illustrates how V2 inputs enrich the Feature Store and eventually inform the Transition Risk decision layer.
Sources: scripts/run_v2_calibration.py:169-210, scripts/profile_engine.py:465-485
Component Comparison Table
| Feature Area | V1 Implementation | V2 Implementation |
|---|---|---|
| Statistical Modeling | Deterministic Rules | HMM (Gaussian) & GMM Clustering |
| Change Detection | Trend Slopes | BOCPD (Bayesian Online Change Point) |
| Sentiment | Event Calendar | Central Bank Text Deterministic Lexicon |
| Breadth | ETF Proxies | PIT Constituent Analysis |
| Decision Logic | Rule Overrides | Weighted Component Scores + Rules |
Sources: scripts/run_v2_calibration.py:11-30, scripts/run_v2_walkforward_gate.py:73-100, src/regime_detection/models.py:494-520
V1 Wire Integrity and V2 Enrichments
A critical requirement of the V2 evolution is the preservation of the "V1 Wire." The V1 Wire refers to specific output fields that downstream systems rely upon for deterministic replay.
The V1 Frozen Replay
The following fields must remain byte-identical between V1 and V2 modes to avoid regressions:
trend_directiontrend_charactervolatility_statebreadth_statetransition_risk_state
V2 Enrichment Fields
V2 introduces new "wins" that are expected to be populated only when V2 seams are active:
- Transition Risk Score: A continuous value between 0.0 and 1.0 representing risk intensity.
- Agent Routing: Cohort-based routing for strategy permissions (e.g.,
active_cohort). - Structural Clusters: Assignments to probabilistic economic regimes (GMM/HMM).
- Network Fragility: Layer 3 cross-asset correlation analysis.
Sources: scripts/run_v2_shadow_ab_gate.py:53-70, src/regime_detection/models.py:545-560
Probabilistic Modeling in V2
V2 introduces two major probabilistic models that require manual "mapping" before they are considered operational.
HMM and GMM Calibration
Unlike V1's static thresholds, V2 models must be fit to historical data (e.g., 2018 to present). The calibration process produces "Candidate" artifacts that require human review to map mathematical states to economic labels.
Sequence of V2 model calibration requiring human-in-the-loop validation.
Sources: scripts/run_v2_calibration.py:15-30, scripts/run_v2_calibration.py:116-160
Validation Gates
The evolution is guarded by three primary gate types used to verify the transition:
- Shadow A/B Gate: Classifies the most recent 60 sessions in both V1 and V2 modes. Disagreements in V1 fields are flagged as P1 regressions.
- Performance Gate: Measures wire-level lit-vs-unlit activation rates (e.g., how often
change_point >= 0.5fires). - Lexicon Validation: Tests the deterministic hawkish/dovish lexicon against labeled corpora (e.g., Shah, Paturi & Chava 2023).
Gate Logic Example
# From scripts/run_v2_shadow_ab_gate.py
# V1 fields whose active_label MUST stay identical
V1_FIELDS = [
"trend_direction",
"trend_character",
"volatility_state",
"breadth_state",
"transition_risk_state",
]
Sources: scripts/run_v2_shadow_ab_gate.py:53-62, scripts/run_v2_walkforward_gate.py:11-20
Conclusion
V1 vs V2 Engine Evolution represents the maturity of the regime detection system from a pure technical indicator suite to a multi-modal macroeconomic analysis platform. By enforcing a "Frozen Replay" on V1 outputs while enabling rich V2 metadata, the project ensures continuous operational stability during the adoption of more advanced statistical methodologies.