CI/CD & Validation Skill
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The CI/CD & Validation Skill represents the systematic approach used in the regime-detection repository to ensure code quality, operational discipline, and data integrity. It encompasses a multi-layered verification strategy that includes automated linting, strict type checking, comprehensive testing suites, and specialized "gate" scripts designed to prevent regressions in financial regime classification. This system enforces non-negotiable rules for both human developers and coding agents to maintain deterministic replay and prevent future-data leakage.
Sources: AGENTS.md:1-20, TECH_DEBT_AUDIT.md:10-25
Automated Verification Pipeline
The project utilizes GitHub Actions to execute a split verification pipeline. This pipeline is divided into fast checks for Pull Requests (PRs) and a full verification suite for the main branch.
Pipeline Structure
- PR-Fast CI: Executes core checks including
black,ruff,pyright(on restricted scopes), and a subset ofpytestthat excludes slow markers. - Full Verification: Runs the complete test suite, including long-running integration tests and exhaustive coverage reports.
- Release Workflow: Ensures all test markers pass before a release is finalized.
Sources: tests/test_readiness_contracts.py:25-50, scripts/validate_agents_md.py:10-25
The diagram shows the tiered execution of validation tasks based on the development lifecycle stage.
Validation Components & Guardrails
The repository employs several specialized scripts and configurations to maintain "readiness contracts."
Canonical Automation Commands
These commands are the single source of truth for repository hygiene and are validated for consistency across AGENTS.md and CI workflow files.
| Command | Purpose |
|---|---|
python -m black --check src tests scripts | Enforces code formatting |
python -m ruff check . | Executes fast linting and code analysis |
python -m pyright | Performs static type checking |
python -m pytest --cov=src --cov-fail-under=80 | Validates logic and ensures code coverage |
Sources: scripts/validate_agents_md.py:10-20, AGENTS.md:105-115
Shadow A/B Gating
A critical validation component for V2 features is the Shadow A/B Gate. This script (scripts/run_v2_shadow_ab_gate.py) simulates live classification over a 60-session window. It compares V1-mode outputs against V2-mode outputs to ensure zero unexpected regressions in legacy fields while confirming new V2 field activations.
Logic flow for the Shadow A/B gate runner used to validate V2 deployments.
Data Integrity & Artifact Validation
Data validation is handled via canonical snapshots and manifest management. The system ensures that data used for regime detection is traceable and byte-identical across environments.
Canonicalization Process
The script publish_canonical_snapshot.py handles artifact drift by recomputing SHA256 hashes and row counts. It strips non-deterministic metadata from Parquet files and ensures rows are sorted, creating a "fixed point" where canonicalize(x) == x.
Sources: scripts/publish_canonical_snapshot.py:15-35, scripts/publish_canonical_snapshot.py:115-130
Key Validation Checks
- SHA256 Verification: Artifacts must match the manifest's digest.
- Calendar Coverage: Daily OHLCV artifacts are checked for gaps against the NYSE calendar.
- Semantic Integrity: Validates required columns (Open, High, Low, Close, Volume, Adjusted Close).
Sources: scripts/publish_canonical_snapshot.py:170-200, tests/test_materialize_regime_data.py:80-100
Operational Discipline
The repository enforces a "Strict Agent" policy defined in AGENTS.md. This includes non-negotiable rules for every unit of work:
- Search Before Writing: Grep for existing concepts before creating functions >15 lines.
- Task Boundaries: Stop and commit at every task boundary; no auto-chaining.
- Validation Layering: Validate at every boundary (e.g., source-specific models before normalization).
- Failure Modes: Follow the "Delete-before-add" bias and "Three-strikes rule" for refactoring.
Sources: AGENTS.md:25-60
Conclusion
The CI/CD & Validation Skill in this repository functions as a comprehensive guardrail system. By combining strict operational discipline with automated A/B testing and rigorous data canonicalization, the project ensures that the regime detection engine remains deterministic and reliable. Future improvements focus on expanding pyright coverage and hardening typed evidence models to further reduce contract drift.
Sources: TECH_DEBT_AUDIT.md:15-30