Testing & Verification
Relevant source files
The following files were used as context for generating this wiki page:
Testing and verification within the Rabid Raccoon project encompass a multi-layered approach to ensure the integrity of market data, the accuracy of trading signals, and the reliability of the underlying financial computational stack. This system ranges from low-level smoke tests of Python financial libraries to sophisticated backtesting of deterministic trading signals and strict validation rules for data ingestion.
The verification framework is designed to prevent "silent data corruption," a critical risk in high-frequency MES futures trading. By combining automated script-based checks with strict architectural constraints defined in the project configuration, the system ensures that every component—from data ingestion to AI-assisted analysis—operates within defined technical parameters.
Financial Stack Verification
The project utilizes a dedicated Python-based smoke-testing utility to verify the availability and correctness of critical financial libraries. This ensures that the environment executing complex calculations has the necessary tools installed and that they return expected mathematical results.
The diagram above illustrates the sequential execution of financial library smoke tests to ensure computational environment readiness.
Key Verification Metrics
The verification script checks specific mathematical outputs to confirm library integrity:
| Library | Verification Check | Expected Result Type |
|---|---|---|
| SciPy | norm.cdf(0.0) | float (0.5) |
| Statsmodels | OLS R-squared | float |
| QuantLib | Business day weekday | integer |
| Pyfolio | Performance stats row count | integer |
Sources: [scripts/verify-finance-stack.py:20-67](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/scripts/verify-finance-stack.py#L20-L67)
Backtesting and Signal Validation
The backtest-signals.ts script provides a deterministic framework for verifying the efficacy of trading signals. It simulates trades based on historical data to calculate hit rates and strategy returns, providing a quantitative baseline for signal reliability.
Backtesting Logic
The backtesting engine follows a specific data flow to evaluate signal performance:
- Data Preparation: Fetches raw candles for symbols like MES, NQ, GC, and CL, then aggregates them into 15-minute periods.
- Warmup: Processes a defined number of "warmup" candles (default: 220) to populate technical indicators before recording observations.
- Signal Generation: Executes the
computeSignalsfunction to determine a "BUY" or "SELL" bias based on signal math. - Evaluation: Compares the signal direction against the actual price movement of the subsequent candle to calculate "hit" status and strategy returns.
Sources: [scripts/backtest-signals.ts:74-114](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/scripts/backtest-signals.ts#L74-L114)
Sequence of operations during the backtesting of deterministic signals.
Data Integrity and Surgical Rules
The project enforces strict validation rules for market data through cubic.yaml to prevent corruption of the trading signal chain. These rules act as a continuous verification layer during development and ingestion.
Critical Validation Rules
- OHLCV Integrity: Every candle insertion must validate that
open,high,low, andcloseare greater than zero, and thathigh >= low,high >= open, andhigh >= close. - Volume Handling: MES volume must always use
BigIntto avoid precision loss fromNumber.MAX_SAFE_INTEGERlimitations. - Append-Only Enforcement: Time-series tables are strictly append-only;
DELETEorUPDATEoperations are flagged as indicators of data corruption. - Deduplication: Insertion logic must use
rowHashcombined withskipDuplicatesto prevent Inngest retries from creating ghost rows.
Sources: [cubic.yaml:61-105](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/cubic.yaml#L61-L105), [cubic.yaml:137-147](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/cubic.yaml#L137-L147)
API and Integration Testing
Verification also extends to external data providers like FRED (Federal Reserve Economic Data). Scripts such as test-fred-releases.ts are used to verify API connectivity and the availability of specific economic release dates.
FRED Connectivity Checks
The testing script validates:
- API Response Status: Ensures the FRED API returns HTTP 200.
- Release ID Mapping: Verifies that specific IDs (e.g., rid=27 for Housing Starts) return valid date arrays.
- Parameter Sensitivity: Tests how the API handles
realtime_startand other query parameters to ensure stable data ingestion.
Sources: [scripts/test-fred-releases.ts:5-46](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/scripts/test-fred-releases.ts#L5-L46)
Verification of Technical Indicators
The instant-analysis.ts and build-lean-dataset.ts modules include internal validation for technical indicator calculations. Before signals are finalized, the system verifies data availability:
- RSI/SMA/EMA: Requires a minimum period of data points; otherwise, it returns
nullto avoid biased signals. - Fibonacci Structure: Validates swing points to ensure anchor levels (High/Low) are established before projecting targets.
- Dataset Consistency: During dataset building, the system performs a "Sanity Check" to ensure the row length matches the header column count exactly before writing to disk.
Sources: [src/lib/instant-analysis.ts:101-150](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/src/lib/instant-analysis.ts#L101-L150), [scripts/build-lean-dataset.ts:285-288](https://github.com/zincdigitalofmiami/rabid-raccoon/blob/HEAD/scripts/build-lean-dataset.ts#L285-L288)
The Testing & Verification systems ensure that Rabid Raccoon maintains a high-fidelity data environment, protecting the automated trading logic from flawed inputs or inconsistent computational states.