Result Matrix Testing

Relevant source files

The following files were used as context for generating this wiki page:

• universal/result.py
• universal/algo.py
• universal/tools.py
• tests/test_algos.py
• universal/algos/crp.py
• universal/algos/mpt.py

Result Matrix Testing in the universal-portfolios project encompasses the validation and analysis of portfolio selection algorithms through structured performance metrics and comparative testing. It utilizes the AlgoResult and ListResult classes to encapsulate strategy outputs, enabling rigorous assessment of wealth growth, risk-adjusted returns (Sharpe ratio), drawdowns, and turnover across various asset universes.

The framework is designed to verify that online portfolio selection (OLPS) algorithms—such as Constant Rebalanced Portfolios (CRP) or Modern Portfolio Theory (MPT)—behave consistently when subjected to different price relatives and transaction cost structures. Testing ensures that the mathematical updates in algorithms like PAMR, OLMAR, or CWMR translate correctly into expected portfolio weights and subsequent financial performance.
Sources: universal/result.py:24-40, universal/algo.py:91-105, tests/test_algos.py:17-29

Core Components for Result Analysis

AlgoResult Class

The AlgoResult class is the primary data structure for capturing the performance of a single algorithm run. It processes price relatives ($X$) and weights ($B$) to calculate logs, fees, and risk-free adjustments.

Performance Metrics Calculation

Metrics are derived through a _recalculate internal method which handles transaction costs and rebalancing logic. Turnover is calculated by evaluating the difference between current weights and the weights held after price changes.
Sources: universal/result.py:84-112, universal/result.py:270-275

Metric	Calculation / Description	Source
Sharpe Ratio	Annualized ratio of excess return to volatility.	result.py:157-160
Profit Factor	Ratio of total log gains to total log losses.	result.py:150-155
Max Drawdown	Highest peak-to-trough decline in percentage.	result.py:265-268
Turnover	Sum of absolute weight changes normalized by year.	result.py:284-289
Information Ratio	Excess return over benchmark (UCRP) divided by tracking error.	result.py:209-220

Algorithmic Validation Workflows

Sequential Testing (Step-by-Step)

Algorithms are validated by simulating a time-series loop where the step function predicts weights for time $t+1$ based on history up to time $t$. This prevents look-ahead bias during the testing phase.

Multi-Subset Testing

The run_subsets method allows testing an algorithm across all possible combinations of $r$ stocks within a dataset. This is critical for assessing the robustness of a strategy across different asset clusters.
Sources: universal/algo.py:179-206

Visualization and Comparative Analysis

ListResult and Multi-Strategy Comparison

ListResult acts as a container for multiple AlgoResult instances, facilitating the comparison of different algorithms or parameter sets (e.g., comparing CRP vs BAH).

Transaction Cost Impact Testing

Transaction costs are modeled as a percentage fee on rebalancing. Tests verify that strategies with high turnover (like TCO1 or aggressive mean reversion) are penalized appropriately compared to low-turnover strategies like BAH.
Sources: universal/result.py:69-82, tests/test_algos.py:32-40

# tests/test_algos.py:32-40
def test_bah(S):
    """There should be no fees for BAH."""
    FEE = 0.01
    result = algos.BAH().run(S)
    wealth_no_fees = result.total_wealth
    result.fee = FEE
    wealth_with_fees = result.total_wealth

    assert abs(wealth_no_fees - wealth_with_fees) < EPS

Advanced Optimization Testing

Testing includes validating quadratic programming solvers (via cvxopt) for algorithms like MPT and Kelly. These tests ensure that weight constraints (long-only, max leverage) are respected during the optimization step.
Sources: universal/algos/mpt.py:270-300, universal/tools.py:228-265

Risk-Free Rate and CAPM Benchmarking

The Result Matrix framework integrates risk-free rates into calculations for Sharpe and Appraisal ratios. It utilizes tools.capm to decompose strategy returns into Alpha and Beta components relative to a benchmark.
Sources: universal/result.py:168-176, universal/tools.py:651-678

Summary: Result Matrix Testing provides the necessary infrastructure to quantify the performance of online portfolio selection strategies. By encapsulating metrics in AlgoResult and providing tools for multi-subset and benchmark comparison, the system ensures that algorithmic implementations are both mathematically sound and practically viable under transaction costs.