Utility Tools
Relevant source files
The following files were used as context for generating this wiki page:
Utility Tools encompass a broad suite of helper functions, mathematical operations, data processing utilities, and visualization modules designed to support Online Portfolio Selection (OLPS) algorithms. These tools facilitate everything from synthetic data generation and financial metric calculation to performance visualization and algorithm profiling.
The scope of these tools extends across the entire lifecycle of a portfolio strategy, including data preparation (cleaning and proxy filling), optimization (simplex projections and Markowitz frameworks), and post-run analysis (Sharpe ratio, drawdown, and turnover metrics).
Sources: README.md:10-14, universal/tools.py:1-25
Data Management and Synthetic Generation
The toolkit provides sophisticated methods for handling financial time series, particularly focusing on the creation of synthetic benchmarks and filling missing data gaps using correlations between assets.
Synthetic Data Filling
Functions like fill_synthetic_data allow the system to reconstruct missing history for ETFs by leveraging correlated proxies (e.g., using TLT history to fill UBT data). This process involves calculating returns, determining correlation thresholds, and applying regression or standard deviation scaling to fill gaps.
Sources: universal/tools.py:465-515
Data Generation Flow
The following diagram illustrates the process of generating random portfolio data using Multivariate Geometric Brownian Motion:
The random_portfolio function implements this flow to provide testable data for algorithms.
Sources: universal/tools.py:73-112
Portfolio Optimization Helpers
A core component of the utility suite is the implementation of mathematical optimization routines required by OLPS algorithms, such as simplex projections and Markowitz efficient frontier calculations.
Key Optimization Functions
| Function | Purpose | Parameters |
|---|---|---|
simplex_proj | Projects a vector onto the probability simplex (weights sum to 1, all $\ge$ 0). | y: Input vector. |
opt_markowitz | Calculates optimal weights in a Markowitz framework. | mu, sigma, long_only, q. |
opt_weights | Finds best constant rebalanced portfolio (CRP) for a specific metric. | X, metric, max_leverage. |
simplex_mesh | Creates a uniform grid of points on a simplex for evaluation. | d (dimensions), points. |
Sources: universal/tools.py:186-224, universal/tools.py:284-307
Simplex Projection Logic
The simplex_proj function is critical for algorithms like PAMR and RPRT to ensure weight vectors remain valid portfolio distributions. It involves sorting the input vector and finding a threshold tmax to subtract from all elements.
Sources: universal/tools.py:284-307, universal/algos/pamr.py:73
Performance Metrics and Analysis
Analysis tools are primarily housed in universal/result.py and supported by mathematical functions in universal/tools.py. These tools compute the efficacy of an algorithm's weights against price relatives.
Risk and Return Metrics
- Sharpe Ratio: Annualized excess return over volatility, with support for standard deviation error estimates.
- Ulcer Index: A measure of "stress" based on the depth and duration of drawdowns.
- Information Ratio: Active return of the portfolio divided by the tracking error relative to a benchmark.
- Turnover: Measures the amount of trading required to maintain the portfolio weights.
Sources: universal/result.py:126-170, universal/tools.py:400-440
Performance Evaluation Sequence
The following sequence diagram shows how an algorithm's output is processed into an AlgoResult object:
Visualization and Ternary Plotting
The project includes specialized plotting utilities for financial data, including equity curves, weight distribution area charts, and ternary plots for 3-asset portfolios.
Ternary Plotting Logic
The universal/algos/ternary/plotting.py module allows for the visualization of trajectories and heatmaps on a 3-simplex. This is particularly useful for visualizing the movement of weights between three assets.
- Project Point: Maps $(x, y, z)$ coordinates where $x+y+z=1$ to planar-simplex $(x, y)$ coordinates.
- Heatmap: Renders a lattice of points to show the density or value of a function across the simplex.
Sources: universal/algos/ternary/plotting.py:1-60
Execution and Profiling
Utility functions also provide high-level entry points for running and testing algorithms quickly.
quickrun: A wrapper that executes an algorithm on random data, prints a summary, and plots result.profile: Usesline_profilerto identify performance bottlenecks in an algorithm'ssteporweightsmethod.run_combination: Facilitates grid searching across algorithm parameters using multiprocessing.
Sources: universal/tools.py:45-71, universal/algo.py:188-230
Source Citations Summary
universal/tools.py: Primary utility for math and data generation. Lines 45-515universal/algo.py: Logic for running weights and combining subsets. Lines 188-230universal/result.py: Analysis metrics and summary generation. Lines 35-170universal/algos/estimators.py: Financial estimators and JPM data maps. Lines 1-700universal/algos/ternary/plotting.py: Visualization logic for 3-asset portfolios. Lines 1-60
Utility Tools provide the foundational mathematical and operational infrastructure that allows the library's OLPS algorithms to function efficiently, ensuring consistency in performance measurement and data handling across diverse strategies.