System Architecture
Relevant source files
The following files were used as context for generating this wiki page:
ZINC Fusion V15 is an institutional-grade quantitative forecasting system designed for US Oil Solutions to predict Soybean Oil (ZL) futures prices. The system employs a multi-layer hierarchical ensemble machine learning architecture to provide probabilistic forecasts across four primary time horizons: 1 week (5d), 1 month (21d), 3 months (63d), and 6 months (126d).
The platform integrates a massive data pipeline consisting of 847+ global sources, processed through a 19-model stack. This stack combines core time-series predictors with domain-specific "Specialist" models to generate high-conviction signals, which are then calibrated and subjected to Monte Carlo risk simulations to support strategic procurement decisions.
Sources: README.md:3-15, AGENTS.md:3-7
Core Layered Architecture
The V15 architecture is structured into four distinct layers (L0 to L3), progressing from raw signal generation to final risk quantification.
L0: Base Predictors and Specialists
The foundation consists of two parallel streams:
- L0 Core: Four AutoGluon TimeSeriesPredictor models (one per horizon) utilizing a 25-model zoo including statistical (ARIMA, ETS), Deep Learning (DeepAR, TFT), and pretrained models (Chronos).
- L0 Specialists: Eleven domain-specific signal generators (The "Big-11") that analyze specialized market segments such as Crush Economics, China Demand, and Tariff Regimes.
L1, L2, and L3: Meta-Learning and Risk
- L1 Meta-Learner: Stacked ensembles that learn to weight the outputs of L0 Core models and Specialist signals.
- L2 Calibration: Uses Conformalized Quantile Regression (CQR) to produce a truthful outer risk envelope (p10_cal/p90_cal).
- L3 Risk Engine: A Monte Carlo simulation engine (N=5,000 to 20,000) that computes Value at Risk (VaR), Conditional VaR (CVaR), and barrier touch probabilities.
Sources: README.md:17-43, AGENTS.md:35-42, scripts/generate_synthesis.py:34-45
Architectural Flow Diagram
The following diagram illustrates the data flow from ingestion through the hierarchical model stack to the final output.
This diagram shows the hierarchical progression from raw data to calibrated risk-weighted forecasts.
Sources: README.md:73-86, AGENTS.md:58-65
The "Big-11" Specialist Taxonomy
Specialists act as horizon-agnostic signal generators. They do not produce price forecasts directly but generate (signal_1, signal_2, confidence) tuples that serve as features for the core models.
| Bucket | Model Type | Domain Focus |
|---|---|---|
| CRUSH | GBM | ZL/ZS/ZM spread, NOPA margins, processing economics |
| CHINA | GBM | GACC import flows, MOFCOM policy, demand outlook |
| TARIFF | Rule-based | Section 301 developments, retaliatory duty risk |
| ENERGY | VAR | Crude correlations, BOHO ratio, renewable diesel |
| BIOFUEL | NLP/EMA | RFS volumes, RIN pricing, EPA mandate tracking |
| FX | ARDL | DXY index, USD/BRL cross-rates, export competitiveness |
| FED | Ridge | Interest rate expectations, FOMC sentiment |
| PALM | ECM/Ridge | MPOB inventories, CPO/SBO substitution spreads |
| VOLATILITY | GARCH | Implied volatility regimes (OVX/VIX), risk-on/off |
| SUBSTITUTES | Random Forest | Canola/Sunflower spreads, arbitrage flows |
| TRUMP_EFFECT | Event Study | Executive policy volatility, trade rhetoric intensity |
Sources: README.md:121-143, AGENTS.md:74-88, frontend/src/lib/ai-intelligence.ts:79-95
Data Infrastructure and Schema Governance
The system utilizes Prisma Postgres as the authoritative cloud-hosted database. The architecture strictly enforces a 12-schema partition to maintain data integrity and prevent cross-contamination.
Database Schemas
The authoritative schema is defined in prisma/schema.prisma. Banned schemas (e.g., raw, gold, silver) are strictly prohibited to maintain the canonical structure.
| Schema | Purpose |
|---|---|
mkt | Market data (Futures OHLCV: ZL, ZS, ZM, CL) |
econ | FRED economic indicators and rates |
alt | Weather, Tariff deadlines, and alternative data |
supply | EIA Biodiesel, USDA WASDE, and production data |
training | Training matrices and Specialist signal outputs |
forecasts | Production model outputs and horizons |
analytics | Derived insights (Board Crush, Vol Regimes, Price Scenarios) |
ops | System health, ingestion logs, and quarantine records |
Sources: README.md:215-225, AGENTS.md:27-32, cubic.yaml:94-118
Inngest Orchestration
Orchestration is handled via Inngest running on Vercel. This manages daily topfills, specialist signal synchronization, and periodic model training jobs.
Sequence of automated tasks for data synchronization and model refreshment.
Sources: README.md:248-252, cubic.yaml:37-60
Risk and Synthesis Layers
Beyond statistical forecasting, V15 implements a synthesis layer to translate quantitative outputs into business intelligence.
Monte Carlo Simulation (L3)
The risk engine performs randomized low-discrepancy sampling (Sobol) to generate path-based outcomes.
- Standard Regime: N = 5,000 runs.
- Crisis Regime: N = 20,000 runs for tail stability.
- Outputs: Barrier probabilities (Time-to-touch) and probability-weighted blended forecasts.
AI Synthesis Engine (L5-C)
An LLM-driven synthesis engine (utilizing Claude 3.5 Sonnet or Opus 4.5) processes the Monte Carlo percentiles and SHAP driver importances to generate natural language briefings. This engine is strictly prohibited from inventing data; it only interprets the quantitative outputs of L4.
Sources: README.md:188-202, scripts/generate_synthesis.py:15-30, frontend/src/lib/ai-intelligence.ts:15-30
System Integrity and Guardrails
The project employs rigorous "Preflight" and "Cubic Review" checks to ensure architectural consistency.
- PIT Correctness: Landing tables are append-only. stand-alone
UPDATEorDELETEcommands on landing tables (e.g.,mkt.*,supply.*) are classified as critical corruption risks. - Quantile Monotonicity: The system enforces a "cross-guardrail" where
p10_cal ≤ p30 ≤ p50 ≤ p70 ≤ p90_cal. - Isolation: Inngest client IDs and database concurrency are strictly capped (Limit: 5) to prevent project cross-contamination on Vercel teams.
Sources: scripts/preflight_check.py:15-40, cubic.yaml:12-30, AGENTS.md:120-130
Conclusion
The ZINC Fusion V15 architecture represents a shift from simple regression to a complex, regime-aware ensemble. By decoupling domain-specific signals (Specialists) from the temporal prediction zoo (Core), the system achieves institutional-grade MAPE targets while providing actionable risk envelopes via its L2/L3 calibration and simulation layers.