Specialized Data Structures

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• deep\_causality\_data\_structures/src/window\_type/mod.rs
• deep\_causality\_data\_structures/src/grid\_type/mod.rs
• website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md
• openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md
• openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md

The DeepCausality framework utilizes a suite of specialized data structures designed to handle the high-performance requirements of context-aware causal reasoning. These structures are engineered for efficiency in dynamic systems where causal rules and environmental contexts change frequently.

The primary focus of these structures is to balance the trade-offs between mutability (required for evolving models) and performance (required for complex causal analysis). Key categories include advanced graph structures, multi-dimensional tensors, and time-aware sliding windows.

Sources: AGENTS.md, website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md

UltraGraph: The Hypergraph Backend

UltraGraph serves as the foundational hypergraph data structure for DeepCausality, functioning as the explicit environment (Context) where Causaloids operate. It is specifically designed to solve the "conundrum" of providing both fast mutation for graph-based contexts and fast traversal for causal reasoning.

Two-State Architecture

UltraGraph employs a unique two-state architecture that transitions between a mutable state and an optimized analytical state.

• DynamicGraph (The "Evolve" State): An adjacency list optimized for constant time O(1) additions of nodes and edges.
• CsmGraph (The "Analyze" State): An immutable, frozen representation using a custom Compressed Sparse Row (CSR) format. This format is cache-friendly and requires zero locking for graph algorithms.

Sources: website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md

The diagram illustrates the lifecycle of a graph as it moves between mutable construction and optimized analysis.

Sources: website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md

Causal Tensor

The CausalTensor is a multi-dimensional array structure used for numerical data processing within the algorithmic and discovery modules of the project. It is the primary input for feature selection and causal discovery algorithms.

Data Model and Variants

The framework supports different configurations of tensors to handle data quality issues, such as missing values (NaN), without introducing bias through premature substitution.

Tensor Variant	Storage Type	Primary Use Case
CausalTensor<f64>	Standard floating point	High-performance numerical computation
CausalTensor<Option<f64>>	Optional floating point	Missing value handling via pairwise deletion

Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md, openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md

Processing Pipeline

When data enters the pipeline, it often undergoes cleaning to prevent bias from NaN substitution. The OptionNoneDataCleaner transforms standard tensors into optional tensors, allowing subsequent steps like the MrmrFeatureSelector to ignore None values during statistical calculations.

Sources: openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md

Specialized Collection Structures

DeepCausality provides dedicated crates for high-performance collections that support specific mathematical and temporal operations.

Sliding Windows and Grid Arrays

The deep_causality_data_structures crate provides low-level data structures such as sliding-window types and grid-arrays. These are essential for processing streaming data and maintaining temporal state within a causal model.

Sources: AGENTS.md, deep_causality_data_structures/src/window_type/mod.rs, deep_causality_data_structures/src/grid_type/mod.rs

Sparse and Tensor Storage

The project distinguishes between dense and sparse data representation:

• deep_causality_tensor: General tensor operations.
• deep_causality_sparse: Implements the Compressed Sparse Row (CSR) format for memory-efficient matrix operations where most elements are zero.

Sources: AGENTS.md

This diagram represents the relationship between the primary data structures used for storage and analysis.

Sources: AGENTS.md, website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md, openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md

Summary

The specialized data structures in DeepCausality—ranging from the two-state UltraGraph to the flexible CausalTensor variants—are designed for high-performance, context-aware reasoning. By utilizing cache-friendly layouts like CSR and symbolic handling of missing data through Option<f64>, the framework ensures that causal analysis remains both fast and numerically sound in dynamic environments.

Sources: website/web/src/content/blog/en/lessons-learned-from-rewriting-ultraGraph.md, openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md