MRMR Feature Selection
Relevant source files
The following files were used as context for generating this wiki page:
- openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md
- openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md
- openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/research.md
- openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md
- openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/tasks.md
- openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md
The Maximum Relevance and Minimum Redundancy (mRMR) feature selection module is a Rust-native implementation within the deep_causality_algorithms crate. Its primary purpose is to select a subset of features from a dataset that are maximally relevant to a target variable while being minimally redundant among themselves. This implementation specifically utilizes the F-test Correlation Quotient (FCQ) variant, chosen for its balance of computational efficiency and robustness in general-purpose applications.
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md:12-14, openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/research.md:4-8
Core Algorithm: FCQ Variant
The mRMR implementation focuses on the FCQ variant, which uses the F-statistic to measure relevance and Pearson correlation to measure redundancy. This model-free approach allows it to function as a general-purpose feature selector without requiring the estimation of complex probability distributions.
The selection logic follows an iterative process:
- Initial Selection: The feature with the highest F-statistic relative to the target variable is selected first.
- Iterative Selection: Subsequent features are chosen by maximizing the mRMR score, calculated as
Relevance / Redundancy. Redundancy is determined by the average correlation of a candidate feature with all previously selected features.
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/research.md:11-18, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:10-15
Mathematical Components
| Component | Metric Used | Formula / Description |
|---|---|---|
| Relevance | F-statistic | Derived from Pearson correlation (r): F = (n-2) * r^2 / (1 - r^2) |
| Redundancy | Pearson Correlation | Standard coefficient measuring linear correlation between two columns |
| mRMR Score | Quotient | Relevance (F-statistic) / Redundancy (Avg Correlation) |
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/research.md:21-30, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:37-41
Logic Flow and Architecture
The algorithm operates on CausalTensor data structures. The workflow includes data preprocessing, such as mean imputation for missing values (NaNs), followed by the iterative selection loop.
The diagram shows the high-level execution flow from tensor input to the final ranked list of features.
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md:104-106, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:37-45
Feature Scoring and Normalization
Unlike basic implementations that only return indices, this module returns a vector of tuples containing the (feature_index, score).
- First Feature Score: The score is the raw F-statistic (relevance).
- Subsequent Scores: The score is the calculated mRMR value (
Relevance / Redundancy). - Normalization: All returned importance scores are normalized to a range between 0.0 and 1.0 (inclusive) to facilitate comparison across different datasets.
Sources: openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:16-18, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/tasks.md:32-38
Parallelization with Rayon
To handle large datasets efficiently, the algorithm supports parallel execution via the rayon crate, guarded by a parallel feature flag.
This diagram illustrates how computationally intensive loops are offloaded to Rayon workers while maintaining sequential control for state updates.
Sources: openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:12-25, openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/plan.md:126-135
Error Handling
The module uses a custom MrmrError enum to manage failure states.
| Error Variant | Description |
|---|---|
InvalidInput | Triggered by empty tensors or invalid column indices. |
SampleTooSmall | Occurs when there are insufficient samples for statistical calculations. |
FeatureScoreError | Returned if a score results in NaN, Infinity, or if division by zero occurs during redundancy calculation. |
NotEnoughFeatures | Requested number of features exceeds available columns. |
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md:94-98, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:46-55
Integration with Causal Discovery Language (CDL)
The MRMR selector is integrated into the broader Causal Discovery Language (CDL) pipeline. A specialized version, mrmr_features_selector_cdl, is designed to work with CausalTensor<Option<f64>>. This allows the selector to ignore missing values represented as None through pairwise deletion, preventing bias that might otherwise be introduced by imputation.
Sources: openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md:9-16, openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md:86-92
Summary
The MRMR Feature Selection module provides a robust, parallelized, and score-aware tool for dimensionality reduction. By implementing the FCQ variant and integrating with the deep_causality data structures, it enables efficient identification of the most informative features for causal modeling and machine learning tasks. Its support for both raw numerical tensors and optional-type tensors ensures compatibility across various data cleaning workflows within the project.
Sources: openspec/changes/archive/spec-kit-old/001-mrmr-feature-selector/spec.md:65-67, openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:200-205