High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• infrastructure/CODEBASE_GUIDE.md
• README.md
• plans/reasoning_engine_analysis.md
• copilot-instructions.md
• ROADMAP.md

NeuralDBG is a causal inference engine designed for deep learning training dynamics. Its primary purpose is to provide structured, semantic explanations for neural network training failures (such as vanishing gradients) rather than simply presenting raw tensor data or metrics. By extracting meaningful transitions in learning dynamics and applying abductive reasoning, it helps developers identify root causes of model instability.

The system follows a "Hub and Spokes" architectural principle. The core engine (the Hub) handles pure business logic and foundational data structures, while various adapters (the Spokes) handle external dependencies like PyTorch hooks, developer interfaces, and infrastructure pipelines. This modularity ensures the system remains resilient to changes in external libraries or deep learning frameworks.

Sources: infrastructure/CODEBASE_GUIDE.md, README.md, copilot-instructions.md

Core Architectural Patterns

Hub and Spokes Design

The project is structured to protect the core reasoning engine from external noise and dependency churn. The "Hub" (represented primarily by neuraldbg.py) contains the semantic logic, while "Spokes" facilitate interaction with the environment.

This diagram illustrates the relationship between the core reasoning logic and the external interfaces.
Sources: copilot-instructions.md, infrastructure/CODEBASE_GUIDE.md

Event-Driven Abductive Reasoning

NeuralDBG processes training data through a four-stage pipeline to transform raw numerical metrics into human-readable causal hypotheses.

Stage	Action	Description
Extraction	Raw Tensor Metrics	Extracts statistics via module hooks during forward/backward passes.
Semanticization	Semantic Events	Converts raw stats into meaningful transitions (e.g., "Gradient Vanishing").
Pattern Detection	Coupling Detection	Identifies temporal or structural links between different events.
Inference	Causal Hypotheses	Uses rule-based reasoning to generate ranked explanations for failures.

Sources: plans/reasoning_engine_analysis.md, README.md

System Components

1. Semantic Event Extractor

The extractor attaches hooks to PyTorch modules to monitor gradients and activations. It classifies the "health" of these tensors based on predefined thresholds.

• ActivationHealth: Classifies states as NORMAL, SATURATED, DEAD, or ANOMALOUS.
• GradientHealth: Classifies states as HEALTHY, VANISHING, EXPLODING, or SATURATED.
• Hooks: Utilizes _install_hooks to attach to every layer without modifying user code.

Sources: infrastructure/CODEBASE_GUIDE.md, plans/reasoning_engine_analysis.md

2. Post-Mortem Reasoner

The reasoning engine performs analysis after training fails or completes. It identifies the first occurrence of a failure and traces its propagation.

The reasoner reduces thousands of minor events into a small set of high-confidence causal chains.
Sources: infrastructure/CODEBASE_GUIDE.md, README.md

3. Compiler-Aware Monitor

A critical architectural constraint is compatibility with torch.compile. The engine is designed to operate at module boundaries, ensuring that semantic extraction survives graph-level optimizations performed by the PyTorch compiler.

Sources: ROADMAP.md, README.md

Data Flow and Logic

The interaction between a developer's training loop and NeuralDBG is managed via a context manager. This ensures that hooks are lifecycle-managed and cleaned up automatically to prevent memory leaks.

This sequence shows how NeuralDBG intercepts training dynamics non-invasively.
Sources: infrastructure/CODEBASE_GUIDE.md, README.md, plans/reasoning_engine_analysis.md

Infrastructure and DevOps

The project architecture includes a robust delivery and reliability spoke defined in the infrastructure planning.

• CI/CD Gates: Automated workflows enforce a 60% minimum test coverage and run security scanners (bandit, safety) before code can be merged.
• Hermetic Workspaces: Standardized Docker configurations ensure reproducible environments for ML research.
• Rule Synchronization: A specific procedure ensures that AI directives and project rules remain consistent across all development branches.

Sources: copilot-instructions.md, ROADMAP.md, infrastructure/CODEBASE_GUIDE.md

Conclusion

The architecture of NeuralDBG prioritizes semantic clarity over raw data volume. By utilizing a "Hub and Spokes" model and focusing on causal inference, the system provides a specialized environment for debugging the complex training dynamics of deep neural networks. Its design ensures non-invasive integration with PyTorch and compatibility with modern optimization techniques like torch.compile.