Semantic Event Extractor

Relevant source files

The following files were used as context for generating this wiki page:

• neuraldbg.py
• logic_graph.md
• infrastructure_planning/CODEBASE_GUIDE.md
• README.md
• plans/reasoning_engine_analysis.md

The Semantic Event Extractor is a core component of the NeuralDBG engine responsible for transforming raw tensor metrics from neural network training into high-level, meaningful "semantic events." Unlike traditional monitoring tools that track raw data, this module identifies significant transitions in learning dynamics—such as a shift from healthy gradients to vanishing gradients or activation saturation—allowing the system to focus on qualitative changes rather than quantitative noise.

Within the project's "Hub & Spokes" architecture, the Semantic Event Extractor resides in the Hub (Core Engine). It operates by attaching non-invasive hooks to PyTorch module boundaries, ensuring it can extract these semantic traces even when models are optimized using torch.compile. These extracted events serve as the primary input for the Causal Hypothesis generation process.

Sources: neuraldbg.py, infrastructure_planning/CODEBASE_GUIDE.md, README.md

Extraction Mechanism

The extraction process relies on PyTorch forward and backward hooks to observe training dynamics without modifying the user's model code. The extractor monitors two primary data streams: activations during the forward pass and gradients during the backward pass.

Hook Lifecycle

The Semantic Event Extractor follows a specific lifecycle managed by the NeuralDbg context manager:

1. Installation: Hooks are attached to every layer/module when entering the context.
2. Observation: Raw statistics (norms, sparsity) are calculated during each training step.
3. Classification: Statistics are compared against thresholds or previous states to determine if a "Semantic Event" has occurred.
4. Storage: If a transition is detected, a SemanticEvent object is instantiated and stored in a central event stream.

Sources: neuraldbg.py, logic_graph.md, infrastructure_planning/CODEBASE_GUIDE.md

Logic Flow Diagram

The following diagram illustrates how the extractor moves from raw training passes to the creation of semantic events.

The extractor focuses on transitions and shifts rather than raw tensor storage, enabling efficient causal inference.
Sources: logic_graph.md, plans/reasoning_engine_analysis.md

Data Structure: SemanticEvent

The SemanticEvent is the pure Python representation of a detected training anomaly or transition. Each event encapsulates the "what, where, and when" of a failure.

Attribute	Description
event_type	The category of the event (e.g., gradient_health, activation_regime).
layer_name	The specific identifier of the module or parameter where the event was detected.
step	The training step number where the transition occurred.
from_state	The classification state before the transition (e.g., HEALTHY).
to_state	The classification state after the transition (e.g., VANISHING).
confidence	A score indicating the certainty of the detection.
metadata	Additional stats such as raw norms or propagation patterns.

Sources: neuraldbg.py, logic_graph.md, README.md

State Classification Logic

The extractor uses a threshold-based abductive reasoning model to classify the "health" of various training components.

Gradient Health Classification

Gradient norms are categorized into four distinct states based on hardcoded heuristics:

State	Threshold / Logic
HEALTHY	Norm resides within the nominal range (default: 1e-6 to 1e3).
VANISHING	Norm falls below a critical threshold (default: < 1e-6).
EXPLODING	Norm exceeds a critical threshold (default: > 1e3).
SATURATED	Norm is very high but stable (default: > 1.0 in specific contexts).

Sources: neuraldbg.py, plans/reasoning_engine_analysis.md, infrastructure_planning/CODEBASE_GUIDE.md

Activation Regime Classification

Activation shifts are detected by monitoring sparsity changes and norm variances:

• NORMAL: Standard activation patterns.
• SATURATED: Activations are pushing against limits (common in Tanh/Sigmoid).
• DEAD: Neurons stop firing (common in ReLU cascades).
• ANOMALOUS: Sudden, unexpected shifts in activation distribution.

Sources: neuraldbg.py, logic_graph.md, plans/reasoning_engine_analysis.md

Integration and Performance

The Semantic Event Extractor is designed for minimal overhead and memory efficiency. Because it only stores high-level events rather than full tensors, it avoids the memory bloat common in traditional debuggers.

Performance Profile

• Time Complexity: Hook installation is $O(L)$ (where $L$ is the number of layers). Forward/Backward processing is $O(1)$ relative to tensor size since it only computes norms.
• Memory Complexity: Event storage is $O(E)$ (where $E$ is the number of detected events).
• Garbage Collection: The system includes mechanisms to remove hooks and can be enhanced with event pruning to maintain a low memory footprint.

Sources: plans/reasoning_engine_analysis.md, README.md

The architecture follows a modular Hub & Spokes design where the extractor acts as the primary observer for the Hub.
Sources: neuraldbg.py, infrastructure_planning/CODEBASE_GUIDE.md

Summary

The Semantic Event Extractor serves as the "eyes" of NeuralDBG, converting the noise of training into a structured, semantic trace. By focusing on transitions in gradient health and activation regimes, it provides the necessary evidence for the Post-Mortem Reasoner to generate causal hypotheses. This transition-centric approach allows for a non-invasive, memory-efficient analysis of deep learning failures that survives modern compiler optimizations.

Sources: README.md, logic_graph.md