Hub & Spokes Modular Design
Relevant source files
The following files were used as context for generating this wiki page:
The Hub & Spokes Modular Design is the foundational architectural principle of the NeuralDBG project. It is designed to insulate the core business logic—the "Hub"—from external dependencies, interfaces, and volatile infrastructures—the "Spokes." By maintaining this separation, the system ensures that the core causal inference logic remains stable and pure, while adapters for various frameworks, user interfaces, or deployment pipelines can be added or modified without impacting the internal engine.
This design emphasizes reversibility and resilience to dependency churn. In the context of NeuralDBG, the "Hub" contains the pure reasoning logic and semantic event definitions, while the "Spokes" facilitate interaction with external tools like PyTorch, CI/CD pipelines, and demonstration scripts.
Sources: copilot-instructions.md:28-36, infrastructure_planning/CODEBASE_GUIDE.md:83-90
Core Architecture Components
The architecture is divided into two primary zones: the stable Core and the flexible peripheral Adapters.
The Hub (Core Logic)
The Hub is the center of the system, containing the essential logic that defines the product's value. In NeuralDBG, this is primarily encapsulated in neuraldbg.py. It is responsible for:
- Defining Semantic Events (e.g., gradient health transitions).
- Executing the Causal Inference Engine (e.g., abductive reasoning).
- Maintaining foundational data structures (e.g.,
CausalHypothesis).
The Hub must remain free of I/O noise and external framework specifics to the greatest extent possible, acting as a "Pure Logic" center.
Sources: copilot-instructions.md:32-33, infrastructure_planning/CODEBASE_GUIDE.md:25-45
The Spokes (Adapters)
Spokes are specialized modules that handle external dependencies. They act as bridges between the Hub and the outside world. Adding a new feature or tool (such as a different ML framework or a new dashboard) involves adding a new spoke rather than modifying the Hub.
Sources: copilot-instructions.md:34-35, infrastructure_planning/CODEBASE_GUIDE.md:83-100
| Component | Category | Description |
|---|---|---|
neuraldbg.py | Hub | The Core Engine containing pure logic for causal reasoning and event extraction. |
| PyTorch (Hooks API) | Spoke | Framework-specific implementation used to observe training dynamics. |
| Demo Scripts | Spoke | Application-level code showing the engine in action (e.g., demo_vanishing_gradients.py). |
| CI/CD Pipeline | Spoke | Infrastructure-level automation (GitHub Actions, security scanners). |
Sources: infrastructure_planning/CODEBASE_GUIDE.md:83-100
Structural Relationships
The following diagram illustrates the unidirectional flow of logic and dependency management where the Hub remains the central point of truth.
This diagram shows the relationship between the central neuraldbg.py logic and its various external spokes.
Sources: infrastructure_planning/CODEBASE_GUIDE.md:83-100, infrastructure_planning/milestone_0_tasks.md:50-70
Implementation Principles
The Hub & Spokes design is governed by strict rules to prevent architectural erosion:
1. Protection of the Hub
Developers are instructed to protect the core from the "noise of the outside world." Modifications should primarily occur in the spokes. For example, DevOps or MLOps engineers should focus their work on the Infrastructure spoke (GitHub Actions, Docker) rather than modifying neuraldbg.py.
Sources: copilot-instructions.md:31, infrastructure_planning/CODEBASE_GUIDE.md:102-104
2. Reversibility and Pivot Readiness
Architectural decisions must be reversible. The system is designed to avoid vendor lock-in, ensuring that if a tool (like a specific experiment tracker) becomes obsolete, the spoke can be replaced without rewriting the core causal reasoning logic.
Sources: copilot-instructions.md:37-38
3. Modular Testing
Testing must reflect the modular architecture. Each module (spoke or hub) is tested independently before integration. This is supported by a testing pyramid requirement:
- 70% Unit Tests: Focusing on Hub logic and isolated Spoke functions.
- 20% Integration Tests: Verifying the interaction between a Spoke and the Hub.
- 10% E2E Tests: Testing the full flow from user input to result.
Sources: copilot-instructions.md:66-70, GAD.md:53-56
Operational Workflow
The Hub & Spokes model is integrated into the development lifecycle through milestone gates and CI/CD "Gators."
This sequence illustrates how the CI/CD Spoke protects the Hub from regressions and security vulnerabilities.
Sources: infrastructure_planning/milestone_0_tasks.md:16-45
Role Synchronization
The architecture defines clear boundaries for different personas:
- CEO/ML Researcher: Remains in the Hub, focusing on improving the core algorithms and causal hypotheses.
- DevOps/MLOps: Works on the Infrastructure Spokes, building "Gators" (CI/CD), registries (DVC), and hermetic workspaces (Docker).
This separation reduces cognitive load by preventing the need for frequent context switching between core research and infrastructure maintenance.
Sources: infrastructure_planning/milestone_0_tasks.md:7-14, infrastructure_planning/RULE_SYNC_PROCEDURE.md:10-25
Conclusion
The Hub & Spokes design is more than a structural pattern; it is a resilience strategy. By isolating the causal inference engine of NeuralDBG from the complexities of framework hooks, experimental tracking, and CI/CD pipelines, the project maintains high technical quality and pivot-readiness. This modularity allows the core to evolve into a powerful reasoning system while the external tools provide the necessary flexibility for modern deep learning workflows.
Sources: copilot-instructions.md:36-40, infrastructure_planning/CODEBASE_GUIDE.md:83-104