The Background Worker Engine
Relevant source files
The following files were used as context for generating this wiki page:
The Background Worker Engine is the high-performance machinery that handles time-consuming network operations like discovery scans, configuration backups, and packet analysis without making you wait for the screen to refresh. It acts as the "hands" of the AI assistant, executing the actual work in the background while you continue to interact with the platform.
This engine solves the primary problem of network automation: tasks like scanning a large subnet or analyzing a multi-gigabyte packet capture take too long for a web browser to wait. By moving these tasks to background workers, the platform remains responsive and reliable even during heavy operations across complex, multi-vendor environments including Cisco, Juniper, Arista, and cloud providers like AWS and Azure.
Sources: README.md, CLAUDE.md, deploy/docker/README.md
1. Overview
The AI Network Operations Platform is designed for enterprise infrastructure teams who need a self-hosted, secure assistant to manage multi-vendor environments. The Background Worker Engine is the component that executes long-running "jobs" requested by specialized AI agents.
Platform Philosophy and Capabilities
The platform operates on a "local-first" and "secure by default" philosophy. It ensures that every change requested is explained by the AI and requires explicit human approval before the worker engine executes it.
- Supported Vendors: Cisco (IOS, IOS-XE, NX-OS), Juniper, Arista, Palo Alto, Fortinet, F5, BlueCat, Infoblox, AWS, Azure, and VMware.
- Specialized Team Members (Agents): The platform uses specialized agents to handle different requests:
- Master Architect: The supervisor that routes your questions to the right specialist.
- Discovery Agent: Enumerates inventory and scans for new devices.
- Troubleshooting Agent: Diagnoses live routing, BGP, OSPF, and ACL faults.
- Configuration Agent: Tracks config drift and handles compliance checks.
- DDI Agent: Manages DNS, DHCP, and IP addresses.
- Security Agent: Audits firewall policies for security posture.
- Documentation Agent: Generates inventory tables and topology maps.
Sources: README.md:3-8, CLAUDE.md:8-25, backend/app/llm/prompts/init.py:291-360
2. Architecture
The Background Worker Engine is one of seven core components that make up the platform. Each component has a specific job in the "production line" of network automation.
Core Platform Components
- frontend: The operations console where you view the topology map, chat with the AI, and approve changes.
- api: The communication hub that takes your requests and translates them for the rest of the system.
- worker: The background engine that runs discovery jobs, takes config backups, and captures packets without blocking your session.
- postgres: The system of record that stores all your device info and audit logs.
- neo4j: A living topology map that tracks how every device, interface, and subnet connects to everything else.
- redis: The message queue that acts as a waiting room for tasks before a worker picks them up.
- ollama: The local brain that provides the AI's reasoning without sending your data to the cloud.
Sources: README.md:16-24, deploy/docker/README.md:8-16
The Worker Queue System
The Background Worker Engine uses a queue-based system to organize work. Different types of work are sent to specific "lanes" or queues so that a heavy packet analysis job doesn't block a quick configuration check.
| Queue Name | Responsibilities |
|---|---|
discovery | SNMP scans, SSH neighbor discovery, and inventory enumeration. |
config | Fetching running configs, comparing them to baselines, and drift detection. |
packet | Running tcpdump or tshark captures and parsing pcap files. |
docs | Deterministic rendering of Markdown inventories and Mermaid topology source. |
Sources: deploy/docker/docker-compose.yml:45-48, backend/app/agents/documentation/agent.py:5-15
Deployment and Scaling
The Background Worker Engine can be deployed in two main ways depending on the size of your network:
- Docker Compose: Ideal for lab environments or small teams. All components run on a single server, making it easy to stand up.
- Kubernetes (Helm): The production-grade option for multi-node, highly available environments. It uses hardened OS isolation to ensure that background tasks (like untrusted packet analysis) are sandboxed and cannot affect the rest of the system.
Sources: deploy/docker/README.md:1-6, deploy/kubernetes/netops/README.md:8-25
Vendor Plugin System
The engine uses standardized connectors (plugins) for each vendor. This means that whether the worker is talking to a Cisco router or a BlueCat Address Manager, it normalizes the output into a consistent format. This allows the AI to understand your network as a whole, regardless of the underlying hardware brands.
Sources: README.md:26-28, deploy/kubernetes/netops/README.md:1-5
3. Diagrams
System Overview
This diagram shows how users interact with the platform and how the platform connects to your infrastructure and AI providers.
Sources: README.md:16-30, CLAUDE.md:15-25
Agent Routing and Specialist Flow
When you ask a question, the Master Architect (supervisor) routes it to the correct specialist.
Change Approval Lifecycle
Every change follows a strict path to ensure network stability.
Topology Data Model: What the Platform Knows
The living topology map (Neo4j) tracks these relationships.
Sources: CLAUDE.md:52-60, README.md:23
Summary
The Background Worker Engine provides the essential asynchronous power needed for modern network operations. By offloading heavy tasks like discovery and pcap analysis to specialized queues, and enforcing a human-in-the-loop approval workflow for all changes, it balances operational speed with enterprise security. Whether deployed via simple Docker Compose or a complex Kubernetes cluster, the engine ensures that the AI Network Operations Platform remains a responsive and reliable partner for network engineering teams.