Remote Backend: Exo
Relevant source files
The following files were used as context for generating this wiki page:
The Exo backend is a planned v2.0.0 feature that enables caro to operate as a distributed intelligence system. It facilitates encrypted, peer-to-peer communication between caro nodes, allowing for organization-wide visibility and privacy-preserving data aggregation, particularly in closed or air-gapped networks. This system is also referred to as the "Karo Distributed Mesh".
The Exo backend extends caro's modular backend architecture, which supports various local and remote inference engines. While other backends like MLX, Ollama, and vLLM focus on generating commands from a single source, Exo is designed for distributed inference, shared context, and aggregate analytics across a cluster of nodes.
Sources: presentation/roadmap-slides.md, specs/005-karo-distributed-mesh/spec.md, specs/004-implement-ollama-and/spec.md
Architecture: The Karo Distributed Mesh
The Exo backend is the implementation of the "Karo Distributed Mesh" concept. This architecture connects individual caro installations (Nodes) into a secure, peer-to-peer network.
Sources: specs/005-karo-distributed-mesh/spec.md
Node Identity and Communication
Each node in the mesh has a unique cryptographic identity. Communication between nodes is secured end-to-end.
- Node Identity: On first startup, each node MUST generate an Ed25519 keypair, stored at
~/.local/share/caro/identity.key. The node's public identity is represented ascaro:ed25519:<base64-public-key>. - Secure Communication: All inter-node communication MUST use TLS 1.3.
- Authentication: Connections require mutual authentication, where both nodes verify each other's Ed25519 identity. Connections from nodes not on a trusted list MUST be rejected.
- Network Configuration: By default, nodes bind only to internal/loopback interfaces and do not require internet connectivity. Peer discovery can be handled via static configuration in a TOML file or optionally via mDNS/Bonjour.
Sources: specs/005-karo-distributed-mesh/spec.md:30-46
This diagram illustrates the secure connection process between two caro nodes in the mesh.
Key Entities
The distributed mesh is composed of several key logical entities that manage data, trust, and communication.
This class diagram shows the relationships between the main entities in the distributed mesh system.
Data Privacy and Aggregation
A core feature of the Exo/Mesh system is its privacy-preserving data aggregation. Raw command data never leaves the local node. Instead, nodes exchange signed, summarized, or aggregated data based on configurable trust policies.
Data Classification Levels
The system defines three strict levels of data to control information sharing.
| Level | Name | Description | Sharing Policy |
|---|---|---|---|
| L0 | Raw | The exact command text, arguments, and other direct metadata. | MUST NOT be transmitted outside the local node. |
| L1 | Summarized | Patterns derived from raw data, such as command counts (git: 45) and tool usage, without raw text. | Can be shared with trusted peers. All L1 data MUST be signed with the node's Ed25519 key. |
| L2 | Aggregated | High-level statistics combined from multiple L1 summaries. | Can be shared with nodes that have a "Supervisor" trust level. |
Sources: specs/005-karo-distributed-mesh/spec.md:48-52, 80-82
Trust and Access Control
Access to data is managed through a configurable trust policy.
- Trust Levels: The system supports multiple trust levels that can be assigned on a per-node or subnet basis:
Untrusted,ShareTo,QueryFrom,Peer, andSupervisor. - Policy Enforcement: Each node evaluates its sharing policy for every incoming query. Deny rules always override allow rules. For example, a
Supervisornode can only view L2 data and is explicitly prevented from accessing L0 or L1 data.
This flowchart shows how a node processes an incoming query from a peer.
Integration with caro Backends
The Exo backend will integrate into caro's existing CommandGenerator trait system, which provides a unified interface for different inference engines.
Sources: exports/session-logs/2025-09-13-readme.txt
Backend Architecture
The caro application uses a trait-based system to handle different backends, allowing for flexible integration of local and remote services.
// From README project structure and architecture description
#[async_trait]
trait ModelBackend {
async fn generate(&self, prompt: &str) -> Result<String>;
}
Source: exports/session-logs/2025-09-13-readme.txt
Implementations like OllamaBackend, VLLMBackend, and the future ExoBackend implement this trait. The main CLI application uses a factory or selector to choose the appropriate backend based on user configuration.
Sources: specs/001-create-a-comprehensive/tasks.md:100-104, exports/session-logs/2025-09-13-readme.txt
Configuration
Like other remote backends (vLLM, Ollama), the Exo backend will be configurable via CLI flags and the main configuration file.
| Option | Description | Default | Relevant File |
|---|---|---|---|
--backend | Inference backend to use (e.g., mlx, vllm, ollama, exo). | mlx | exports/session-logs/2025-09-13-readme.txt |
--endpoint | The endpoint URL for the remote backend. | http://localhost:8000 | exports/session-logs/2025-09-13-readme.txt |
Additionally, the mesh requires specific configuration for peer management, which is expected to be in ~/.caro/config.toml or a similar file.
- Static Peers: A list of trusted peer node addresses for use in air-gapped networks.
- Trust Policy: Configuration defining trust levels for other nodes or subnets.
Sources: specs/004-implement-ollama-and/spec.md:44-47, specs/005-karo-distributed-mesh/spec.md:40, 78-80
Roadmap and Status
The Exo backend, along with the underlying distributed intelligence system, is a planned feature for v2.0.0 of caro. It is categorized under "Advanced Features" and "Innovation Areas". The presence of src/backends/remote/exo.rs indicates that foundational work or a placeholder for this feature exists in the codebase.
Sources: presentation/roadmap-slides.md