Overall System Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The J.A.R.V.I.S. (Jarvis-Glas) project is a local-first, highly personalized AI assistant framework designed as a private "Control Room" for desktop automation and orchestration. The system follows a modular, three-tier architecture comprising a sandboxed Electron frontend, a local loopback-only TypeScript service, and a specialized agent-orchestration layer. The architecture emphasizes high technical aesthetics through a "Grok-faithful" 3D HUD (Heads-Up Display) and strict security principles regarding data locality and human-in-the-loop approvals.
Sources: instructions/design-backend-prompt.txt:8-15, instructions/CONTEXT.md:4-8, SOUL.md:4-10
Core Architectural Components
The system is partitioned into three authoritative domains to ensure clean seams and security boundaries:
- Electron Shell (Host): Owns the machine lifecycle, process management, and native filesystem/Git access. It communicates with other layers via a narrow, typed capability bridge.
- Renderer (HUD): Manages the presentation layer, navigation, and ephemeral interaction state using React and Three.js. It is sandboxed with disabled Node integration and restrictive CSP.
- Backend Service: A local loopback-only TypeScript service (Bun/TypeScript) that handles sessions, tool execution, model adapters (xAI Grok/Ollama), and RAG (Retrieval-Augmented Generation) workflows.
Sources: instructions/AGENTS.md:10-21, instructions/design-backend-prompt.txt:8-15, instructions/CONTEXT.md:12-18
System Domain Mapping
| Component | Responsibility | Technology Stack |
|---|---|---|
| Electron Main | Machine control, IPC, System-Tray, Window management | Electron, TypeScript |
| Renderer UI | 3D Orb Visualization, HUD Layout, Presentation | React, Three.js, Vanilla CSS |
| Backend Service | Agent workflows, RAG, xAI/Ollama adapters | Bun, TypeScript |
| Barehands | Optional hand-tracking interface (Port 8794) | Python |
Sources: instructions/CONTEXT.md:11-18, CONTRIBUTING.md:46-52
Data Flow and Communication
The system utilizes a combination of RESTful API calls and Server-Sent Events (SSE) for communication between the UI and the local backend. A pairing mechanism ensures that only authorized local clients can access sensitive memory or action routes.
Logical Service Flow
The following diagram illustrates the interaction between the User, the Renderer HUD, and the Backend Service during a standard command execution involving tool calls.
Agent Identity and Intelligence (SOUL & LifeOS)
The "Brain" of the system is governed by two primary configuration sources that are injected into every model turn to provide context-aware responses without hardcoding user data:
- SOUL.md: Defines the agent's identity, role, and behavioral directives (e.g., "Maximum Intelligence, Minimum Interruption"). It mandates that the agent speaks as a "Discrete Butler" and addresses the user as "Master".
- LifeOS (TELOS): A collection of Markdown files (
MISSION.md,VALUES.md,CURRENT_STATE.md) stored in the user's Documents folder. This provides the agent with the user's current goals, location, and preferences, allowing for disambiguation of entities like "start the project" or "check the weather."
Sources: backend/src/handler.ts:39-85, SOUL.md:12-25
Functional Modules
The backend orchestrates several specialized adapters to interact with the system and external APIs:
- Action Engine: Implements a human-in-the-loop approval architecture. Actions (e.g.,
app.open_url,system.execute_command) transition through states:proposed->executing->completed. - Memory Adapter: Manages long-term persistence of operator preferences, structured facts, and semantic documents via a file-based adapter.
- Model Adapters: Provider-neutral interface that selects xAI Grok (if
XAI_API_KEYis present) or falls back to Ollama. - Voice Engine: Privacy-first architecture with a hard-mute default. It handles STT via xAI Whisper and TTS via xAI Zenith or Fish Audio.
Sources: instructions/design-backend-prompt.txt:39-78, backend/src/handler.ts:98-130
System State Hierarchy
The system uses an observable "ladder" to resolve authority and handle failures across different environments.
Visual Interface (HUD) Design
The interface follows a "Sci-Fi / Cyberpunk Minimalism" style. The central element is a 3D shader orb that reflects the system's internal state through visual language (e.g., luminous cyan for ready, amber for executing-approved, violet faster orbit for thinking).
Sources: instructions/design-frontend-prompt.txt:7-14, instructions/design-backend-prompt.txt:88-100
Design Token Samples
| Token | Semantic Use |
|---|---|
--ui-stroke-tertiary | Default in-panel divider / list hairline |
--stroke-nous | Elevation hairline for floating overlays |
--shadow-nous | Layered contact shadow for modal surfaces |
--ui-bg-quaternary | Soft fill for secondary controls |
Sources: instructions/DESIGN.md:65-76, src/renderer.css:32-47
Conclusion
The architecture of J.A.R.V.I.S. is built to be an "Advanced Personal Operating System" rather than a simple chatbot. By strictly separating the Electron host, the HUD presentation, and the local backend service, the project maintains a high degree of security and modularity. The use of file-based context injection (SOUL/LifeOS) ensures that the assistant remains deeply personalized while keeping all sensitive data on the user's local machine.
Sources: SOUL.md:7-10, instructions/design-backend-prompt.txt:104-110