Agent Orchestrator & Workflows

Relevant source files

The following files were used as context for generating this wiki page:

• backend/src/agent-orchestrator.ts
• backend/src/workflow-engine.ts
• backend/src/handler.ts
• src/renderer.tsx
• backend/src/service.ts
• instructions/design-backend-prompt.txt

The Agent Orchestrator and Workflow Engine constitute the core execution logic of the J.A.R.V.I.S. system. While the Workflow Engine manages predefined, structured sequences of system actions (routines), the Agent Orchestrator facilitates dynamic, multi-agent collaboration to solve complex, non-linear goals. These systems interface with the Action Engine to perform human-in-the-loop operations, ensuring that automated tasks remain under the operator's control.

Sources: backend/src/workflow-engine.ts, backend/src/agent-orchestrator.ts, instructions/design-backend-prompt.txt

Agent Orchestration

The DefaultJarvisAgentOrchestrator manages a multi-agent architecture where a single high-level goal is decomposed into tasks for specialized sub-agents. The orchestration logic follows a sequential "Planner -> Researcher -> Coder -> Reviewer" pipeline to synthesize comprehensive solutions.

Multi-Agent Pipeline

The collaboration process involves four distinct roles:

1. Planner: Analyzes the initial goal and structures the task breakdown.
2. Researcher: Performs parallel web searches (via BrowserAdapter) and document retrieval (via FileAdapter RAG) to gather context.
3. Coder/Analyst: Utilizes the LLM (xAI Grok) to elaborate on technical solutions based on researched data.
4. Reviewer/Synthesizer: Conducts final quality checks and generates a summary report for the user.

Sources: backend/src/agent-orchestrator.ts:13-138

Component Architecture

This diagram illustrates the sequential flow of information through the specialized sub-agents within the orchestrator.
Sources: backend/src/agent-orchestrator.ts:25-138

Workflow Engine

The DefaultJarvisWorkflowEngine executes "Single-Command Routines"—predefined lists of steps triggered by specific phrases or IDs. Unlike the dynamic Agent Orchestrator, workflows are deterministic sequences that utilize the system's bounded capabilities.

Standard Workflows

ID	Name	Description	Key Steps
morning-routine	Morgen-Routine	System prep for the day.	TTS Greeting, Open WebStark/Google, System Check.
dev-environment	Dev Start	Coding session setup.	TTS, Open VS Code, Open Chrome.
work-mode	Arbeitsmodus	General productivity setup.	Open Antigravity IDE, Proton Mail, Brave, Spotify.
system-check	System-Check	Health diagnostics.	TTS, Execute System Diagnose.

Sources: backend/src/workflow-engine.ts:11-57

Workflow Execution Logic

Workflows are executed through the runWorkflow method, which matches an input string against IDs, names, or triggerPhrases. The engine iterates through defined steps, converting them into JarvisActionIntent objects that are then processed by the ActionEngine.

The sequence shows how the Workflow Engine acts as a wrapper around the Action Engine to automate sequential approvals.
Sources: backend/src/workflow-engine.ts:64-106

Integration with Local Service

Both systems are exposed through the backend API handler, allowing the desktop renderer to trigger complex logic via HTTP requests.

API Endpoints

Endpoint	Method	System	Description
/v1/agents/list	GET	Orchestrator	Returns the current list of active sub-agent tasks.
/v1/agents/collaborate	POST	Orchestrator	Initiates a multi-agent collaboration for a given goal.
/v1/workflows/list	GET	Workflow	Returns all available predefined workflows.
/v1/workflows/run	POST	Workflow	Triggers a workflow execution by ID or trigger phrase.

Sources: backend/src/handler.ts:503-535

Agent Tool Loop

When the system is in "Cloud" mode using the xAI Adapter, it enters a runToolLoopChat. This is a sophisticated orchestration loop where the LLM can autonomously call tools (including memory search and file operations) up to a hard cap of turns to fulfill a request.

Tool Loop Flow

1. Inference: The model receives the system prompt (including SOUL.md and LifeOS context) and current messages.
2. Tool Call: If the model requests a tool (e.g., web.search), the orchestrator executes the tool locally.
3. Recursion: The result is fed back to the model to determine the next step.
4. Finalization: Once the tool chain ends, the final content is streamed to the user.

Sources: backend/src/handler.ts:404-501

Conclusion

The Agent Orchestrator and Workflow Engine provide J.A.R.V.I.S. with the ability to handle both structured routines and unstructured complex problem-solving. By utilizing a modular sub-agent architecture and a strict action-approval workflow, the system balances high-level autonomy with required human-in-the-loop safety.