Ollama Model Integration
Relevant source files
The following files were used as context for generating this wiki page:
The Ollama Model Integration serves as the local Large Language Model (LLM) provider for the J.A.R.V.I.S. system. It allows the application to perform chat inference, text generation, and tool-calling entirely on the user's local machine, ensuring privacy and offline availability. Within the project architecture, it is implemented as a provider-neutral JarvisModelAdapter that the backend service consumes when no cloud-based provider (like xAI) is configured or when the user explicitly selects local mode.
Sources: instructions/design-backend-prompt.txt:75-77, backend/src/handler.ts:457-463
Architecture and Components
The integration is built around the OllamaClient, which communicates with a local Ollama instance over HTTP loopback. The client is responsible for checking model readiness, performing non-streaming chat completions, and handling streaming responses using the NDJSON format.
Core Client Logic
The createOllamaClient function initializes the client with a base URL (defaulting to http://127.0.0.1:11434). It enforces security by requiring that the URL is a credential-free HTTP loopback origin.
The diagram above illustrates the selection logic where Ollama serves as the primary fallback or zero-config default.
Sources: backend/src/ollama.ts:13-25, backend/src/ollama.ts:98-103, instructions/design-backend-prompt.txt:77-79
Model Readiness Lifecycle
The system distinguishes between different states of the Ollama runtime to provide actionable instructions to the user. This is handled via the getReadiness method.
| Status | Meaning | Instruction Provided |
|---|---|---|
ready | Ollama is online and the required model is installed. | None (Ready for chat). |
model-missing | Ollama is online, but qwen2.5-coder:7b is not found. | ollama pull qwen2.5-coder:7b |
runtime-missing | Ollama binary is not found on the system. | Link to https://ollama.com/download |
unreachable | Ollama is installed but the background service is not running. | ollama serve |
Sources: backend/src/ollama.ts:35-71, packages/shared/src/index.ts:54
Data Flow and API Interaction
Chat Completion and Tool Calling
Ollama's /api/chat endpoint is used for both standard messages and tool-supported turns. The client sanitizes messages to roles (user, assistant, system) and handles tool definitions provided by the orchestrator.
The client extracts tool_calls from the Ollama response, assigning unique IDs if they are missing, and serializes arguments for the backend execution loop.
Sources: backend/src/ollama.ts:105-144, backend/src/handler.ts:311-385
Streaming Responses
For a responsive UI, the integration supports NDJSON (Newline Delimited JSON) streaming. The readNdjson generator processes chunks from the HTTP readable stream, emitting typed JarvisChatStreamEvent objects such as chat.start, chat.delta, and chat.done.
Configuration and Environment
The integration is managed through several layers of configuration across the Electron main process and the Bun backend.
| Parameter | Default Value | Description |
|---|---|---|
DEFAULT_OLLAMA_MODEL | qwen2.5-coder:7b | The specific model tag JARVIS expects to find in Ollama. |
DEFAULT_OLLAMA_BASE_URL | http://127.0.0.1:11434 | The local loopback address for the Ollama service. |
DEFAULT_OLLAMA_NUM_PREDICT | 1024 | Maximum tokens to generate per response. |
Sources: backend/src/ollama.ts:10-11, packages/shared/src/index.ts:54
Main Process Orchestration
The Electron main process includes an IPC handler jarvis:ensure-ollama that attempts to "ping" the local API. If unreachable, it attempts to spawn the ollama serve command as a detached background process to simplify the user onboarding experience.
Sources: src/main.ts:892-918
Implementation Details
NDJSON Parsing Logic
The client implements a robust NDJSON parser to handle cases where JSON records might be split across network chunks.
async function* readNdjson(body: ReadableStream<Uint8Array>): AsyncIterable<unknown> {
const reader = body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const chunk = await reader.read();
if (chunk.done) break;
buffer += decoder.decode(chunk.value, { stream: true });
const lines = buffer.split(/\r?\n/);
buffer = lines.pop() ?? "";
for (const line of lines) {
if (line.trim() !== "") yield JSON.parse(line) as unknown;
}
}
}
Sources: backend/src/ollama.ts:79-91
Request Normalization
The normalizeOllamaUrl function prevents common misconfigurations such as using remote hostnames or including credentials in the URL, which would violate the "local-only" security principle.
Sources: backend/src/ollama.ts:22-33, backend/tests/ollama.test.ts:31-36
Summary
The Ollama Model Integration provides J.A.R.V.I.S. with a powerful, private, and localized brain. By implementing a standardized adapter interface, it allows the system to remain functional without internet access while providing detailed guidance for users to manage their local model runtime. It handles the complexities of NDJSON streaming and tool-call parsing, ensuring a seamless transition between local and cloud-based inference modes.