Models (Qwen & Nomic)
Relevant source files
The following files were used as context for generating this wiki page:
Nyansa utilizes two primary local AI models managed via Ollama to power its "second brain" functionality: qwen2.5:3b for conversational chat and nomic-embed-text for generating vector embeddings. These models are selected for their ability to run on consumer hardware, including CPU-only machines with limited RAM.
The stack orchestrates these models through a dedicated nyansa-ollama service. The memory-api service interacts with these models to perform Retrieval-Augmented Generation (RAG) by embedding Obsidian vault notes for semantic search and utilizing the chat model to answer queries based on retrieved context.
Sources: README.md:16-19, docker-compose.yml:82-95
Model Infrastructure
Ollama serves as the local inference engine for all AI models within the Nyansa ecosystem. It runs in a container named nyansa-ollama and exposes an internal API for other services like n8n, open-webui, and memory-api.
Model Initialization
A one-shot container, nyansa-ollama-pull-model, handles the initial download of models upon the first start of the stack. It pulls the specific versions of Qwen and Nomic to ensure the environment is ready for inference.
Sources: docker-compose.yml:97-112, README.md:52-60
Hardware Profiles
Nyansa supports different hardware configurations through Docker Compose profiles. While the models are optimized for CPU, GPU acceleration is supported for NVIDIA and AMD hardware.
| Profile | Image Variant | Logic |
|---|---|---|
cpu | ollama/ollama:0.34.4 | Standard CPU inference |
gpu-nvidia | ollama/ollama:0.34.4 | Uses NVIDIA container toolkit for GPU reservation |
gpu-amd | ollama/ollama:0.34.4-rocm | Uses ROCm and direct device access (/dev/kfd, /dev/dri) |
Sources: docker-compose.yml:256-291, README.md:18-19
Qwen Chat Model (qwen2.5:3b)
Qwen2.5:3b is the default large language model (LLM) used for generating natural language responses. It is accessed by nyansa-open-webui for direct user interaction and by n8n for workflow automation.
- Role: Text generation and conversational interface.
- Access: Through the Ollama API at
http://nyansa-ollama:11434. - Client: Open WebUI connects to this model to provide the primary chat interface.
Sources: README.md:82-88, docker-compose.yml:200-201
Nomic Embedding Model (nomic-embed-text)
The nomic-embed-text model generates semantic vector representations of text. This is critical for the RAG pipeline, enabling the system to search for notes based on meaning rather than just keywords.
Embedding Process
The memory-api service sends text chunks to the Nomic model. It applies specific prefixes to the input text as required by the model:
search_document:for indexing vault content.search_query:for processing user search queries.
Configuration Parameters
The embedding behavior is controlled by environment variables within the memory-api and docker-compose.yml files.
| Variable | Default Value | Description |
|---|---|---|
EMBEDDING_MODEL | nomic-embed-text | The model name used by Ollama for embeddings. |
CHUNK_MAX_CHARS | 1500 | Maximum length of text sent to the model per chunk. |
OLLAMA_URL | http://nyansa-ollama:11434 | The endpoint for the embedding service. |
Sources: services/memory-api/src/config.ts:10-15, docker-compose.yml:235-241
Integration and Data Flow
Models interact with the system through the internal nyansa Docker network. The memory-api serves as the coordinator for the embedding model, while n8n and open-webui coordinate the chat model.
Search and Ingestion Flow
- Ingestion:
memory-apireads Markdown notes, splits them into chunks, and calls Nomic via Ollama to get vectors. - Storage: Vectors are stored in the Qdrant vector database.
- Search:
memory-apireceives a query, calls Nomic to embed the query, and performs a similarity search in Qdrant. - Response: Relevant chunks are returned to the requester (usually
n8nor a manual API call).
Summary
Nyansa relies on qwen2.5:3b and nomic-embed-text to provide a private, local AI experience. The models are abstracted behind the Ollama service, allowing for hardware-specific optimizations (CPU, NVIDIA GPU, or AMD GPU) while maintaining a consistent internal API for the rest of the application stack. Changes to these models, such as switching to a larger Qwen variant, generally require updating the .env configuration and pulling the new models into the nyansa_ollama_storage volume.
Sources: README.md:126-140, docker-compose.yml:107-112