Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• cmd/app/main.go
• internal/search/service.go
• internal/qdrant/client.go
• internal/embeddings/ollama.go
• internal/embeddings/openai.go
• setup-ollama.md

The core functionality of MauQNotes revolves around its semantic search and indexing capabilities. This system is designed to take a collection of text notes, convert them into numerical vector representations (embeddings), and store them in a vector database for efficient similarity search. The main application entrypoint demonstrates a simple workflow: ingesting a predefined list of notes and then performing a query against them to find the most relevant results.

This process is orchestrated by the search service, which integrates an embeddings generation module and a qdrant client. The system is configurable to use different embedding models via environment variables, which in turn determines the vector dimensions used for storage and search. The primary embedding provider is Ollama, with an alternative implementation for OpenAI.

Architecture Overview

The system is composed of several key packages that work together to provide the indexing and search functionality. The main application acts as the entry point, utilizing the search service to orchestrate the process. The search service, in turn, uses the embeddings package to generate vectors and the qdrant package to communicate with the Qdrant vector database.

This diagram illustrates the high-level component interaction.

Configuration

The system is configured using environment variables, which are loaded via godotenv.

Variable	Description	Default Value	Source File
EMBEDDING_MODEL	Specifies the model to use for generating embeddings.	nomic-embed-text	internal/search/service.go:12, internal/embeddings/ollama.go:29
QDRANT_URL	The base URL for the Qdrant database instance.	(none)	internal/qdrant/client.go:72
QDRANT_API_KEY	The API key for authenticating with Qdrant.	(none)	internal/qdrant/client.go:73
QDRANT_COLLECTION	The name of the collection in Qdrant to use.	(none)	internal/qdrant/client.go:74
OLLAMA_URL	The URL for the Ollama API service.	http://localhost:11434	internal/embeddings/ollama.go:23
OPENAI_API_KEY	The API key for the OpenAI embeddings API.	(none)	internal/embeddings/openai.go:27

Sources: internal/search/service.go:11-14, internal/qdrant/client.go:70-76, internal/embeddings/ollama.go:21-31, internal/embeddings/openai.go:25-29

Indexing Flow (Ingestion)

The indexing process, or "ingestion," is initiated by the search.IngestNotes function. This function takes an array of strings (notes) and processes them for storage in Qdrant.

The sequence of operations is as follows:

1. A new Qdrant client is created.
2. The search.Init function is called to determine the correct vector dimension based on the EMBEDDING_MODEL environment variable.
3. The Qdrant client ensures a collection exists with the correct dimension and distance metric (Cosine).
4. The function iterates through each note. For each note:
   a. It calls embeddings.FromOllama to convert the text into a vector embedding.
   b. It constructs a qdrant.Point object containing a unique ID, the generated vector, and a payload with the original text and tags.
5. Finally, all created qdrant.Point objects are sent to the Qdrant database in a single Upsert operation.

Search Flow (Query)

The search process is handled by the search.Query function. It takes a text query and an integer topK (the number of results to return).

The sequence of operations is as follows:

1. A new Qdrant client is created.
2. The query string is converted into a vector embedding using embeddings.FromOllama.
3. The Qdrant client's Search method is called with the query vector and topK limit.
4. The search service performs a "naive rerank" on the results. It iterates through the results and slightly boosts the score (+0.01) of any result whose payload text contains the query string (case-insensitive).
5. The potentially reranked results are returned to the caller.

Components

Embedding Generation

The embeddings package is responsible for converting text to vectors. It supports two providers.

Ollama (embeddings/ollama.go)

This is the primary provider used in the search service. It sends a POST request to the /api/embeddings endpoint of an Ollama instance.

• Function: FromOllama(text string) ([]float32, error)
• Configuration: Reads OLLAMA_URL (defaults to http://localhost:11434) and EMBEDDING_MODEL (defaults to nomic-embed-text).
• Models: The dimension of the generated vector depends on the model.
  • nomic-embed-text: 768 dimensions
  • all-minilm: 384 dimensions
  • mxbai-embed-large: 512 dimensions (Note: setup-ollama.md lists this as 1024, but service.go has it as 512. The service logic takes precedence.)

Sources: internal/embeddings/ollama.go, internal/search/service.go:18-29, setup-ollama.md

OpenAI (embeddings/openai.go)

An alternative provider that is not directly used by the search service but is available in the codebase. It sends a POST request to the https://api.openai.com/v1/embeddings endpoint.

• Function: FromOpenAI(text string) ([]float32, error)
• Configuration: Requires OPENAI_API_KEY and EMBEDDING_MODEL to be set.

Sources: internal/embeddings/openai.go

Qdrant Client

The qdrant package provides a client for interacting with the Qdrant vector database's HTTP API. It encapsulates the logic for creating collections, upserting points, and performing searches.

The diagram below shows the main client struct and the data structures it uses for API requests.

Key methods of the Client:

• CreateCollectionIfNotExists: Sends a PUT request to /collections/{name}. It handles a 409 Conflict status code gracefully, meaning it won't fail if the collection already exists.
• Upsert: Sends a PUT request to /collections/{name}/points?wait=true to insert or update data points. The wait=true query parameter ensures the operation is completed before the API returns a response.
• Search: Sends a POST request to /collections/{name}/points/search with a query vector to find the most similar points based on the collection's distance metric.

Sources: internal/qdrant/client.go:103-181