Qdrant Vector Database
Relevant source files
The following files were used as context for generating this wiki page:The qdrant package serves as the client-side interface to the Qdrant vector database, which is the core component for storing and retrieving note embeddings in the MauQNotes application. It encapsulates all HTTP interactions with the Qdrant API, providing a simplified Go interface for creating collections, upserting vector data points, and performing semantic searches. This module is central to the application's ability to find relevant notes based on the semantic meaning of a query rather than just keyword matching.
The overall process involves generating vector embeddings from text notes using an embedding model (like Ollama or OpenAI) and then using this qdrant client to manage these embeddings within the Qdrant database. The main application entrypoint demonstrates this flow by first ingesting notes and then executing a search query.
Sources: internal/qdrant/client.go, internal/search/service.go:40-43, cmd/app/main.go:15-28
Client Architecture
The interaction with the Qdrant database is managed through the qdrant.Client struct. This client holds the necessary configuration to connect and authenticate with a Qdrant instance.
Sources: internal/qdrant/client.go:66-71
Client Configuration
The client is instantiated via the qdrant.New() function, which reads configuration from environment variables loaded from a .env file.
| Environment Variable | Description | Default Value |
|---|---|---|
QDRANT_URL | The base URL of the Qdrant instance. | (None) |
QDRANT_API_KEY | The API key for authentication (optional). | (None) |
QDRANT_COLLECTION | The name of the collection to use for storing points. | (None) |
Sources: internal/qdrant/client.go:73-79
A validation step (validate()) ensures that QDRANT_URL and QDRANT_COLLECTION are not empty and that the URL starts with http:// or https://.
Sources: internal/qdrant/client.go:82-92
The diagram below shows the structure of the qdrant.Client and its methods.
Core Operations
The client exposes three main operations for interacting with the Qdrant database: creating a collection, upserting data points, and searching for data points.
1. Collection Management
Before any data can be stored, a collection must exist in Qdrant. The CreateCollectionIfNotExists method handles this by sending a PUT request to the /collections/{collection_name} endpoint.
This function takes two arguments:
vecSize: The dimension of the vectors that will be stored. This is determined by the embedding model used.distance: The distance metric for comparing vectors. The client currently hardcodes this toDistanceCosine.
The method is idempotent; if the collection already exists (HTTP status 409), it returns nil without an error.
Sources: internal/qdrant/client.go:102-124, internal/qdrant/client.go:13-15
The vector size (vecSize) is determined in the search service based on the EMBEDDING_MODEL environment variable.
| Model | Vector Dimension |
|---|---|
nomic-embed-text | 768 |
all-minilm | 384 |
mxbai-embed-large | 512 |
Sources: internal/search/service.go:19-29, setup-ollama.md:21-23
2. Data Ingestion (Upsert)
The Upsert method is used to add or update vector points in the collection. It takes a slice of qdrant.Point objects and sends them in a PUT request to the /collections/{collection_name}/points endpoint.
The Point data structure is central to this operation.
This diagram illustrates the components of a Qdrant Point.
Sources: internal/qdrant/client.go:24-29
The Payload is a flexible map[string]any that can store arbitrary metadata associated with the vector. In this application, it is used to store the original note text and tags.
Sources: internal/search/service.go:53-56
The following sequence diagram shows the complete data ingestion flow:
3. Semantic Search
The Search method performs a similarity search. It takes a query vector and a limit (top K results) and sends a POST request to the /collections/{collection_name}/points/search endpoint.
The request body is defined by the searchRequest struct, which includes the query vector and parameters like limit, with_payload, and with_vector. The client is configured to always include the payload but not the vector in the search results.
Sources: internal/qdrant/client.go:35-42, internal/qdrant/client.go:156-162
The results are returned as a slice of maps, where each map contains the point's id, score, and payload.
Sources: internal/qdrant/client.go:180-188
The search flow is illustrated below:
Embedding Generation
While not part of the qdrant package itself, the generation of vectors is a critical prerequisite. The application primarily uses the embeddings.FromOllama function to convert text into vectors by calling a local Ollama instance. An alternative, embeddings.FromOpenAI, is also available for using the OpenAI API. The choice of embedding provider and model directly impacts the vector dimension required when creating a Qdrant collection.
Sources: internal/search/service.go:49-52, internal/embeddings/ollama.go, internal/embeddings/openai.go
Summary
The qdrant package provides a clean and focused Go client for the MauQNotes project to interface with the Qdrant vector database. It abstracts the underlying HTTP API calls for collection creation, data upsertion, and semantic search into a few straightforward methods. This module, in conjunction with the embeddings package, forms the backbone of the application's semantic search capability, enabling users to find notes based on conceptual similarity.
Sources: internal/qdrant/client.go, internal/search/service.go