Request Lifecycle

Relevant source files

The following files were used as context for generating this wiki page:

• cmd/app/main.go
• internal/search/service.go
• internal/qdrant/client.go
• internal/embeddings/ollama.go
• internal/embeddings/openai.go
• setup-ollama.md

The MauQNotes application provides a semantic search capability over a collection of text notes. The request lifecycle is divided into two primary operations: Ingestion and Query. The ingestion process converts text notes into numerical vector representations and stores them in a Qdrant vector database. The query process converts a search query into a vector and uses it to find the most semantically similar notes from the database.

The entire workflow is orchestrated from the main function, which first ingests a predefined set of notes and then performs a sample query. The system relies on external services for generating embeddings (Ollama or OpenAI) and for vector storage/search (Qdrant). Configuration is managed through environment variables.

System Initialization and Configuration

Before any operation can occur, the application initializes its configuration by loading environment variables and determining the correct vector dimensions for the chosen embedding model.

Sources: internal/search/service.go:12-34, internal/qdrant/client.go:73-82, internal/embeddings/ollama.go:26-37, internal/embeddings/openai.go:26-32

Environment Variables

The application's behavior is configured via environment variables loaded using godotenv.

Variable	Description	Default Value	Source File
EMBEDDING_MODEL	The name of the embedding model to use.	nomic-embed-text	internal/search/service.go:14
OLLAMA_URL	The base URL for the Ollama service.	http://localhost:11434	internal/embeddings/ollama.go:28
OPENAI_API_KEY	The API key for the OpenAI service.	(none)	internal/embeddings/openai.go:27
QDRANT_URL	The base URL for the Qdrant vector database instance.	(none)	internal/qdrant/client.go:76
QDRANT_API_KEY	The API key for the Qdrant service, if required.	(none)	internal/qdrant/client.go:77
QDRANT_COLLECTION	The name of the collection within Qdrant to store the note vectors.	(none)	internal/qdrant/client.go:78

Embedding Model Dimensions

The search.Init() function determines the vector dimension based on the EMBEDDING_MODEL environment variable. This dimension is crucial for creating the Qdrant collection with the correct vector size.

Model Name	Vector Dimensions
nomic-embed-text	768
all-minilm	384
mxbai-embed-large	512

Sources: internal/search/service.go:20-30, setup-ollama.md:21-25

Ingestion Flow

The ingestion flow is responsible for converting text notes into vector embeddings and storing them in the Qdrant database. This process is initiated by the search.IngestNotes function.

Sources: cmd/app/main.go:15-18, internal/search/service.go:36-63

The diagram below illustrates the sequence of operations during the ingestion of notes.

Key Steps:

1. Initiation: The main function calls search.IngestNotes with a slice of strings.
   Sources: cmd/app/main.go:16
2. Initialization: IngestNotes creates a new qdrant.Client and calls search.Init() to get the vector dimension.
   Sources: internal/search/service.go:37-41
3. Collection Creation: It calls qdrant.Client.CreateCollectionIfNotExists to ensure the target collection exists in Qdrant. This sends a PUT request to /collections/{name}. If the collection already exists (HTTP 409), it is ignored.
   Sources: internal/search/service.go:42, internal/qdrant/client.go:102-118
4. Vectorization: For each note string, it calls embeddings.FromOllama to generate a []float32 vector. This function sends the text to the Ollama /api/embeddings endpoint.
   Sources: internal/search/service.go:46, internal/embeddings/ollama.go:47
5. Point Creation: A qdrant.Point struct is created for each note, containing a unique ID, the generated vector, and a payload with the original text and tags.
   Sources: internal/search/service.go:50-57
6. Upsert: All created qdrant.Point objects are sent to Qdrant in a single batch via qdrant.Client.Upsert. This sends a PUT request to /collections/{name}/points?wait=true.
   Sources: internal/search/service.go:61, internal/qdrant/client.go:120-139

Query Flow

The query flow finds the most relevant notes for a given text query. This process is handled by the search.Query function.

Sources: cmd/app/main.go:21-23, internal/search/service.go:65-87

The following diagram shows the sequence for handling a search query.

Key Steps:

1. Initiation: The main function calls search.Query with a topK limit and a query string.
   Sources: cmd/app/main.go:22
2. Initialization: A new qdrant.Client is created.
   Sources: internal/search/service.go:66
3. Query Vectorization: The query string is converted into a vector by calling embeddings.FromOllama.
   Sources: internal/search/service.go:67
4. Search: The qdrant.Client.Search method is called with the query vector and topK limit. This sends a POST request to the Qdrant /collections/{name}/points/search endpoint.
   Sources: internal/search/service.go:71, internal/qdrant/client.go:141-174
5. Reranking: A naive reranking logic is applied to the results. If a result's payload text contains the query string (case-insensitive), its score is boosted by 0.01.
   Sources: internal/search/service.go:77-85
6. Return Results: The processed search results are returned to the caller.
   Sources: internal/search/service.go:86

Core Components and Data Structures

Qdrant Client

The internal/qdrant/client.go file defines the client for interacting with the Qdrant vector database.

Sources: internal/qdrant/client.go

This diagram shows the Client struct and the primary data structures it uses for requests.

Embedding Services

The application can generate embeddings using two different services, defined in internal/embeddings/. The search.service currently only uses the Ollama provider.

Sources: internal/embeddings/ollama.go, internal/embeddings/openai.go

Ollama (FromOllama)

• Reads OLLAMA_URL and EMBEDDING_MODEL from environment variables.
• Sends a POST request to {OLLAMA_URL}/api/embeddings.
• The request body is a JSON object with model and prompt fields.
• Returns a []float32 vector.

Sources: internal/embeddings/ollama.go:24-71

OpenAI (FromOpenAI)

• Reads OPENAI_API_KEY and EMBEDDING_MODEL from environment variables.
• Sends a POST request to https://api.openai.com/v1/embeddings.
• The request body is a JSON object with model and input fields.
• This function exists but is not currently called by the main search service.

Sources: internal/embeddings/openai.go:24-63

Summary

The request lifecycle in MauQNotes is a straightforward implementation of a semantic search system. It effectively decouples text-to-vector conversion (embeddings) from vector storage and search (Qdrant). The main application logic in cmd/app/main.go and internal/search/service.go orchestrates these two phases, providing a clear path for both ingesting new information and querying existing data. The system is configurable via environment variables, allowing flexibility in choosing embedding models and connecting to different service instances.