Vector Databases (Vespa & OpenSearch)
Relevant source files
The following files were used as context for generating this wiki page:
Onyx utilizes high-performance vector databases to facilitate efficient semantic search and document retrieval. While the architecture supports both OpenSearch and Vespa, Vespa serves as the primary vector database for core search operations, including hybrid retrieval and document indexing. These databases store document chunks, their corresponding embeddings, and associated metadata to enable fast, relevant search results in a Retrieval-Augmented Generation (RAG) pipeline.
The system is designed to handle multi-tenant environments, complex filtering semantics, and real-time updates. The vector database is a critical component of the indexing pipeline, receiving processed document chunks from background workers and serving them during user queries.
Sources: AGENTS.md:126-130, backend/onyx/document_index/vespa/index.py:270-280
Architecture and Integration
The vector database integration is abstracted through a standard interface, allowing the application to interact with different backends. Vespa is the default implementation for large-scale deployments, managing the storage of embeddings and performing complex YQL (Yahoo Query Language) queries.
Component Relationship
The following diagram illustrates how the document index (Vespa/OpenSearch) interacts with the rest of the Onyx backend during the indexing and search phases.
The vector database serves as the centralized storage for searchable content, updated by background workers and queried by the API server.
Sources: AGENTS.md:31-50, backend/onyx/document_index/vespa/index.py:465-470
Key Classes and Structures
The primary interface for interacting with Vespa is the VespaIndex class, which implements the DocumentIndex interface. It manages index creation, document insertion, updates, and various retrieval strategies.
| Class / Structure | Description |
|---|---|
VespaIndex | Main class for Vespa integration, handling deployment, indexing, and retrieval. |
VespaDocumentFields | Data structure containing document-level metadata like access control lists (ACL), document sets, and boost factors. |
InferenceChunk | The standard unit of data returned from the vector database, containing content, metadata, and scores. |
IndexFilters | Used to define search constraints such as source types, tags, or time ranges. |
Sources: backend/onyx/document_index/vespa/index.py:164-200, backend/onyx/document_index/vespa/index.py:646-660
Vespa Index Management
Vespa uses schemas defined in .sd (schema definition) files. Onyx dynamically generates these schemas using Jinja2 templates to accommodate different embedding dimensions and precision requirements.
Index Deployment Flow
When the system starts or when a new tenant is registered, Onyx prepares a Vespa application package, which includes services.xml and the schema definitions, and activates it via the Vespa configuration server.
Deployment involves rendering XML and SD templates into a zip file and pushing it to the Vespa management API.
Sources: backend/onyx/document_index/vespa/index.py:226-267
Indexing and Batching
To optimize performance, Vespa operations are batched. Although Vespa's API is document-based, Onyx uses a ThreadPoolExecutor to parallelize multiple HTTP requests.
- Batch Size: 128 (default)
- Threads: 32 (default)
- Timeout: 10 seconds
Sources: backend/onyx/document_index/vespa_constants.py:20-31, backend/onyx/document_index/vespa/index.py:534-555
Retrieval Strategies
The vector database supports multiple retrieval modes to balance between keyword matching (BM25) and semantic similarity.
Hybrid Retrieval
Hybrid retrieval combines semantic search (using embeddings) with keyword search. The system uses a ranking.profile in Vespa to score documents based on both signals.
ID-Based and Admin Retrieval
- ID-Based Retrieval: Specifically fetches document sections based on document IDs and chunk indices.
- Admin Retrieval: A search mode that includes hidden documents and uses a specific "admin_search" ranking profile.
- Random Retrieval: Used for creating assistant starter messages by sampling random chunks matching specific filters.
Sources: backend/onyx/document_index/vespa/index.py:720-756, backend/onyx/document_index/vespa/index.py:758-780, backend/onyx/document_index/vespa/index.py:868-888
Metadata and Knowledge Graph (KG) Integration
Vespa also stores Knowledge Graph fields, allowing the system to filter and retrieve documents based on entities and relationships.
KG Fields in Vespa
Documents in Vespa are enriched with fields such as:
kg_entities: A list of entities associated with the chunk.kg_relationships: Structured relationships between entities (source, relationship type, target).
Sources: backend/onyx/document_index/vespa/index.py:85-115, backend/onyx/prompts/kg_prompts.py:538-555
Metadata Management Table
The following table describes fields indexed within the Vespa schema used for search and filtering.
| Field Name | Type | Description |
|---|---|---|
document_id | String | Unique identifier for the source document. |
chunk_id | Int | Index of the chunk within the document. |
embeddings | Tensor | Vector representation of the chunk content. |
access_control_list | Map | User IDs or groups allowed to view the document. |
boost | Float | Manual weight applied to the document ranking. |
hidden | Boolean | Whether the document is excluded from standard search. |
doc_updated_at | Long | Timestamp (seconds since epoch) of the last update. |
Sources: backend/onyx/document_index/vespa_constants.py:38-75, backend/onyx/document_index/vespa/index.py:782-800
Summary
The vector database system in Onyx, primarily centered around Vespa, provides a robust and scalable foundation for enterprise search. By combining flexible Jinja2-based schema management, parallelized batch processing, and hybrid retrieval strategies, the system ensures that document search is both fast and semantically accurate. Integration with background workers via specialized interfaces allows for real-time metadata updates and Knowledge Graph enrichment, supporting complex RAG workflows.