Vector Store Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Vector Store Architecture in the Memory Context Browser (MCB) provides the high-performance storage and retrieval backbone for semantic code search and persistent memory. It follows a multi-provider strategy, allowing the system to interface with various vector database engines through a standardized set of domain ports and adapter implementations. This architecture enables natural language queries over indexed codebases by leveraging vector embeddings to perform similarity searches.
Sources: README.md:10-12, README.md:100-112
Core Components and Multi-Provider Strategy
The system supports five primary vector store providers, each catering to different performance, local-first, or cloud-scale requirements. These providers are implemented within the mcb-providers crate and are decoupled from the core business logic through Clean Architecture boundaries.
| Provider | Description | Characteristics |
|---|---|---|
| Milvus | Distributed vector database | Cloud-scale, gRPC-based, high throughput |
| Qdrant | Open-source vector engine | Cosine similarity, high performance |
| EdgeVec | Embedded high-performance store | HNSW-based, local optimization |
| Pinecone | Managed cloud vector service | API-based, batch upsert support |
| Encrypted | Secure vector storage | AES-GCM encryption for stored vectors |
Sources: README.md:12-13, crates/mcb-providers/Cargo.toml:104-107, crates/mcb-infrastructure/Cargo.toml:44-46
Architectural Positioning
The vector store logic is distributed across three main layers to maintain strict dependency flows:
- mcb-domain: Defines the port traits and standard data structures (e.g.,
SearchResult,FileInfo). - mcb-providers: Contains the concrete adapter implementations for Milvus, Qdrant, etc.
- mcb-utils: Provides a Single Source of Truth (SSOT) for shared constants like field names and default configurations.
The diagram shows the Clean Architecture flow where providers implement domain ports to provide vector operations to the rest of the system.
Sources: AGENTS.md:214-234, README.md:100-112
Data Schema and Metadata Handling
MCB uses a standardized schema for all vector stores to ensure interoperability between different database engines. Metadata is typically stored alongside vectors in JSON format or specific database fields.
Standard Vector Fields
All providers utilize a set of constant field names for indexing and querying:
| Constant | Field Name | Purpose |
|---|---|---|
VECTOR_FIELD_ID | id | Unique identifier for the chunk |
VECTOR_FIELD_FILE_PATH | file_path | Source file location in the repository |
VECTOR_FIELD_START_LINE | start_line | Starting line of the code segment |
VECTOR_FIELD_CONTENT | content | Plain text of the code or memory |
VECTOR_FIELD_VECTOR | vector | The embedding array |
VECTOR_FIELD_LANGUAGE | language | Programming language of the source |
Sources: crates/mcb-utils/src/constants/vector_store.rs:134-161
Metadata Extraction Logic
The system includes shared utilities to parse search results from JSON payloads, ensuring consistent handling of line numbers (with fallbacks for legacy fields) and language tags across providers like Pinecone and Qdrant.
Sources: crates/mcb-providers/src/utils/vector_store.rs:48-77
Implementation Details
Milvus Integration
The Milvus provider utilizes the milvus-sdk-rust and is optimized for batch operations and low-latency queries using L2 distance metrics.
- Collection Naming: Prefixed with
mcb_and validated against regex^[a-zA-Z_][a-zA-Z0-9_]*$. - Batching: Query results are paginated into batches of 100 to stay within gRPC limits.
- Parameters: Uses
IvfFlatwith annlistof 128 by default for indexing.
Sources: crates/mcb-utils/src/constants/vector_store.rs:40-80, crates/mcb-providers/Cargo.toml:107
EdgeVec and HNSW Configuration
EdgeVec is a high-performance local store that uses the HNSW (Hierarchical Navigable Small World) algorithm.
- Connections: Configured with
M=16for upper layers andM0=32for the base layer. - Accuracy:
ef_constructionis set to 200, whileef_searchdefaults to 64 for a balance between speed and precision. - Quantization: Uses
scalarquantization by default.
Sources: crates/mcb-utils/src/constants/vector_store.rs:11-29
Qdrant Integration
Qdrant is configured to use Cosine Similarity as its primary distance metric and defaults to port 6333 for its API connection.
Sources: crates/mcb-utils/src/constants/vector_store.rs:104-108
Vector Operation Flow
The following sequence represents how a search request is handled through the vector store architecture:
This diagram illustrates the transformation of database-specific JSON metadata into standardized domain search results.
Sources: README.md:55-75, crates/mcb-providers/src/utils/vector_store.rs:48-77
Conclusion
The Vector Store Architecture in MCB is designed for flexibility and performance. By isolating database-specific logic within the mcb-providers crate and using a centralized set of constants in mcb-utils, the project ensures that adding new vector engines or modifying existing configurations is a surgical process that does not impact core business logic.
Sources: AGENTS.md:214-234, CONVENTIONS.md:14-22