Semantic Code Search
Relevant source files
The following files were used as context for generating this wiki page:
Semantic Code Search is a core feature of the Memory Context Browser (MCB) that enables natural language queries over indexed codebases. Unlike traditional keyword-based search, it utilizes vector embeddings to understand the intent and context of a query, allowing developers to find relevant code snippets even when exact keywords are not present.
The system is built on a Clean Architecture, separating the domain logic of search results and embeddings from the infrastructure-level providers like Milvus, Qdrant, and various embedding models such as OpenAI, Ollama, and FastEmbed. It further enhances result relevance through a hybrid search approach that combines semantic vector similarity with lexical BM25 ranking.
Sources: README.md:14-15, AGENTS.md:164-168
System Architecture
The Semantic Code Search system follows a tiered architecture where search requests originate from MCP-compatible clients, pass through the server layer for argument validation, and are processed by specialized search engines.
High-Level Data Flow
The following diagram illustrates the flow of a search request from the user to the result generation.
The search process involves parallel lexical and semantic processing before results are fused and ranked.
Sources: crates/mcb-providers/src/hybrid_search/engine.rs:13-31, crates/mcb-server/src/args/search.rs:107-133
Hybrid Search Engine
The HybridSearchEngine is responsible for combining results from vector stores with keyword-based BM25 scores. It maintains a CollectionIndex for each repository or dataset, mapping code chunks to their lexical characteristics.
- BM25 Weight: Default weight for keyword matching (typically 0.4).
- Semantic Weight: Default weight for vector similarity (typically 0.6).
Sources: crates/mcb-providers/src/hybrid_search/engine.rs:44-51, crates/mcb-providers/src/hybrid_search/mod.rs:36-40
Search Workflow
The search lifecycle is divided into five distinct phases, ranging from initial code ingestion to final ranking and delivery.
Workflow Phases
| Phase | Action | Description |
|---|---|---|
| 1 | Parsing & Chunking | Code is split into CodeChunk entities based on language-specific logic. |
| 2 | State Management | CodebaseSnapshot is created to track file integrity and metadata. |
| 3 | Embedding Generation | Text chunks are converted into floating-point vectors (embeddings). |
| 4 | Search Execution | Query embeddings are compared against stored chunk vectors. |
| 5 | Ranking & Filtering | Results are sorted by a hybrid score and filtered by thresholds. |
Sources: crates/mcb-domain/tests/integration/semantic_search_workflow.rs:16-65
Result Score Fusion
The hybrid score is calculated by normalizing the BM25 score using a sigmoid function and applying weights to both BM25 and semantic components.
// Formula: hybrid = bm25_weight * bm25_normalized + semantic_weight * semantic_score
let bm25_score = index.scorer.score_with_tokens(document, query_terms);
let normalized_bm25 = 1.0 / (1.0 + (-bm25_score).exp());
let hybrid_score = self.bm25_weight * normalized_bm25 + self.semantic_weight * result.score;
Sources: crates/mcb-providers/src/hybrid_search/engine.rs:100-116, crates/mcb-providers/src/hybrid_search/mod.rs:25-26
Tooling and Interfaces
MCB exposes semantic search through specific MCP tools. These tools allow external agents (like Claude) to interact with the code index.
MCP Search Tools
| Tool Name | Purpose | Parameters |
|---|---|---|
search_code | Search indexed source files | query, limit, extensions, min_score |
search_memory | Search persistent agent memories | query, limit, tags, min_score |
Sources: crates/mcb-server/src/args/search.rs:107-159, README.md:65-71
Data Structures
The search system relies on core domain entities to represent code and its location.
pub struct SearchResult {
pub id: String,
pub file_path: String,
pub start_line: u32,
pub content: String,
pub score: f64,
pub language: String,
}
Sources: crates/mcb-domain/tests/integration/semantic_search_workflow.rs:141-148, crates/mcb-server/src/formatter/search.rs:64-75
Formatting and Presentation
Search results are processed by a formatter that generates Markdown-rich responses for the user, including syntax highlighting hints and relevance scores.
Code Preview Logic
The formatter truncates large snippets based on CODE_PREVIEW_MAX_LINES and determines the appropriate language hint (e.g., rust, python, javascript) based on file extensions.
Performance Metrics
If a search operation exceeds the SEARCH_SLOW_THRESHOLD_MS, the formatter appends a performance note suggesting more specific queries.
Sources: crates/mcb-server/src/formatter/search.rs:81-87
Summary
Semantic Code Search in MCB provides a powerful, multi-layered approach to codebase exploration. By integrating vector-based similarity for conceptual matching and BM25 for precise keyword relevance, it ensures high accuracy for developer queries. The system's clean architecture allows it to scale across different embedding providers and vector stores while maintaining a consistent interface for MCP clients.