FTS5 Hybrid Search

Relevant source files

The following files were used as context for generating this wiki page:

• code_review_graph/search.py
• tests/test_search.py
• code_review_graph/main.py
• code_review_graph/tools/query.py
• tests/test_integration_v2.py
• code_review_graph/tools/docs.py

FTS5 Hybrid Search is a specialized retrieval engine within the code-review-graph project designed to find code entities (Functions, Classes, Files, etc.) by combining traditional full-text keyword matching with semantic vector similarity. It leverages SQLite's FTS5 extension to index node names, qualified names, and signatures, providing a fast and token-efficient alternative to broad codebase grepping.

Sources: CLAUDE.md, code_review_graph/main.py:164-184, code_review_graph/search.py

Architecture and Components

The search system architecture integrates multiple ranking signals into a single unified result list. It primarily operates on a dedicated virtual table, nodes_fts, which stores searchable text data indexed from the primary nodes table.

Data Flow Overview

The following diagram illustrates how a user query is processed through the hybrid search pipeline:

The system processes queries by simultaneously running a keyword search and an optional semantic vector search. These results are merged using Reciprocal Rank Fusion (RRF) and then adjusted based on "Kind" heuristics (e.g., boosting Class results for PascalCase queries) and contextual relevance.
Sources: code_review_graph/search.py, tests/test_search.py:101-140

Search Methodology

FTS5 Indexing

The system maintains an FTS5 virtual table called nodes_fts. This table indexes the name, qualified_name, and signature of every node in the graph. The index rebuild process is atomic, utilizing a transaction to drop, recreate, and repopulate the table to ensure database integrity.
Sources: code_review_graph/search.py, tests/test_search.py:220-234

Reciprocal Rank Fusion (RRF)

To combine results from different search providers (Keyword vs. Semantic), the project implements rrf_merge. This algorithm calculates a combined score based on the rank of a node in each result list rather than their raw scores, ensuring that items appearing high in both lists are prioritized.

Parameter	Type	Description
lists	list[list[tuple[int, float]]]	Multiple lists of (node_id, score) pairs to merge.
k	int	The RRF constant (default 60) used to prevent low-rank items from overwhelming the score.

Sources: code_review_graph/search.py, tests/test_search.py:118-140

Query Boosting and Heuristics

The search engine applies several heuristics to improve relevance without requiring complex natural language processing:

• PascalCase Detection: If a query follows PascalCase (e.g., UserService), the engine applies a multiplier boost to nodes of kind Class.
• snake_case Detection: Queries using snake_case (e.g., get_users) trigger a boost for nodes of kind Function.
• Qualified Name Boosting: Dotted queries (e.g., api.get_users) provide a _qualified boost to prioritize matches in the qualified_name column.
• Contextual Boosting: If context_files are provided (e.g., currently open files in an IDE), nodes located within those files receive a score multiplier.

Sources: code_review_graph/search.py, tests/test_search.py:101-116, code_review_graph/tools/query.py:202-212

Key Implementation Details

Hybrid Search Functionality

The hybrid_search function is the primary entry point. It handles the fallback to standard SQL LIKE matching if the FTS5 extension or table is unavailable.

# code_review_graph/search.py

def hybrid_search(
    store: GraphStore,
    query: str,
    kind: str | None = None,
    limit: int = 20,
    context_files: list[str] | None = None,
    model: str | None = None,
    provider: str | None = None,
) -> list[dict]:
    # 1. Clean query and detect boosts
    # 2. Execute FTS5 keyword search
    # 3. Execute Vector search (if EmbeddingStore available)
    # 4. Merge results using RRF
    # 5. Apply kind-based and context-based multipliers
    # 6. Retrieve node details and return

Sources: code_review_graph/search.py, tests/test_search.py:192-202

Search Result Data Model

Search results return a standardized dictionary containing both metadata and the relevance score:

Field	Type	Description
name	str	Short name of the entity.
qualified_name	str	Fully qualified name including file path.
kind	str	Type of node (Function, Class, File, Test, Type).
file_path	str	Path to the source file.
line_start	int	Starting line number.
signature	str	The indexed code signature/definition.
score	float	Final calculated relevance score.

Sources: tests/test_search.py:169-183, code_review_graph/search.py

Integration with MCP

The FTS5 Hybrid Search is exposed to AI agents via the semantic_search_nodes_tool. This tool is frequently recommended by the system as a first step for "Exploring code" or "Finding functions/classes by name" to minimize token usage compared to grep or glob.
Sources: code_review_graph/main.py:164-184, skills/explore-codebase/SKILL.md

Summary

The FTS5 Hybrid Search system provides a robust, multi-modal retrieval mechanism that is central to the project's goal of token-efficient code review. By combining SQLite's keyword indexing with modern vector similarity and case-sensitive heuristics, it ensures that developers and AI agents can accurately locate relevant code structures within large, complex repositories.