Retrieval-Augmented Generation (RAG) Engine
Relevant source files
The following files were used as context for generating this wiki page:
The Retrieval-Augmented Generation (RAG) Engine is the core orchestration module of the KnowledgeVault platform. It manages the end-to-end lifecycle of a user query, from initial intent classification and multi-vector search to context distillation and final Large Language Model (LLM) response generation. The engine integrates diverse knowledge sources, including raw document chunks, user memories, compiled wiki claims, and Knowledge Management System (KMS) entries.
The system is designed to provide high-fidelity, grounded answers by prioritizing structured domain knowledge (Wiki/KMS) while falling back to raw document retrieval when necessary. It employs advanced retrieval techniques such as Reciprocal Rank Fusion (RRF) for multi-query variants, hybrid search (dense + FTS), and context window expansion to ensure maximum coherence and accuracy in technical documentation contexts.
Architecture and Orchestration
The RAG Engine operates as a central coordinator, delegating specific tasks to specialized services. It supports two primary operational modes: Thinking (high-latency, high-reasoning) and Instant (low-latency, optimized for speed).
Core Components
| Component | Description |
|---|---|
| Query Classifier | Determines query intent (e.g., entity lookup, procedural) to gate knowledge source selection. |
| Embedding Service | Converts text into vector representations for semantic search. |
| Vector Store | Performs similarity search using LanceDB for document chunks. |
| Memory Store | Retrieves durable user context (preferences and prior facts) via hybrid search. |
| Wiki/KMS Retrieval | Fetches high-confidence, structured claims from compiled knowledge tables. |
| Context Distiller | Deduplicates overlapping chunks and synthesizes complex evidence using LLMs. |
| Prompt Builder | Assembles the final LLM context with multi-namespace citations ([S#], [M#], [W#], [K#]). |
Sources: backend/app/services/rag_engine.py:82-155, specs/wire-wiki-chat/SPEC.md:144-156
Data Flow Overview
The following diagram illustrates the standard flow of a query through the RAG Engine.
The engine executes retrieval across different storage backends in parallel where possible, such as concurrent memory and document searches.
Sources: backend/app/services/rag_engine.py:465-535, specs/wire-wiki-chat/SPEC.md:144-156
Retrieval Strategies and Quality Optimization
Multi-Query Fusion (RRF)
To improve recall, the engine generates query variants (e.g., "step-back" or "hyde") and executes them concurrently. Results are merged using Reciprocal Rank Fusion (RRF).
- Exact-Match Promotion: If enabled, the original query's top-ranked dense result is nudged to the top 5 to ensure semantic accuracy isn't lost in fusion math.
- Recency Tiebreaking: Tiebreaking can be applied based on
processed_attimestamps to prioritize fresher documents.
Sources: backend/app/services/rag_engine.py:750-845, specs/rag-retrieval-quality-issue-11/SPEC.md:15-20
Knowledge Source Gating
The engine uses a "Wiki-first" approach for entity-heavy queries. Raw document RAG is skipped for entity_lookup queries if a high-confidence, fresh Wiki claim is found.
- Wiki Labels:
[W#]for compiled knowledge. - KMS Labels:
[K#]for curated documentation entries. - Source Labels:
[S#]for raw document chunks. - Memory Labels:
[M#]for user-specific context.
Sources: backend/app/services/rag_engine.py:65-80, specs/wire-wiki-chat/SPEC.md:158-163
Token Packing and Context Budgeting
The engine manages context using a reserved_best_fit strategy.
- Reserved Slots: The top-3 chunks are always included regardless of budget to prevent "lost-in-the-middle" issues for the most relevant data.
- Best-Fit: Subsequent chunks are evaluated; if one doesn't fit, it is skipped, but the engine continues checking smaller subsequent chunks until the
context_max_tokenslimit is reached.
Sources: backend/app/services/rag_engine.py:1260-1325, specs/rag-retrieval-quality-issue-11/SPEC.md:15-18
Detailed Logic: Retrieval Execution
The _execute_retrieval method handles the complex interaction between embeddings, vector stores, and evaluation.
Retrieval Metrics Table
| Parameter | Default | Function |
|---|---|---|
retrieval_top_k | 10 | The final number of chunks passed to the LLM. |
initial_retrieval_top_k | 20 | Number of chunks fetched before reranking. |
max_distance_threshold | null | Chunks with distance greater than this are dropped. |
hybrid_alpha | 0.6 | Weight distribution between dense (0.0) and FTS (1.0). |
token_pack_strategy | reserved_best_fit | Strategy for filling the LLM context window. |
Sources: backend/app/services/rag_engine.py:230-335, specs/rag-retrieval-quality-issue-11/SPEC.md:25-30
Context Enhancement
Parent Window Expansion
To provide the LLM with sufficient surrounding context for technical documents, the engine can expand small chunks into larger windows.
- Logic: It reads pre-computed
parent_window_textfrom metadata (computed during ingestion). - Rendering: The
PromptBuilderwraps the original matching chunk in[[MATCH: ...]]tags within the expanded parent text to guide LLM attention.
Sources: backend/app/services/rag_engine.py:1210-1235, specs/resolve-swarm-issues/SPEC.md:21-28
Anchor Best Chunk
To mitigate the "lost-in-the-middle" problem where LLMs ignore context in long prompts, the highest-scoring chunk can be repeated at the very end of the context block with a [BEST MATCH — repeated for emphasis] header.
Sources: specs/rag-retrieval-quality-issue-11/SPEC.md:38-41
Conclusion
The RAG Engine provides a robust, multi-layered retrieval architecture that balances speed and accuracy. By utilizing intent classification, multi-query fusion, and structured knowledge prioritization, it ensures that generated responses are grounded in the most relevant and coherent evidence available in the KnowledgeVault.
Sources: backend/app/services/rag_engine.py:615-690, specs/wire-wiki-chat/SPEC.md:46-60