Semantic Chunking & Context Expansion
Relevant source files
The following files were used as context for generating this wiki page:
Semantic Chunking and Context Expansion represent the architectural strategy for breaking down technical documentation into ingestible units and rebuilding relevant context during RAG retrieval. The system shifts from a token-based chunking approach to a character-based model to align backend processing with frontend configuration, ensuring that LLMs receive coherent, high-fidelity context for technical procedures and code blocks.
By implementing deterministic "adjacent-chunk windowing," the platform ensures that retrieved information includes surrounding context from the same document. This prevents the fragmentation of critical technical steps or data structures, allowing the generator to see a continuous flow of logic even when the vector search only identifies a specific snippet.
Chunking Architecture & Metadata
The ingestion pipeline transforms raw document text into chunks with specific character-based boundaries. To support context expansion, each chunk is enriched with stable identifiers that track its position within the source document.
Chunk Persistence Model
During ingestion, the following metadata fields are persisted for every chunk to enable windowing:
- doc_id: A stable identifier for the parent document.
- chunk_index: A 0-indexed sequence number representing the chunk's order in the document.
- chunk_uid: A composite unique identifier (e.g.,
doc_id:chunk_index). - source_path: The origin path of the file.
Sources: Plan.md:156-163, frontend/src/lib/api.ts:506-513
Logical Flow of Chunk Ingestion
The diagram below shows how raw text is processed into chunks with adjacent-chunk metadata.
Sources: Plan.md:156-163, Plan.md:183-195
Retrieval Context Expansion (Windowing)
Context expansion, or "adjacent-chunk windowing," rebuilds document coherence during the retrieval phase. This process goes beyond simple vector similarity by fetching neighboring chunks to provide the LLM with the full "window" of a technical procedure.
Expansion Logic
- Initial Fetch: The system retrieves the
top_kchunks based on vector distance. - Window Retrieval: For each initial hit, the system identifies adjacent indices based on the
retrieval_windowsetting (e.g., if window is 1, it fetcheschunk_index - 1andchunk_index + 1). - Deduplication & Sorting: The results are merged, unique'd by
chunk_uid, and sorted bychunk_index. - Capping: The final context is capped to
max_context_chunks.
Sources: Plan.md:165-176
Context Windowing Sequence
The sequence below illustrates the communication between the RAG Engine and the vector database to expand context.
Sources: Plan.md:165-180
Configuration & Units
The system enforces character-based units to eliminate semantic drift between the UI and the backend.
Core Configuration Settings
| Setting | Type | Default | Description |
|---|---|---|---|
chunk_size_chars | Integer | 2000 | The maximum character length of a single chunk. |
chunk_overlap_chars | Integer | 200 | Number of characters to overlap between adjacent chunks. |
retrieval_top_k | Integer | 12 | The initial number of chunks to retrieve via vector search. |
retrieval_window | Integer | 1 | Number of adjacent chunks to include on either side of a hit. |
max_distance_threshold | Float | 0.0 | Filter chunks where vector _distance > threshold (0 to disable). |
Sources: Plan.md:214-222, frontend/src/lib/api.ts:251-274
Technical Document Safety
To prevent fragmentation of critical technical elements, the system implements deterministic boundary checks during the chunking process.
Boundary Preservation
- Fenced Code Blocks: The system detects if a chunk ends inside a triple-backtick block (```). If it does, it appends the subsequent chunk until the block closes, up to a
max_merge_charslimit (default 8192). - Markdown Tables: Chunks are adjusted to avoid splitting lines that start with pipe characters (
|). - Instruction-Aware Prefixes: For specific models like Qwen3, prefixes are applied to both documents and queries to improve retrieval performance (e.g.,
Instruct: Represent this technical documentation...).
Sources: Plan.md:130-142, Plan.md:183-195
Conclusion
Semantic Chunking and Context Expansion ensure that the KnowledgeVault RAG pipeline remains optimized for technical documentation. By leveraging character-based constraints and adjacent-chunk metadata, the system maintains the integrity of complex code blocks and procedures, providing LLMs with the contiguous context required for accurate technical synthesis.