Customizing RAG Layers
Relevant source files
The following files were used as context for generating this wiki page:
The AI Code Reviewer utilizes a multi-layered Retrieval-Augmented Generation (RAG) system to provide context-aware feedback on Pull Requests. Instead of analyzing diffs in isolation, the system retrieves information from four distinct knowledge layers: codebase context, review memory, rules/standards, and dependency graphs. This architectural approach ensures that AI comments are consistent with existing patterns, past human decisions, and established project standards.
Sources: CLAUDE.md:15-18, packages/review/src/orchestrator.ts:140-143
RAG Architecture Overview
The system orchestrates retrieval from multiple layers in parallel during the review process. Each layer serves a specific purpose, ranging from identifying local code patterns to adhering to high-level architectural constraints.
The Four Layers of Context
| Layer | Name | Purpose | Data Source |
|---|---|---|---|
| Layer 1 | Codebase Context | Finds similar functions/classes to flag inconsistencies. | Indexed repository code (AST-parsed). |
| Layer 2 | Review Memory | Learns from past accepted/dismissed review decisions. | ReviewMemory and ReviewConversation tables. |
| Layer 3 | Rules and Standards | Cites project documentation and linting configs. | README.md, CONTRIBUTING.md, ADRs. |
| Layer 4 | Dependency Graph | Understands "blast radius" of changes. | Tree-sitter parsed import graphs. |
Sources: CLAUDE.md:50-108, packages/db/prisma/schema.prisma:500-650
Retrieval Flow
The orchestrator manages the retrieval of these layers using an embedding of the PR diff as the query vector.
The diagram shows how a diff patch is transformed into a vector to trigger simultaneous searches across different RAG layers before being compressed into the final prompt context.
Sources: packages/review/src/orchestrator.ts:145-165, CLAUDE.md:130-155
Layer 1: Codebase Context
This layer is the highest priority and focuses on identifying similar existing patterns within the same repository. It uses Tree-sitter for AST parsing to chunk code at meaningful boundaries like functions or classes, rather than arbitrary character counts.
- Trigger: Triggered when a repository is enabled or upon every push webhook for incremental re-indexing.
- Metadata: Chunks store
function_name,file_path, and line boundaries to allow the LLM to provide specific references. - Retrieval: Returns the top 5-10 similar functions from the same
repo_id.
Sources: CLAUDE.md:52-70, packages/review/src/orchestrator.ts:153-157
Layer 2: Review Memory & Knowledge
Review Memory creates a "compounding moat" by storing the outcome of every AI review. This allows the system to recognize if a specific pattern was previously flagged and subsequently accepted or dismissed by the engineering team.
Memory Storage and Feedback Loop
The system tracks developer interactions via the ReviewMemoryOutcome enum:
- ACCEPTED: Developer acknowledged the issue and applied a fix.
- DISMISSED: Developer explicitly ignored the finding, indicating it might be a false positive or intentional.
This sequence illustrates how developer feedback is processed to update the system's internal memory or extract new codebase knowledge.
Sources: packages/review/src/comment-processor.ts:40-90, packages/db/prisma/schema.prisma:580-610
Data Structures for Memory
// From packages/db/prisma/schema.prisma
enum ReviewMemoryOutcome {
PENDING
ACCEPTED
DISMISSED
}
enum KnowledgeKind {
ARCHITECTURE
CONVENTION
FEATURE
CONSTRAINT
TERMINOLOGY
}
Sources: packages/db/prisma/schema.prisma:575-585, packages/db/prisma/schema.prisma:615-625
Layer 3: Rules and Standards
This layer performs hybrid search (BM25 + Semantic) over the repository's documentation. It is designed to capture specific, keyword-heavy rules like "never use eval" which might be missed by purely semantic vector search.
- Indexed Content:
CONTRIBUTING.md,README.md,.eslintrc, and Architecture Decision Records (ADRs). - Implementation: At review time, the system retrieves the top 3 relevant rules and injects them into the prompt.
Sources: CLAUDE.md:84-93, packages/review/src/prompts.ts:98-106
Prompt Construction and Injection
The ReviewLLMClient uses specific builder functions to format the retrieved context before sending it to the LLM. This ensures that context from different layers is clearly delineated for the model.
// packages/review/src/prompts.ts:114-125
export function buildRepoKnowledgeContext(knowledge: Array<{
content: string;
kind: string;
}>): string {
if (knowledge.length === 0) return '';
const lines = knowledge.map((k) => `- [${k.kind}] ${k.content}`);
return `\n\n## Codebase knowledge for this repo (learned from developer feedback):\n${lines.join('\n')}`;
}
The buildFileReviewPrompt function assembles the full context, including the prTitle, similarCode (Layer 1), pastReviewsContext (Layer 2), and repoKnowledgeContext (Layer 2/3).
Sources: packages/review/src/prompts.ts:35-80, packages/review/src/llm-client.ts:80-95
Technical Implementation Details
The implementation relies heavily on pgvector for storing embeddings and performing similarity searches.
- Vector Filtering: Every query must be filtered by
repositoryIdto prevent cross-tenant context leakage. - Parallelism: Retrieval for all active layers is executed using
Promise.allin the orchestrator to minimize latency. - Consistency: The system uses
ReviewSeverity(INFO, WARNING, ERROR, CRITICAL) to categorize findings across all layers.
Sources: packages/review/src/orchestrator.ts:145-160, packages/db/prisma/schema.prisma:10-15, packages/review/src/schemas.ts:13-15
Summary
Customizing RAG layers allows the AI Reviewer to move beyond basic diff analysis into a sophisticated understanding of project-specific context. By integrating codebase search, historical review memory, and explicit project rules, the system provides feedback that mirrors a senior engineer's knowledge of the repository's history and standards.