Introduction to AI Code Reviewer

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• CLAUDE.md
• packages/review/src/orchestrator.ts
• packages/review/src/prompts.ts
• packages/db/prisma/schema.prisma
• packages/ingestion/src/process-webhook.ts

AI Code Reviewer is a GitHub App designed to automate the pull request (PR) review process using advanced Language Models (LLMs). The system integrates directly with GitHub via webhooks to fetch diffs, analyze code changes, and post structured, actionable feedback as inline comments and PR summaries. It is built to move beyond simple pattern matching by incorporating Retrieval-Augmented Generation (RAG) to provide context-aware reviews based on existing codebase patterns, past review decisions, and repository-specific standards.

The system architecture is split between a web server for handling high-volume webhook intake and a background worker for executing long-running LLM and RAG operations. This ensures that the system remains responsive to GitHub's event delivery while performing complex analysis that can take 5–30 seconds per file.

Sources: README.md:3-8, README.md:104-109, CLAUDE.md:8-13

Core System Architecture

The AI Code Reviewer operates as a distributed system composed of two main processes sharing a single PostgreSQL database.

Process Decomposition

• Web Server (apps/web): A Next.js server that manages HTTP routes, GitHub OAuth, and webhook intake. It focuses on validating and storing incoming events before enqueuing them for asynchronous processing.
• Worker (apps/worker): A Node.js background process that handles heavy lifting, including GitHub API communication, LLM interaction, and vector database searches.

The following diagram illustrates the high-level workflow from a PR event to the posted review.

The system uses pg-boss as a job queue, leveraging the existing PostgreSQL instance to manage tasks without additional infrastructure like Redis.

Sources: README.md:104-124, README.md:132-136

The AI Review Pipeline

The review pipeline is orchestrated by the review.pr job. It follows a multi-step sequence to ensure that the LLM has maximum context while maintaining technical accuracy (e.g., avoiding comments on non-existent line numbers).

Execution Flow

1. Validation: Checks if the repository is enabled and if the PR author is on an ignore list.
2. File Filtering: Filters out removed files, binary files, and files matching ignorePaths glob patterns.
3. Retrieval (RAG): For each changed file, the system embeds the diff and retrieves relevant context from the vector database.
4. LLM Inference: Sends the diff and retrieved context to the LLM (e.g., Claude or GPT-4).
5. Post-Processing: Validates that LLM-suggested line numbers actually exist in the "new-file" side of the diff.
6. Publication: Posts a single COMMENT review to GitHub containing all valid inline comments and a summary.

Sources: packages/review/src/orchestrator.ts:51-160, README.md:214-230

Knowledge Layers (RAG)

To improve review quality, the system utilizes four layers of context:

Layer	Name	Purpose
Layer 1	Codebase Context	Finds similar functions/classes in the repo to identify inconsistencies.
Layer 2	Review Memory	References past accepted or dismissed review decisions.
Layer 3	Rules & Standards	Indexes README.md, CONTRIBUTING.md, and lint configs to cite team rules.
Layer 4	Dependency Graph	Analyzes the "blast radius" of changes using AST import parsing.

Sources: CLAUDE.md:23-74, packages/review/src/orchestrator.ts:133-149

Data Model & Schema

The system uses Prisma with PostgreSQL and the pgvector extension for storing both relational data and vector embeddings.

Key Data Entities

• AiReview: Tracks a specific review execution for a repository, PR number, and commit SHA.
• AiReviewComment: Stores individual inline findings including severity, body, and suggested code.
• ReviewMemory: A searchable unit of "past decisions" that links an LLM finding to its outcome (Accepted/Dismissed).
• RepoKnowledge: Stores factual codebase information extracted from developer-AI conversations.

Prompt Engineering & LLM Control

Prompts are centralized to allow tuning without changing core logic. The system uses structured JSON output for all LLM calls to ensure programmatically accessible results.

Prompt Components

• System Instructions: Define the persona ("expert AI code reviewer"), severity levels (INFO, WARNING, ERROR, CRITICAL), and strict formatting rules.
• Context Injection: PR title, description, and RAG-retrieved blocks (similar code, past decisions).
• Validation: LLM output is validated against Zod schemas (e.g., FileReviewOutputSchema) before processing.

export const FILE_REVIEW_SYSTEM = `You are an expert AI code reviewer. 
Rules:
- Only comment on lines present in the diff.
- Line numbers must be actual file line numbers.
- Severity guide: INFO, WARNING, ERROR, CRITICAL.
- Respond ONLY with valid JSON.`;

Sources: packages/review/src/prompts.ts:10-21, packages/review/src/llm-client.ts:98-115

Summary of Technical Decisions

The project prioritizes operational simplicity and accuracy through several specific design choices:

• Idempotency: Every webhook delivery is stored and checked against a processedAt timestamp to prevent duplicate reviews from GitHub redeliveries.
• AST Parsing: Uses tree-sitter for code chunking rather than regex, ensuring chunks represent complete semantic units like functions or classes.
• Comment Mode: Reviews are always posted in COMMENT mode rather than REQUEST_CHANGES to ensure the AI remains advisory and does not block developer workflows.
• Line Validation: The system parses unified diffs to generate a set of commentableLines, dropping any LLM suggestions that target invalid lines to prevent GitHub API errors.

Sources: README.md:243-255, packages/ingestion/src/index-repository.ts:107-115, packages/review/src/orchestrator.ts:172-180