Extending Support for New LLMs

Relevant source files

The following files were used as context for generating this wiki page:

• packages/llm/src/factory.ts
• packages/llm/src/types.ts
• packages/llm/src/providers/anthropic.ts
• packages/review/src/llm-client.ts
• README.md

The AI Code Reviewer project utilizes a provider-based abstraction to interact with Large Language Models (LLMs). This architecture allows the system to remain model-agnostic, supporting multiple providers such as OpenAI and Anthropic through a unified interface. By decoupling the core review logic from specific API implementations, developers can easily integrate new LLMs as they become available.

The system relies on a factory pattern to instantiate the appropriate provider based on configuration. This ensures that adding a new model only requires implementing a standard interface and updating the factory, without modifying high-level orchestrators or business logic.

Sources: README.md:280-285, packages/llm/src/factory.ts:1-10

LLM Architecture and Abstraction

The architecture is built around the LLMProvider interface. This interface defines the contract that any new LLM integration must fulfill, covering synchronous chat responses, streaming, and validation.

Core Interface: LLMProvider

Every provider must implement the following methods and properties:

• name: A string identifier for the provider (e.g., 'openai', 'anthropic').
• chat(request: ChatRequest): Processes a structured prompt and returns a ChatResult.
• chatStream(request: ChatRequest): Returns an AsyncIterable for streaming tokens.
• validate(): Checks if the provided credentials (API keys) and configuration are valid.
• estimateTokens(text: string): Provides a heuristic or exact count of tokens for a given string.

Sources: packages/llm/src/types.ts:25-39, packages/llm/src/providers/anthropic.ts:25-30

Provider Data Flow

The following diagram illustrates how the LLMProviderFactory generates a provider which is then consumed by the ReviewLLMClient to perform code reviews.

The ReviewLLMClient wraps the raw provider to add system-specific logic like JSON extraction and Zod schema validation.
Sources: packages/review/src/llm-client.ts:20-35, packages/llm/src/factory.ts:12-25

Implementation Steps for New Providers

To add support for a new LLM provider, follow these structured steps:

1. Define Provider Configuration

Update the LLMProviderConfig type to include a new discriminant for the new provider.

// Example configuration structure
export type LLMProviderConfig =
  | { kind: 'openai'; apiKey: string; baseUrl?: string; model?: string }
  | { kind: 'anthropic'; apiKey: string; baseUrl?: string; model?: string }
  | { kind: 'new_provider'; apiKey: string; model: string };

Sources: packages/llm/src/types.ts:10-20

2. Create the Provider Class

Create a new file in packages/llm/src/providers/. The class must implement the LLMProvider interface.

Method	Responsibility	Implementation Note
chat	Execute completion	Must map external API usage to inputTokens and outputTokens.
chatStream	Stream tokens	Should yield individual string chunks.
validate	Health check	Usually a small request (e.g., max_tokens: 1) to verify the API key.
estimateTokens	Token counting	Providers often have different char-to-token ratios (e.g., ~4 chars/token for Anthropic).

Sources: packages/llm/src/providers/anthropic.ts:32-110, packages/llm/src/types.ts:41-55

3. Update the Factory

Modify packages/llm/src/factory.ts to include the new provider in the create method.

export class LLMProviderFactory {
  static create(config: LLMProviderConfig): LLMProvider {
    switch (config.kind) {
      case 'openai':
        return new OpenAIProvider(config);
      case 'anthropic':
        return new AnthropicProvider(config);
      case 'new_provider':
        return new NewProvider(config);
      default:
        throw new Error(`Unsupported provider kind`);
    }
  }
}

Sources: packages/llm/src/factory.ts:12-25

Data Structures and Types

The system uses standardized types for requests and responses to ensure compatibility across different models.

Chat Request and Result

Type	Fields	Description
ChatRequest	messages, model, temperature, maxTokens, stop	The input payload for the LLM.
ChatMessage	role ('system', 'user', 'assistant'), content	Individual turns in the conversation.
ChatResult	content, inputTokens, outputTokens, finishReason	The structured output from the provider.

Sources: packages/llm/src/types.ts:60-95

Class Diagram

The relationship between the interfaces and concrete implementations is shown below:

Usage in Review Pipeline

The ReviewLLMClient consumes the LLMProvider. When extending support, ensure the new LLM can handle high-temperature tasks like PR summarization and low-temperature tasks like structured JSON code review. The ReviewLLMClient uses extractJson to strip markdown fences from the LLM's response before parsing it with Zod schemas.

Sources: packages/review/src/llm-client.ts:40-60, packages/review/src/llm-client.ts:85-100

Conclusion

Extending the AI Code Reviewer with new LLMs is a streamlined process involving the implementation of the LLMProvider interface and registration in the LLMProviderFactory. This modular approach ensures that the system can leverage the latest advancements in AI models with minimal friction.