LLM Provider Integration

Relevant source files

The following files were used as context for generating this wiki page:

• packages/models/src/providers/registry.ts
• packages/models/src/facade.ts
• packages/core/src/clients/openrouter/openrouterClient.ts
• packages/core/src/messages/api/sendAI/sendAI.ts
• packages/core/README.md
• mcp-server/src/index.ts
• packages/core/src/utils/llm-models/pricing.ts

The LLM Provider Integration system provides a unified, abstract interface for interacting with various Large Language Model (LLM) providers. It is designed to decouple the core application logic from the specifics of any single AI provider's API, enabling flexible model selection, centralized configuration, and consistent request handling. The system manages multiple providers like OpenAI, Anthropic, and OpenRouter through a central registry and facade.

This architecture simplifies making AI requests by handling provider-specific client instantiation, authentication, and request formatting. It also incorporates features like automatic retries with exponential backoff, cost tracking, and structured response processing. This allows different parts of the application, such as the Autonomous Agent Architecture, to consume LLM capabilities without needing to know the underlying implementation details of each provider.

Sources: packages/core/README.md:20-24, 114-123

Architecture Overview

The system is layered to abstract away the complexity of dealing with multiple LLM providers. A request flows from the application through a facade, which uses a registry to find the correct provider client. This client then formats and sends the request to the external LLM API.

The following diagram illustrates the high-level data flow for an AI request.

Core Components

Provider Registry

The ProviderRegistry is a singleton class that serves as a central repository for all available LLM provider clients. It is responsible for initializing and managing instances of providers like OpenAI, Anthropic, and OpenRouter. The registry ensures that each provider is instantiated only once with the necessary API keys, which are retrieved from environment variables.

Key responsibilities:

• Registering all supported LLM providers.
• Lazily initializing provider clients on first request.
• Providing a get method to retrieve a specific provider client by name.
• Handling API key resolution from environment variables.

Sources: packages/models/src/providers/registry.ts:13-79

The diagram below shows the structure of the ProviderRegistry and its relationship with the provider clients.

Model Facade

The ModelFacade class provides a simplified, high-level API for interacting with the LLM provider ecosystem. It abstracts the process of selecting a provider, instantiating a client, and making a generation request. It uses the ProviderRegistry to access the underlying provider clients.

The primary method is generate, which takes a provider name and generation parameters, and returns the model's output.

Sources: packages/models/src/facade.ts:10-33

The sequence diagram below shows a typical call to ModelFacade.generate.

Unified AI Request Handling

The sendAI function in packages/core/src/messages/api/sendAI/sendAI.ts is the core orchestrator for sending requests to LLMs. It acts as a wrapper around the provider-specific clients and adds a layer of resilience and control.

Its key features include:

• Automatic Retries: Implements an exponential backoff strategy for handling transient errors or rate limits.
• Cost Tracking: Calculates and returns the USD cost of the API call.
• Structured Response: Processes the raw API response into a standardized ProcessedResponse format, which includes the message, tool calls, and usage metrics.
• Model Selection: Can dynamically select models based on categories like nano, medium, and high.

Sources: packages/core/src/messages/api/sendAI/sendAI.ts:25-108, packages/core/README.md:214-221

The sendAI function's logic is outlined in the flowchart below.

Provider-Specific Clients

Each LLM provider has a dedicated client that implements the logic for communicating with its specific API. For example, openrouterClient.ts handles interactions with the OpenRouter service.

These clients are responsible for:

• Formatting the request body and headers.
• Handling provider-specific authentication (e.g., adding HTTP-Referer and X-Title headers for OpenRouter).
• Calling the create function from the Vercel AI SDK to get a provider-specific instance.

Sources: packages/core/src/clients/openrouter/openrouterClient.ts:9-30

The following snippet shows how the OpenRouter client is created, including its specific headers.

// packages/core/src/clients/openrouter/openrouterClient.ts:9-21
export const getOpenRouterClient = (apiKey: string) => {
  if (!apiKey) {
    throw new Error("OpenRouter API key is required.")
  }
  return openrouter({
    apiKey,
    headers: {
      "HTTP-Referer": "https://lucky.dev",
      "X-Title": "lucky.dev",
    },
    // ...
  })
}

Sources: packages/core/src/clients/openrouter/openrouterClient.ts:9-21

Configuration and Authentication

Authentication with LLM providers is handled via API keys, which are configured as environment variables. The system reads these keys at startup to initialize the provider clients. The mcp-server is capable of extracting API keys from either environment variables or HTTP headers.

Sources: packages/mcp-server/src/index.ts:27-42, packages/models/src/providers/registry.ts:47-78

Environment Variables

The following table lists the key environment variables used for configuring LLM providers.

Variable	Description	Example	Source File
LUCKY_API_KEY	The primary API key for the Lucky service, also used for self-hosted instances.	fc-YOUR_API_KEY	mcp-server/README.md
LUCKY_API_URL	Optional custom endpoint for self-hosted Lucky instances.	https://lucky.your-domain.com	mcp-server/README.md
OPENROUTER_API_KEY	API key for the OpenRouter service.	sk-or-v1...	packages/models/src/providers/registry.ts:60
OPENAI_API_KEY	API key for the OpenAI service.	sk-...	packages/models/src/providers/registry.ts:50
ANTHROPIC_API_KEY	API key for the Anthropic service.	sk-ant-...	packages/models/src/providers/registry.ts:70

Sources: mcp-server/README.md, packages/models/src/providers/registry.ts:47-78

Header-Based Authentication

For services running in a cloud environment, the mcp-server can extract the API key from HTTP headers, providing a more flexible authentication mechanism than environment variables alone.

The server checks the following headers in order:

1. x-lucky-api-key or x-api-key
2. Authorization: Bearer <token>

Sources: mcp-server/src/index.ts:27-42

Model and Pricing Management

The system includes a centralized registry for model pricing information. This allows for accurate cost tracking of all LLM operations. The pricing data is stored in packages/core/src/utils/llm-models/pricing.ts and is used by the calculateCost function within the sendAI request handler.

The project also defines a model selection strategy, categorizing models into nano, medium, and high tiers to balance cost and capability.

Tier	Default Model	Use Case
nano	gemini-2.5-flash	Fast operations, summaries, validation
medium	gpt-4o-mini	Standard processing, improvement
high	claude-3-5-sonnet-20241022	Complex reasoning, crossover

Sources: packages/core/README.md:214-221, packages/core/src/utils/llm-models/pricing.ts, packages/core/src/messages/api/sendAI/sendAI.ts:102-104