Quick Start and Setup

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• AGENTS.md
• CONTRIBUTING.md
• CLAUDE.md
• GEMINI.md
• litellm/integrations/gitlab/README.md
• litellm-proxy-extras/migration_runbook.md

LiteLLM provides a unified interface for over 100 LLMs, translating inputs to provider-specific endpoints while maintaining a consistent OpenAI-format output. It includes a core library for programmatic integration and a Proxy Server (LLM Gateway) for centralized management of authentication, budgets, and rate limits.

The setup process involves preparing the local development environment, installing core or proxy-specific dependencies, and configuring LLM providers via environment variables or YAML files.

Sources: AGENTS.md:5-10, GEMINI.md:31-35

Local Development Environment Setup

Setting up the project for development requires cloning the repository and installing dependencies using make. LiteLLM utilizes poetry for environment management.

Installation Steps

1. Repository Setup: Fork and clone the repository.
2. Environment Preparation: Install core or proxy-specific dependencies.
3. Database Initialisation: For proxy development, the system runs Prisma migrations on boot.

Sources: CONTRIBUTING.md:43-58, AGENTS.md:188-193

Command	Description
make install-dev	Installs core development dependencies.
make install-proxy-dev	Installs proxy dependencies with a full feature set.
make install-test-deps	Installs dependencies required for running the test suite.

Sources: GEMINI.md:5-10, CLAUDE.md:5-10

Dependency and Runtime Environment

LiteLLM is optimized for Python 3.12. The virtual environment is typically managed via Poetry and stored in ~/.cache/pypoetry/virtualenvs/.

Sources: AGENTS.md:183-186

Proxy Server Configuration

The LiteLLM Proxy Server (LLM Gateway) acts as a FastAPI application that handles routing to various LLM providers.

Execution Flow

The following diagram illustrates the startup and request flow for the proxy server:

The proxy requires approximately 15-20 seconds to fully start as it executes database migrations.
Sources: AGENTS.md:188-193, GEMINI.md:42-45

Running the Proxy

To start the proxy with a specific configuration:

poetry run litellm --config dev_config.yaml --port 4000

Sources: AGENTS.md:195-197

Integration with External Providers

LiteLLM supports seamless integration with various providers such as GitLab for prompt management or standard LLM providers (OpenAI, Anthropic, Bedrock).

GitLab Prompt Management Setup

GitLab integration allows fetching .prompt files from repositories for team-based access control.

import litellm

gitlab_config = {
    "project": "org/repo",
    "access_token": "your-token",
    "base_url": "gitlab-url",
    "prompts_path": "src/prompts",
    "branch": "main"
}

litellm.set_global_gitlab_config(gitlab_config)

Sources: litellm/integrations/gitlab/README.md:41-52

Environment Variables

Each provider requires specific API keys. For example, GitLab integration uses access_token, while the Proxy server can manage multiple provider keys via a config.yaml file.

Sources: litellm/integrations/gitlab/README.md:144-154, AGENTS.md:155-160

Testing and Verification

Verification is a critical step in the setup process to ensure the translation logic and proxy routing function correctly.

Core Testing Commands

Type	Command	Purpose
Unit Tests	make test-unit	Runs mocked tests in tests/test_litellm/ using 4 parallel workers.
Integration	make test-integration	Tests translation against real provider APIs.
Linting	make lint	Runs Ruff, MyPy, and Black checks.

Sources: GEMINI.md:13-22, CLAUDE.md:13-22

Test Structure Sequence

The testing architecture follows a structured flow from local unit checks to integration:

Database Migrations (Engineers Only)

For developers modifying the database schema (schema.prisma), a specific runbook is followed to ensure consistency across the root and proxy packages.

1. Sync Schema: Ensure schema.prisma in the repo root matches litellm/proxy/schema.prisma and litellm-proxy-extras/litellm_proxy_extras/schema.prisma.
2. Generate Migration: Use a temporary PostgreSQL instance to generate SQL files.

python ci_cd/run_migration.py "migration_name"

Sources: litellm-proxy-extras/migration_runbook.md:3-23

LiteLLM's setup ensures a robust development environment by combining unified LLM access with comprehensive tooling for management, testing, and scaling. Programmatic usage via litellm.completion() or deployment via the LLM Proxy allows for flexible architectural integration.

Sources: AGENTS.md:5-15, GEMINI.md:31-35