Getting Started / Quickstart

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• docker-compose.yml
• requirements.txt
• app/main.py
• app/core/config.py

The eval-platform provides a framework for evaluating large language models (LLMs) and AI agents. This guide explains how to set up the environment, install dependencies, and run the platform for the first time.

The platform relies on a containerized architecture to manage the web server, worker processes, and database connections. You can deploy the system using Docker or through a manual Python installation.

Sources: README.md:1-10, docker-compose.yml:1-5

System Prerequisites

Before installation, ensure your environment meets the following requirements:

• Docker & Docker Compose: Required for containerized deployment.
• Python 3.10+: Required for local development and running scripts.
• PostgreSQL: Used as the primary data store.
• Redis: Used for task queuing and caching.

Sources: docker-compose.yml:10-40, requirements.txt:1-5

Installation and Setup

Docker Deployment

The most efficient way to start the platform is using Docker Compose. This command builds the images and starts all necessary services, including the API, worker, and database.

docker-compose up --build

Sources: docker-compose.yml:1-10

Local Python Setup

If you prefer running the application directly on your host machine, install the dependencies listed in requirements.txt.

1. Create a virtual environment: python -m venv venv
2. Activate the environment: source venv/bin/activate
3. Install dependencies: pip install -r requirements.txt

Sources: requirements.txt:1-20

Service Architecture

The platform consists of three main components: the API Gateway, the Background Worker, and the Persistent Storage layer.

The diagram above shows the data flow from a user request through the API, into the task queue, and finally processed by the worker.

Sources: docker-compose.yml:5-50, app/main.py:1-15

Configuration

Configuration is managed through environment variables or a .env file. The app/core/config.py file defines the schema for these settings.

Variable	Description	Default
DATABASE_URL	Connection string for PostgreSQL	postgresql://user:pass@localhost/db
REDIS_URL	Connection string for Redis	redis://localhost:6379/0
API_KEY	Security key for authenticating LLM requests	None
LOG_LEVEL	Verbosity of system logs	INFO

Sources: app/core/config.py:10-50, docker-compose.yml:15-25

Running the API

The entry point for the web service is app/main.py. It uses FastAPI to serve endpoints for evaluation management.

# app/main.py
from fastapi import FastAPI
from app.api.v1 import api_router

app = FastAPI(title="Eval Platform API")
app.include_router(api_router, prefix="/api/v1")

Sources: app/main.py:1-10

Execution Flow

When you initiate an evaluation, the system follows this sequence:

Summary

Setting up the eval-platform involves configuring environment variables and launching the services via Docker or Python. Once the API and Worker are active, you can submit evaluation tasks through the REST interface to analyze LLM performance.

Sources: README.md:1-5, app/main.py:5-15