Cloud Deployment Guide
Relevant source files
The following files were used as context for generating this wiki page:
The Cloud Deployment Guide provides instructions for deploying the Evaluation Platform (eval-platform) to cloud environments. It outlines the necessary infrastructure, container orchestration, and environment configurations required to host the platform effectively.
This guide covers the containerized architecture used for production deployments. It ensures consistent environments between local development and cloud hosting providers.
Deployment Architecture
The Evaluation Platform uses a microservices-based architecture managed by Docker Compose. This structure allows for independent scaling of the web application, worker processes, and database layers.
Component Overview
The platform consists of several interconnected services that handle user requests, evaluation tasks, and data storage.
The diagram above illustrates the request flow from the user through the gateway to the core services and backing stores.
Sources: docker-compose.yml
Service Definitions
The deployment utilizes the following primary services:
| Service | Responsibility | Technology |
|---|---|---|
web | Serves the user interface and API endpoints. | Python/FastAPI |
worker | Processes long-running evaluation tasks asynchronously. | Celery |
db | Persists user data, evaluation results, and logs. | PostgreSQL |
cache | Manages task queues and temporary session data. | Redis |
Sources: docker-compose.yml, README.md
Environment Configuration
Before deploying, you must configure the environment variables to match your cloud environment's security and connectivity requirements.
Required Variables
The platform relies on a .env file to manage secrets and connection strings. Copy the template from .env.example to get started.
DATABASE_URL=postgresql://user:password@db:5432/eval_db
REDIS_URL=redis://cache:6379/0
SECRET_KEY=your_secure_random_string
OPENAI_API_KEY=sk-...
Sources: .env.example
Deployment Procedure
Deploying to a cloud instance involves preparing the host machine, configuring dependencies, and launching the containers.
Prerequisites
Ensure the target cloud instance (e.g., AWS EC2, GCP Compute Engine, Azure VM) meets the following requirements:
- Docker Engine 20.10+ installed.
- Docker Compose V2 installed.
- Network ports 80 and 443 open for inbound traffic.
Sources: README.md
Execution Steps
- Clone the repository: Download the source code to the cloud instance.
- Configure environment: Create the
.envfile based on the provided example. - Run the deployment script: Execute the automated setup script.
# Set permissions and run the deployment script
chmod +x scripts/deploy.sh
./scripts/deploy.sh
Sources: scripts/deploy.sh
Deployment Logic
The deploy.sh script automates the standard container lifecycle.
The sequence shows the script pulling the latest images, building local changes, and running database migrations to ensure the schema is current.
Sources: scripts/deploy.sh, docker-compose.yml
Scaling and Maintenance
The evaluation platform supports horizontal scaling of the worker service to handle increased load during concurrent evaluation runs.
Scaling Workers
To increase processing capacity, use the Docker Compose scale command:
docker-compose up -d --scale worker=3
This command launches three separate worker containers, all consuming tasks from the shared Redis queue.
Sources: docker-compose.yml, README.md
Summary
The Evaluation Platform deployment utilizes Docker Compose for a portable and reliable cloud presence. By configuring the .env file and utilizing the deploy.sh script, administrators can quickly provision the web, worker, and database services required for large-scale LLM evaluation.