Deployment Guide
Relevant source files
The following files were used as context for generating this wiki page:The monGARS project is designed for flexible deployment across various environments, from local development workstations to containerized clusters. The primary methods of deployment involve using Docker Compose for orchestrating the full stack of services and helper scripts for automating setup and validation. For more advanced use cases, Kubernetes manifests are also provided.
This guide covers the main deployment workflows, service architecture, configuration, and validation tooling. The system can be run locally using Python/Uvicorn, packaged via Docker/Compose, or deployed to Kubernetes. Optional profiles for ML-specific tasks like inference and training are also available within the Docker Compose setup.
Sources: README.md:120-122, 240-247, monGARS_structure.txt:51-52
Docker Compose Deployment
The most common method for running the complete monGARS stack is through Docker Compose. The project includes an interactive menu script to simplify this process, alongside a detailed docker-compose.yml file that defines the services and their interactions.
Sources: README.md:126-128, docker-compose.yml
Interactive Docker Menu
The scripts/docker_menu.py script provides a user-friendly terminal interface for managing the Docker Compose stack. It automates several setup and maintenance tasks to ensure a smooth deployment experience.
Key features of the Docker menu include:
- Environment Bootstrapping: On the first run, it creates a
.envfile from the.env.exampletemplate and populates it with strong, randomly generated secrets forSECRET_KEY,DJANGO_SECRET_KEY,DB_PASSWORD, andVAULT_TOKEN. - Port Management: It scans for port collisions for all exposed services (API, webapp, Postgres, Redis, etc.) and automatically finds the next available port, persisting the changes to the
.envfile. - Profile Selection: It allows the user to enable optional profiles for
inference(Ollama) andray(Ray orchestration) before starting the stack. - Lifecycle Management: It provides standard commands to deploy, start, stop, restart, view logs, and destroy the stack.
- Diagnostics and Healing: It includes a diagnostic tool to check for common issues and an "auto-heal" function to restart unhealthy services.
Sources: scripts/docker_menu.py:122-136, 258-283, 706-710, 786-799, README.md:129-152
The menu offers the following options:
| Option | Description |
|---|---|
| 1 | Deploy/Update stack (build + up): Builds images and starts the stack. |
| 2 | Start stack without rebuild: Starts the stack using existing images. |
| 3 | Stop running services: Stops all services without removing them. |
| 4 | Restart services: Restarts specified or all services. |
| 5 | Show service status: Displays the current status of all services. |
| 6 | Tail service logs: Streams logs from specified or all services. |
| 7 | Rebuild images: Forces a rebuild of all service images. |
| 8 | Pull latest images: Pulls the latest base images for services. |
| 9 | Open shell inside a service: Executes a command (e.g., bash) in a running container. |
| 10 | View resolved docker-compose config: Prints the fully resolved Compose configuration. |
| 11 | Destroy stack (containers + volumes): Stops and removes all containers, networks, and volumes. |
| 12 | Run diagnostics & auto-remediation checks: Validates the environment and configuration. |
| 13 | Auto-heal unhealthy services: Restarts services that are not in a healthy state. |
| 14 | Show environment variable summary: Displays the current .env values with secrets masked. |
| 15 | Fine-tune base model and install wrapper: Launches the LLM fine-tuning pipeline. |
Sources: scripts/docker_menu.py:904-929
Service Architecture
The docker-compose.yml file defines the core services, their dependencies, and networking. The diagram below illustrates the relationships between the main components.
This diagram shows the dependencies between services as defined in docker-compose.yml.
Sources: docker-compose.yml:1-300
The services can be categorized as follows:
| Service | Description |
|---|---|
postgres | The primary database using pgvector for storing application data and vector embeddings. |
redis | A Redis instance used for caching and message queuing. |
mlflow | An MLflow server for tracking machine learning experiments. |
vault | A HashiCorp Vault dev server for managing secrets. |
migrations | A one-off job that runs database migrations (init_db.py) before the API starts. |
api | The core FastAPI application that serves the main API and WebSocket endpoints. |
webapp-migrations | A one-off job that runs Django-specific database migrations. |
webapp | The Django-based operator console and web interface. |
nginx | An Nginx reverse proxy that routes traffic to the api and serves static files for the webapp. |
unsloth-trainer | (Optional training profile) A service to run the Unsloth fine-tuning pipeline. |
ollama | (Optional inference profile) An Ollama service for running local LLMs. |
ray-head / rayserve | (Optional ray profile) Services for distributed ML serving with Ray Serve. |
Sources: docker-compose.yml:2-300, README.md:158-165
Health Checks
Services are configured with health checks to ensure they are running correctly before dependent services start. For example, the api service waits for migrations to complete, and the webapp waits for the api to start.
The api service health check is defined as:
# docker-compose.yml:138-143
healthcheck:
test:
- "CMD-SHELL"
- |
python3 -c "import http.client,sys; c=http.client.HTTPConnection('127.0.0.1',8000,timeout=5); c.request('GET','/healthz'); sys.exit(0 if c.getresponse().status==200 else 1)"
interval: 30s
timeout: 10s
retries: 5
Sources: docker-compose.yml:138-143
Automated Deployment Scripts
The repository contains several scripts to automate parts or all of the deployment process.
Full-Stack Visual Deployer
The scripts/full_stack_visual_deploy.py script provides a step-by-step guided deployment for the entire stack. It handles prerequisite checks, dependency installation, environment configuration, and container orchestration.
The deployment flow is as follows:
Fine-Tune and Deploy Launcher
For ML-focused workflows, the scripts/finetune_deploy_launch.py script automates a complete cycle of fine-tuning a model, updating the model registry, deploying the stack, and launching the web UI.
This script's workflow is:
Deployment Validation
The scripts/deployment_simulator.py script is a tool for validating deployment configurations without needing a running Docker or Kubernetes environment. It performs static checks on configuration files and orchestration manifests to catch common issues early.
The simulator performs the following checks:
- Application Settings: Loads settings from
.envand validatesSECRET_KEY, JWT configuration, andDATABASE_URL. - Docker Compose Files: Parses
docker-compose.yml, checking for syntax errors, missing image/build directives, missingenv_filereferences, and host port conflicts. - Kubernetes Manifests: Parses Kubernetes
Deploymentmanifests, checking for missing container images, resource limits/requests, and health probes (livenessProbe,readinessProbe).
Sources: scripts/deployment_simulator.py:72-80, 194-229, 276-335
Configuration
Deployment is primarily configured through environment variables defined in a .env file at the project root. Scripts like docker_menu.py and full_stack_visual_deploy.py help manage this file.
Key Environment Variables
| Variable | Description | Default | Source |
|---|---|---|---|
SECRET_KEY | JWT signing key for the FastAPI app. | Rotated automatically | scripts/docker_menu.py:31 |
DJANGO_SECRET_KEY | Secret key for the Django webapp. | Rotated automatically | scripts/docker_menu.py:32 |
DB_PASSWORD | Password for the PostgreSQL database user. | Rotated automatically | scripts/docker_menu.py:33 |
API_PORT | Host port for the FastAPI service. | 8000 | docker-compose.yml:137 |
WEBAPP_PORT | Host port for the Django webapp. | 8001 | docker-compose.yml:181 |
POSTGRES_PORT | Host port for the PostgreSQL database. | 5432 | scripts/docker_menu.py:270 |
REDIS_PORT | Host port for Redis. | 6379 | docker-compose.yml:25 |
MLFLOW_PORT | Host port for the MLflow server. | 5000 | docker-compose.yml:43 |
VAULT_PORT | Host port for the Vault dev server. | 8200 | docker-compose.yml:62 |
OLLAMA_PORT | Host port for the Ollama service. | 11434 | docker-compose.yml:266 |
RAY_HTTP_PORT | Host port for the Ray Serve HTTP endpoint. | 8005 | docker-compose.yml:296 |
USE_RAY_SERVE | Set to true to enable the Ray Serve integration. | false | docker-compose.yml:99 |
WS_ALLOWED_ORIGINS | JSON array of allowed origins for WebSocket connections. | ["http://localhost:8000", ...] | docker-compose.yml:111 |
SEARCH_SEARX_ENABLED | Set to true to enable the SearxNG search service. | true | docker-compose.yml:112 |
Sources: docker-compose.yml, scripts/docker_menu.py:266-281, README.md:191-237
Summary
The monGARS project provides a robust and developer-friendly set of tools for deployment. The use of Docker Compose, managed by interactive and automated scripts, simplifies the setup of the complex multi-service architecture. With built-in support for configuration management, health checks, optional ML profiles, and pre-deployment validation, operators can confidently deploy and manage the application in various environments.
Sources: README.md:120-122, scripts/docker_menu.py:8-10