Docker Infrastructure
Relevant source files
The following files were used as context for generating this wiki page:
The Aether-X Ultimate platform utilizes a containerized architecture powered by Docker and Docker Compose to manage a complex ecosystem of microservices, polyglot databases, and real-time data processing tools. The infrastructure is designed to provide a consistent development environment that mirrors production settings, facilitating seamless deployment across various environments.
Sources: README.md:9-30, setup-dev.sh:5-10, IMPLEMENTATION_COMPLETE.md:8-15
Infrastructure Architecture
The system is partitioned into three primary Docker Compose layers: Core Application Services, Database Infrastructure, and Kafka/Messaging Infrastructure. All components communicate over a shared bridge network named aether-x-network.
The diagram above illustrates the high-level connectivity between application microservices and the underlying data and messaging layers.
Sources: docker-compose.yml:3-195, infrastructure/databases/docker-compose.yml:3-170
Microservices Layer
The application layer consists of eight primary microservices. Each service is built from a local Dockerfile and includes health checks to ensure availability within the Docker ecosystem.
| Service Name | Port | Primary Dependency | Description |
|---|---|---|---|
text-generation | 8000 | Redis, Elasticsearch | GPT-2 based text generation |
image-generation | 8001 | Redis, MongoDB | Stable Diffusion image creation |
video-generation | 8002 | Redis, MongoDB | Video synthesis with TTS |
ai-models | 8003 | Redis, Elasticsearch | LLM and specialized models |
no-code-development | 8004 | Postgres, Redis | Natural language app creation |
security | 8005 | Redis, Elasticsearch | Auth, encryption, and protocols |
data-processing | 8006 | Kafka, Redis, ES | ETL and real-time streaming |
api-gateway | 80 | Security, Redis | Request routing and load balancing |
Sources: docker-compose.yml:6-140, README.md:12-20
Data Persistence & Caching
The infrastructure employs a polyglot persistence strategy, utilizing multiple database engines specialized for different data models.
SQL and NoSQL Databases
The environment initializes with four major database systems. Initialization scripts located in ./infrastructure/databases/init-databases.sh automate the creation of tables, collections, and keyspaces during the first container startup.
- MySQL (Port 3306): Used for user management and structured application data.
- PostgreSQL (Port 5432): Stores workflow definitions and relational data for the no-code engine.
- MongoDB (Port 27017): Handles unstructured content storage and generated metadata.
- Cassandra (Port 9042): Provides high-availability storage for distributed data.
Sources: infrastructure/databases/docker-compose.yml:6-85, infrastructure/databases/init-databases.sh:5-150
Caching and Search
- Redis (Port 6379): Acts as the primary session store and caching layer for all microservices.
- Elasticsearch (Port 9200): Provides full-text search capabilities and analytics.
- Kibana (Port 5601): Visualizes logs and search data stored in Elasticsearch.
Sources: infrastructure/databases/docker-compose.yml:128-170, docker-compose.yml:180-195
Messaging and Event Streaming
Real-time data processing is facilitated by an Apache Kafka stack. This infrastructure is defined in a specialized compose file and includes components for schema management and connectivity.
This diagram shows the internal dependencies of the messaging stack, highlighting Zookeeper as the coordinator for the Kafka broker.
Sources: infrastructure/kafka/docker-compose.yml:4-100
Monitoring and Management
The Docker infrastructure includes a built-in observability stack to monitor service health and resource utilization.
- Prometheus (Port 9090): Collects metrics from microservices using a
prometheus.ymlconfiguration. - Grafana (Port 3001): Provides dashboards for visualizing service performance, with default credentials
admin/admin. - Nginx (Port 80/443): Serves as a reverse proxy and load balancer, sitting in front of the API Gateway and Dashboard.
Sources: docker-compose.yml:157-195, setup-dev.sh:160-170
Development Workflow
The project provides a setup-dev.sh script to prepare the environment. This script creates necessary directories (logs, temp, docs), generates .dockerignore files to prevent build context bloat, and sets up initial configuration files.
Common Docker Commands
# Start all services in detached mode
docker-compose up -d
# Rebuild services after code changes
docker-compose up --build -d
# View real-time logs for all containers
docker-compose logs -f
# Stop and remove all containers and networks
docker-compose down
Sources: setup-dev.sh:10-60, setup-dev.sh:190-200, IMPLEMENTATION_COMPLETE.md:30-40
The Aether-X Ultimate Docker infrastructure provides a robust, modular foundation for running high-performance AI services. By isolating databases, messaging, and application logic into discrete container groups, the system ensures scalability and simplifies the complex dependency management inherent in a large-scale AI platform.