AWS Deployment Options
Relevant source files
The following files were used as context for generating this wiki page:
- deployment/aws_ecs_fargate/cloudformation/README.md
- deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml
- deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_background_server_service_template.yaml
- deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml
- deployment/aws_ecs_fargate/cloudformation/services/onyx_vespaengine_service_template.yaml
- deployment/aws_ecs_fargate/cloudformation/services/onyx_web_server_service_template.yaml
- deployment/terraform/modules/aws/onyx/variables.tf
Onyx provides multiple production-grade deployment paths on Amazon Web Services (AWS), primarily focusing on containerized orchestration via Elastic Container Service (ECS) with Fargate and infrastructure-as-code using Terraform. These options are designed to handle the modular architecture of Onyx, which includes API servers, background workers, model servers, and vector databases.
The deployment infrastructure is structured to support scalability, high availability, and security through AWS-native services like Application Load Balancers (ALB), Elastic File System (EFS), and Secrets Manager.
Sources: deployment/aws_ecs_fargate/cloudformation/README.md:1-5, deployment/terraform/modules/aws/onyx/variables.tf:1-10
ECS Fargate Deployment Architecture
The ECS Fargate deployment utilizes CloudFormation templates to provision a suite of microservices. This architecture separates the frontend, backend API, background processing, and data engines into distinct services.
Core Service Components
The deployment is organized into infrastructure stacks and service stacks. Infrastructure is established first (EFS, Cluster, ACM), followed by data stores and functional Onyx services.
| Service Name | Purpose | Resource Requirements (Default) |
|---|---|---|
| Vespa Engine | Vector database for document search | 4 vCPU, 16GB RAM |
| Backend API Server | Handles incoming REST requests | 2 vCPU, 4GB RAM |
| Background Server | Processes long-running tasks and connectors | 2 vCPU, 4GB RAM |
| Web Server | Serves the Next.js frontend | 1 vCPU, 2GB RAM |
| Nginx | Reverse proxy and SSL termination | 0.5 vCPU, 1GB RAM |
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_vespaengine_service_template.yaml:12-14, deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml:12-14, deployment/aws_ecs_fargate/cloudformation/services/onyx_web_server_service_template.yaml:12-14, deployment/aws_ecs_fargate/cloudformation/README.md:38-51
Infrastructure Layout
The following diagram illustrates how the ECS services interact within the AWS VPC environment.
This diagram shows the traffic flow from the public internet through the load balancer and proxy to the internal Onyx services and data stores.
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml:150-165, deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml:96-125
Network and Security Configuration
VPC and Connectivity
Deployments can either create a new VPC or reuse an existing one. Services are typically deployed across multiple Subnet IDs in different Availability Zones to ensure high availability.
- Service Discovery: Internal communication between services (e.g., API server to Vespa) is managed via AWS Cloud Map (Service Discovery) using a private DNS namespace.
- Load Balancing: An Application Load Balancer (ALB) handles internet-facing traffic on port 80 (and optionally 443 via ACM), forwarding it to the Nginx service.
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml:150-180, deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml:61-75, deployment/terraform/modules/aws/onyx/variables.tf:12-40
Security and Secrets
Sensitive credentials are not stored in the templates but are retrieved at runtime.
- AWS Secrets Manager: Used to inject
POSTGRES_PASSWORDandUSER_AUTH_SECRETinto container environments. - Security Groups: Each service has a dedicated security group. For example, the Backend API server allows ingress on port 8080, while Vespa allows ingress on ports 19071 and 8081.
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml:126-130, deployment/aws_ecs_fargate/cloudformation/services/onyx_vespaengine_service_template.yaml:63-80
Auto-Scaling and Resilience
Services like the Web Server and Backend API Server include AWS::ApplicationAutoScaling::ScalableTarget and ScalingPolicy resources.
- CPU Scaling: Scales out when average CPU utilization exceeds 75%.
- Memory Scaling: Scales out when average memory utilization exceeds 80%.
- Deployment Circuit Breakers: Enabled in the ECS Service definition to automatically roll back failed deployments.
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_web_server_service_template.yaml:85-133, deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_api_server_service_template.yaml:135-183
Storage Integration (EFS)
Onyx relies on AWS Elastic File System (EFS) for persistent data storage across container restarts and for synchronizing configurations.
- Vespa Data: Mounts EFS volumes at
/opt/vespa/varand/var/tmpusing specific Access Points to persist the search index. - Nginx Config: Uses a sidecar container (
github-sync-container) to fetch the latestapp.conf.templateandrun-nginx.shfrom the Onyx GitHub repository, storing them on an EFS volume shared with the main Nginx container.
Sources: deployment/aws_ecs_fargate/cloudformation/services/onyx_vespaengine_service_template.yaml:125-165, deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml:105-145
Terraform Configuration Options
For teams using Terraform, Onyx provides a modular approach to provisioning the necessary AWS infrastructure. Key variables allow customization of the environment:
| Variable | Type | Default | Description |
|---|---|---|---|
create_vpc | bool | true | Whether to provision a new VPC for the cluster. |
enable_opensearch | bool | false | Whether to create an Amazon OpenSearch domain. |
opensearch_instance_type | string | r8g.large.search | Instance type for vector search nodes. |
waf_enable_logging | bool | true | Enable WAF logging to CloudWatch for security audits. |
postgres_backup_retention_period | number | 7 | Retention days for RDS automated backups. |
Sources: deployment/terraform/modules/aws/onyx/variables.tf:12-14, deployment/terraform/modules/aws/onyx/variables.tf:135-155, deployment/terraform/modules/aws/onyx/variables.tf:225-230
Deployment Summary
Onyx on AWS is built for modularity and production stability. By utilizing ECS Fargate, the system removes the need for manual server management while maintaining high performance for AI search tasks through resource-heavy Vespa nodes and auto-scaling API layers. Deployment can be managed through the provided CloudFormation deploy.sh script or integrated into larger infrastructure workflows using Terraform.
Sources: deployment/aws_ecs_fargate/cloudformation/README.md:1-25, deployment/aws_ecs_fargate/cloudformation/services/onyx_backend_background_server_service_template.yaml:1-10