Terraform Infrastructure as Code
Relevant source files
The following files were used as context for generating this wiki page:
The Poneglyph project utilizes Terraform to manage its cloud infrastructure on Google Cloud Platform (GCP). This Infrastructure as Code (IaC) approach ensures consistent deployment of the Poneglyph API server and its associated worker services, specifically targeting Google Cloud Run for container orchestration.
The infrastructure stack is designed to support a multi-service architecture, including a Hono-based API server and a Rust-based upload processing worker. By defining resources like Cloud Run services, Pub/Sub topics, and IAM roles in Terraform, the project achieves reproducible environments and automated resource provisioning.
Sources: terraform/README.md:1-5, terraform/server.tf:130-140
Infrastructure Architecture
The Poneglyph infrastructure is centered around Google Cloud Run, leveraging serverless containers to scale based on demand. The architecture integrates with Google Pub/Sub for asynchronous processing and S3-compatible storage for data persistence.
Component Overview
- API Server (
poneglyph-server): The main backend service built with Hono. It handles user requests, authentication via Better-Auth, and initiates background jobs by publishing to Pub/Sub. - Upload Worker (
poneglyph-upload-worker): A specialized service that processes file uploads. It is triggered by Pub/Sub push subscriptions and operates with internal-only ingress for security. - Messaging Pipeline: Google Pub/Sub serves as the glue between the server and worker, facilitating decoupled communication for upload notifications.
Sources: terraform/README.md:7-22, terraform/variables.tf:13-20
Resource Relationship Diagram
The following diagram illustrates how the Terraform-managed resources interact within the GCP environment:
The diagram shows the flow from user requests through the API server to the asynchronous processing worker via Pub/Sub.
Sources: terraform/README.md:15-25, terraform/server.tf:140-150
Service Configuration
Terraform manages the deployment and environment configuration for the services using a set of centralized variables and local definitions.
API Server Configuration
The API server requires a significant number of environment variables to interact with various providers, including database connections, AI services (Google Generative AI, Groq), and external storage.
| Variable Name | Description | Source |
|---|---|---|
NODE_ENV | Environment mode (production) | terraform/server.tf:3 |
DATABASE_URL | PostgreSQL connection string | terraform/variables.tf:76 |
S3_BUCKET_NAME | Target bucket for data storage | terraform/variables.tf:101 |
PUBSUB_UPLOAD_TOPIC | Topic ID for upload events | terraform/server.tf:87 |
Deployment Specifications
The services are deployed using the GoogleCloudPlatform/cloud-run/google module. Key configurations include:
- Autoscaling: Configured with a
max_scale(default 2 for server, 1 for workers) andminScaleof 0 to optimize costs. - Health Checks: A startup probe is defined for the server at the
/healthpath. - Ingress: The server allows
allingress for public API access, while workers are restricted.
Sources: terraform/server.tf:144-180, terraform/variables.tf:31-41
Messaging and IAM
To enable secure communication between components, Terraform configures specific IAM roles and Pub/Sub resources.
Pub/Sub Integration
Terraform creates a topic for upload notifications and a corresponding push subscription for the worker service. The worker is invoked via a service account with the necessary permissions to receive push messages.
Sources: terraform/README.md:19-21, terraform/variables.tf:58-65
The sequence shows the asynchronous notification flow managed by the Pub/Sub resources defined in the Terraform stack.
Sources: terraform/README.md:20-25, terraform/server.tf:121-128
Project Integration
The Terraform configuration is integrated into the wider Poneglyph monorepo, which uses Turborepo for task management.
- Build Tasks: Defined in
turbo.json, the project coordinates builds and checks acrossapps/serverandterraformvariables. - Dependencies: The API server includes the
@google-cloud/pubsubclient to interact with the infrastructure provisioned by Terraform. - Environment Management: Terraform variables (e.g.,
BETTER_AUTH_URL,CORS_ORIGIN) align with the frontend and backend requirements defined in the application source code.
Sources: package.json:20-30, turbo.json:14-35, apps/server/package.json:25
Conclusion
The Terraform infrastructure for Poneglyph provides a robust, serverless foundation on GCP. By automating the deployment of Cloud Run services and their supporting messaging and storage infrastructure, the project ensures that the humanitarian intelligence platform can scale and process data reliably.
Sources: terraform/README.md:1-10, terraform/server.tf:130-140