Terraform Infrastructure

Terraform infrastructure in the Fleet project provides a standardized, "infrastructure-as-code" approach to deploying and managing Fleet at scale. It supports multiple environments including Amazon Web Services (AWS), Google Cloud Platform (GCP), and Kubernetes clusters. By using Terraform modules, the project enables automated provisioning of all necessary components—including databases, caches, load balancers, and container services—while maintaining version control and scalability.

The infrastructure is designed with a "Bring Your Own" (BYO) philosophy, allowing users to choose the level of abstraction they need. This ranges from a "BYO-Nothing" root module that provisions an entire environment to highly specific modules that integrate with existing VPCs, databases, or container orchestration systems.

Sources: articles/fleet-terraform-module.md, articles/deploy-fleet-on-gcp.md

Cloud Architecture & Platform Support

Fleet's Terraform implementation is multi-cloud, with detailed support for AWS and GCP, as well as provider-agnostic Kubernetes deployments.

AWS Infrastructure

On AWS, Fleet typically utilizes Amazon ECS (Elastic Container Service) for the application layer, Amazon RDS (Aurora) for the MySQL database, and Amazon ElastiCache for Redis. The architecture is modularized to support varying levels of existing infrastructure.

The AWS deployment supports a range of 10 to 350,000 hosts and includes automated secret management via AWS Secrets Manager.
Sources: articles/deploy-fleet-on-aws-with-terraform.md, infrastructure/loadtesting/terraform/infra/README.md

GCP Infrastructure

The GCP Terraform project automates the creation of a complete environment, including a new project factory, VPC networking, and managed services.

Sources: articles/deploy-fleet-on-gcp.md

Modular Design (BYO Pattern)

The Fleet Terraform module is organized using a nested hierarchy. This allows users to start with a standard installation and customize specific internal resources as requirements evolve.

• BYO-Nothing: The root module; creates all resources from scratch.
• BYO-VPC: Deploys Fleet into an existing network.
• BYO-Database: Uses existing networking and provides hooks for an external database.
• BYO-ECS: Provides the most control, focusing only on the application container orchestration.

Sources: articles/fleet-terraform-module.md

Load Testing Infrastructure

Fleet maintains a specialized Terraform environment for load testing. This environment is designed to simulate thousands of hosts using osquery-perf containers. It supports advanced configurations like OpenTelemetry tracing with SigNoz and specific engine overrides for Valkey (a Redis-compatible cache).

Configuration Variables

The load testing infrastructure uses specific variables to control scale:

Sources: infrastructure/loadtesting/terraform/infra/README.md, infrastructure/loadtesting/terraform/readme.md

Load Test Deployment Flow

Kubernetes Deployment via Terraform

For Kubernetes, Terraform serves as a wrapper or alternative to Helm. It manages the deployment of the Fleet application, service accounts, and ingress configurations.

Key Kubernetes Configurations

The Terraform module for Kubernetes requires specific objects to define the application state:

• Database Config: Defines address, database, username, and password_key (referencing K8s secrets).
• Cache Config: Defines the Redis/Valkey address and authentication requirements.
• Ingress: Handles TLS termination (e.g., via Nginx) and hostname routing.

Sources: articles/deploy-fleet-on-kubernetes.md

Implementation Details

Fleet Configuration Object

The fleet_config variable is a complex object used across AWS and GCP modules to define the application's runtime environment.

variable "fleet_config" {
  type = object({
    mem                         = optional(number, 512)
    cpu                         = optional(number, 256)
    image                       = optional(string, "fleetdm/fleet:v4.22.1")
    extra_environment_variables = optional(map(string), {})
    database = object({
      password_secret_arn = string
      user                = string
      database            = string
      address             = string
    })
    # ... additional fields for redis, awslogs, etc.
  })
}

Sources: articles/fleet-terraform-module.md

Networking Data Flow (GCP Example)

In the GCP implementation, the flow of data is strictly managed between public-facing and private zones using Cloud NAT for outbound access to vulnerability resources.

The Terraform infrastructure is the primary method for deploying Fleet in production-ready environments. Its modular nature ensures that it remains adaptable to both small-scale testing and enterprise-grade deployments across major cloud providers and Kubernetes.