System Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• internal/cloud/backend.go
• internal/command/test_test.go
• internal/plans/planproto/planfile.proto
• internal/rpcapi/terraform1/dependencies/dependencies.pb.go
• internal/backend/local/backend_plan.go
• internal/command/e2etest/automation_test.go
• BUGPROCESS.md

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Its architecture is designed around a core engine that interprets declarative configuration files, builds a dependency graph of resources, and interacts with provider plugins to manage infrastructure on various platforms. This modular, plugin-based architecture allows Terraform to be extended to manage a wide range of services.

The key architectural features of Terraform include its approach to Infrastructure as Code, the generation of Execution Plans, the use of a Resource Graph to manage dependencies, and the automation of complex infrastructure changes. This document outlines the high-level system architecture, focusing on the core components, the integration with HCP Terraform (Terraform Cloud), the testing framework, and the structure of execution plans.

Core Components

Terraform's core is composed of the Command Line Interface (CLI) and a central graph engine. The engine is responsible for parsing configurations, managing state, building the resource graph, and communicating with provider plugins.

Sources: README.md

This diagram shows the high-level interaction between the Terraform CLI, the Core Engine, and the provider plugins.

Resource Graph

Terraform builds a dependency graph of all resources defined in the configuration. This allows it to parallelize the creation and modification of any non-dependent resources, leading to efficient infrastructure provisioning. The graph also provides operators with insight into the dependencies within their infrastructure.

Sources: README.md:23-25

Provider Plugins

Providers are implemented as separate plugins that Terraform Core communicates with, typically over gRPC. Terraform can automatically download providers from the Terraform Registry. This plugin-based architecture is fundamental to Terraform's ability to support a vast ecosystem of infrastructure platforms. The interfaces for these plugins, including dependency management and schema retrieval, are defined in protobuf files.

Sources: README.md:50-52, internal/rpcapi/terraform1/dependencies/dependencies.pb.go

HCP Terraform Integration (Cloud Backend)

Terraform can integrate with HCP Terraform or Terraform Enterprise using the cloud backend. This backend enables features like remote state storage, remote operations, and collaborative workflows.

Sources: internal/cloud/backend.go

Configuration

The cloud backend is configured within the terraform block of a configuration file. Key configuration attributes are resolved from the configuration block, environment variables, or defaults.

Attribute	Type	Description	Source
hostname	string	The Terraform Enterprise hostname. Defaults to app.terraform.io. Can be set by TF_CLOUD_HOSTNAME.	internal/cloud/backend.go:618-621
organization	string	The name of the organization. Required. Can be set by TF_CLOUD_ORGANIZATION.	internal/cloud/backend.go:623-624
token	string	The authentication token. It's recommended to use terraform login instead of setting this directly.	internal/cloud/backend.go:626-629
workspaces	block	A block to configure workspace mapping using either name or tags.	internal/cloud/backend.go:193-213

The following diagram illustrates the configuration resolution flow.

Operation Modes

The cloud backend can operate in two modes:

1. Remote Operations: plan and apply operations are executed on HCP Terraform or Terraform Enterprise. This is the default when the cloud block is configured and the organization has operations enabled.
2. Local Operations: Operations are executed on the local machine, and HCP Terraform is used only for state storage. This mode is activated if the workspace's execution-mode is set to local or if the TF_FORCE_LOCAL_BACKEND environment variable is set.

The decision logic is as follows:

Workspace Mapping

The cloud backend maps the local working directory to remote workspaces in HCP Terraform using one of two strategies defined in the workspaces block:

Strategy	Configuration	Description
Name	workspaces { name = "my-workspace" }	Maps to a single, explicitly named remote workspace. This conflicts with the tags attribute.
Tags	workspaces { tags = ["app", "prod"] }	Maps to a set of remote workspaces matching the specified tags. This allows for dynamic workspace selection.

Sources: internal/cloud/backend.go:193-213, internal/cloud/backend.go:1162-1178

State migration from other backends (like local or remote) to the cloud backend is also supported, with interactive prompts to guide the user.

Sources: internal/cloud/e2e/migrate_state_multi_to_tfc_test.go, internal/cloud/e2e/migrate_state_remote_backend_to_tfc_test.go

Testing Framework

Terraform includes a built-in testing framework for writing automated tests for modules. Tests are defined in .tftest.hcl files.

Sources: internal/command/test_test.go

Test File Structure

A test file consists of variables, provider, and run blocks.

• run blocks: Define individual test cases. Each run block executes a terraform apply or terraform plan and then runs assertions.
• State Management: run blocks can share state by using the same state_key. Runs with the same key are executed sequentially. Runs with different keys can be executed in parallel.
• Assertions: assert blocks within a run block define the conditions that must be true for a test to pass.
• Mocking: The framework supports mocking providers using mock_provider blocks to return predefined values, which is useful for unit testing modules without creating real infrastructure.

Sources: internal/command/test_test.go:107-110, internal/command/test_test.go:400-403, internal/command/test_test.go:1425-1428

Test Execution Lifecycle

The terraform test command orchestrates the execution of these test files.

The following diagram shows the lifecycle of a single run block.

The framework also supports skipping cleanup for specific runs (skip_cleanup = true), which leaves resources intact for inspection. A separate terraform test -cleanup command can be used to destroy these resources later.

Sources: internal/command/test_test.go:1740-1744, internal/command/test_test.go:1003-1007

Execution Plan Structure

A Terraform plan is a structured file that describes the changes Terraform will make to the infrastructure. The structure of this file is defined by a Protocol Buffers schema.

Sources: internal/plans/planproto/planfile.proto

The main components of a plan file are:

• version: A version number for the plan format.
• variables: The input variable values used to create the plan.
• resource_changes: A list of proposed changes to resource instances. Each change includes the action (create, update, delete, noop, etc.) and the before/after values.
• resource_drift: A list of changes detected between the prior state and the refreshed state from the real-world infrastructure.
• output_changes: Proposed changes to root module outputs.
• terraform_version: The version of Terraform that created the plan.
• backend: Configuration of the backend used.

The diagram below shows the entity relationships within a plan file.

Automation Workflow

In automated environments, a common workflow is to separate the plan and apply steps to allow for manual review or approval.

This diagram illustrates the recommended automation workflow, ensuring that the apply command executes exactly what was planned. The TF_IN_AUTOMATION environment variable can be set to tailor Terraform's output for non-interactive shells.

Sources: internal/command/e2etest/automation_test.go

Development and Release Process

The project follows a structured process for handling bug reports and managing releases.

Bug Triage

A formal bug triage process is documented to ensure timely responses to issues. The process involves several stages, identified by GitHub labels:

1. new: Initial filtering and categorization.
2. waiting for reproduction: If a core team member cannot reproduce the issue.
3. confirmed: When the issue is successfully reproduced.
4. explained: After the root cause in the codebase is identified.
5. Fix: A pull request is created to resolve the issue.

Sources: BUGPROCESS.md

Release Management

The scripts/changelog.sh script automates parts of the release process, including changelog generation and version bumping. It supports different release types: dev, alpha, beta, rc, release, and patch. This script uses changie and semver to manage changelog entries and version numbers, ensuring a consistent release workflow.

Sources: scripts/changelog.sh