Resource Sweepers

Resource Sweepers are a specialized component of the Terraform AWS Provider's testing framework, enabled by the sweep build tag. Their primary purpose is to automatically discover and delete AWS resources that may have been left behind after acceptance test runs. This cleanup process is crucial for maintaining a clean testing environment, managing costs, and preventing resource-related test failures.

Sweepers are implemented on a per-service basis and are registered against specific Terraform resource types. They are designed to be resilient, handling API errors, resource dependencies, and concurrent deletion. The core orchestration logic is centralized, while each AWS service has its own file (internal/service/<service>/sweep.go) containing the specific implementation for listing and deleting its resources.

Core Architecture

The sweeper framework is built around a few key components that work together to provide a robust and extensible cleanup mechanism.

Sources: internal/sweep/sweep.go

Sweeper Registration

Sweepers are registered during program initialization using the init() function in each service's sweep.go file. The resource.AddTestSweepers function from the Terraform Plugin SDK is used for this purpose.

A resource.Sweeper struct defines the behavior of a sweeper for a particular resource type.

Sources: internal/service/ec2/sweep.go:102-109, internal/sweep/sweep.go:19-22

The following diagram illustrates the registration process for an EC2 instance sweeper.

Client Management

To optimize performance and avoid redundant AWS client initializations, the framework uses a shared, cached client mechanism.

• SharedRegionalSweepClient(region string): This function is the entry point for obtaining an AWS client. It maintains a map[string]interface{} named SweeperClients that caches clients by region.
• Configuration: The client is configured using environment variables such as AWS_PROFILE, AWS_ACCESS_KEY_ID, and AWS_ASSUME_ROLE_ARN, similar to the main provider configuration.

Sources: internal/sweep/sweep.go:30-88

Deletion Orchestration

The SweepOrchestrator function is the heart of the execution logic. It manages the concurrent deletion of resources and handles transient errors like API throttling.

1. It accepts a slice of *SweepResource objects.
2. It uses a multierror.Group to execute the deletion of each resource in a separate goroutine, enabling parallel cleanup.
3. For each resource, it employs tfresource.RetryConfigContext to wrap the deletion call. This automatically retries the operation if it fails with a "Throttling" error, waiting up to SweepThrottlingRetryTimeout (10 minutes).

The SweepResource struct is a simple container that bundles the necessary components for deletion.

The following diagram shows the high-level workflow of the SweepOrchestrator.

Error Handling

A critical feature of the sweeper framework is its ability to gracefully handle API errors that don't necessarily indicate a failure. The SkipSweepError function checks for specific error codes and messages that mean a sweep for a particular resource or region should be skipped.

Common skippable errors include:

• UnsupportedOperation
• InvalidAction
• AccessDeniedException
• BadRequestException with "not supported" message

This prevents the entire test suite from failing due to a single service not being available in a region (e.g., GovCloud) or an account lacking specific permissions.

Sources: internal/sweep/sweep.go:134-192

Service-Specific Sweeper Implementation

Each AWS service with sweepable resources has a sweep.go file that follows a consistent pattern.

1. init() function: Registers one or more resource.Sweeper instances for the service's resources.
2. sweep<ResourceName>(region string) function: Contains the logic for cleaning up a specific resource type.

General Deletion Pattern

The sweep<ResourceName> functions typically perform the following steps:

1. Get a shared AWS client via sweep.SharedRegionalSweepClient(region).
2. Access the service-specific connection (e.g., client.(*conns.AWSClient).EKSConn).
3. Use the AWS SDK's "List" or "Describe" methods (often with pagination helpers) to find all existing resources.
4. Iterate over the results.
5. Apply filters to avoid deleting protected or default resources (e.g., default VPCs, WAFv2 ACLs managed by AWS Firewall Manager).
6. For each resource to be deleted, either call the delete API directly or, more commonly, create a sweep.SweepResource and add it to a slice.
7. Pass the slice of SweepResource objects to sweep.SweepOrchestrator for concurrent deletion.

The following sequence diagram shows the process for sweeping EKS Add-ons, which involves nested API calls.

Example: EC2 Security Groups

The sweepSecurityGroups function in internal/service/ec2/sweep.go is a good example of handling dependencies. To avoid DependencyViolation errors, it must first revoke all ingress and egress rules from a security group before it can be deleted.

1. First Pass: It describes all security groups and, for non-default groups, issues RevokeSecurityGroupIngress and RevokeSecurityGroupEgress calls to clear all rules.
2. Second Pass: It describes the security groups again.
3. Deletion: It calls DeleteSecurityGroup. This call is wrapped in a resource.Retry block to handle eventual consistency issues where the API might initially report a dependency violation even after rules have been revoked.

Sources: internal/service/ec2/sweep.go:1418-1486

Example: AppConfig Deployment Strategies

The sweepDeploymentStrategies function in internal/service/appconfig/sweep.go shows how sweepers can filter resources that should not be deleted. It uses a regular expression to identify and skip predefined AppConfig strategies, which cannot be deleted via the API.

// internal/service/appconfig/sweep.go:210-213
if regexp.MustCompile(`^AppConfig\.[A-Za-z0-9]{9,40}$`).MatchString(id) {
    log.Printf("[DEBUG] Skipping AppConfig Deployment Strategy (%s): predefined strategy cannot be deleted", id)
    continue
}

Sources: internal/service/appconfig/sweep.go:210-214