AWS Provider
Relevant source files
The following files were used as context for generating this wiki page:
The AWS Provider enables Cloud Custodian to manage Amazon Web Services (AWS) resources through a unified rules engine. It translates YAML-defined policies into actions and filters that execute against the AWS API to ensure security, compliance, and cost optimization. The provider integrates with native AWS serverless capabilities, such as CloudWatch Events and AWS Config Rules, for real-time policy enforcement.
Sources: README.md:16-25, README.md:46-55
Provider Architecture and Governance
The AWS Provider operates as a distinct functional area within the Cloud Custodian project. It is managed by Area Maintainers who oversee specific resource implementations, triaging issues, and managing releases. The provider relies on boto3 as the primary SDK for interacting with AWS services.
Sources: GOVERNANCE.md:7-13, pyproject.toml:11
Maintenance Roles
- Area Maintainers: Triage PRs and issues specifically for AWS; manage CI and releases within the AWS domain.
- Core Maintainers: Coordinate cross-area releases and project-wide changes that affect all providers.
Sources: GOVERNANCE.md:22-38
Core Architecture Flow
The diagram shows how a user-defined policy is loaded into the AWS Provider, which then uses resource managers to apply specific filters and actions via the Boto3 SDK.
Resource Definition and Schema
AWS resources are defined using a structured vocabulary of filters and actions. Each resource type (e.g., aws.s3, aws.ec2) maps to a specific ResourceManager class.
Sources: README.md:31-33, tools/c7n_sphinxext/c7n_sphinxext/docgen.py:53-62
Resource Components
| Component | Description |
|---|---|
| Filters | Identify specific subsets of resources based on attributes, tags, or boolean conditions. |
| Actions | Execute operations on resources matched by filters (e.g., terminate, tag, remove-statements). |
| Modes | Define execution context (e.g., pull for cron/CLI, cloudtrail for event-driven). |
Sources: README.md:79-114, tools/c7n_sphinxext/c7n_sphinxext/docgen.py:104-118
Documentation Generation
The c7n_sphinxext tool generates documentation by resolving resource schemas. When generating permissions, it uses dummy resource types like kinesis to simulate policy construction if a specific resource is not provided.
Sources: tools/c7n_sphinxext/c7n_sphinxext/docgen.py:53-65
AWS Cloud Control (AWSCC) Provider
The project includes a specialized provider for the AWS Cloud Control API (awscc). This provider utilizes the CloudFormation resource type schemas to automate resource management.
Sources: tools/c7n_awscc/build.py:1-8
AWSCC Build Process
The build process fetches the latest CloudFormation Schema catalog and maps AWS resource types to Boto3 service names.
Sources: tools/c7n_awscc/build.py:108-135
Service Name Mapping
The system maintains a mapping between CloudFormation resource prefixes and Boto3 service identifiers where they differ.
| AWS CC Prefix | Boto3 Service Name |
|---|---|
acmpca | acm-pca |
aps | amp |
cassandra | keyspaces |
elasticloadbalancingv2 | elbv2 |
route53recoverycontrol | route53-recovery-control-config |
Sources: tools/c7n_awscc/build.py:50-93
IAM and ARN Management
The provider relies on accurate Amazon Resource Name (ARN) references for permission validation and resource targeting. The data_updatearnref.py tool automates the retrieval of these references by scraping AWS service authorization documentation.
Sources: tools/dev/data_updatearnref.py:65-75
ARN Scraping Logic
- Access the base AWS documentation URL for service authorization.
- Extract links for individual services (e.g.,
list_amazons3.html). - Parse HTML tables to identify resource types, ARN formats, and condition keys.
- Generate a structured JSON mapping of service prefixes to resource ARNs.
Sources: tools/dev/data_updatearnref.py:79-110, tools/dev/data_updatearnref.py:280-305
Policy Execution Modes
The AWS Provider supports multiple execution modes that determine how and when policies run.
Sources: README.md:97-104, tools/c7n_sphinxext/c7n_sphinxext/docgen.py:137-148
Serverless Execution
When a policy uses a mode like cloudtrail, Cloud Custodian automatically provisions the necessary infrastructure.
Sources: README.md:52-54, README.md:97-104
Summary
The AWS Provider functions as a comprehensive rules engine interface for AWS, leveraging YAML policies to manage a vast library of cloud resources. It integrates deep schema resolution, automated ARN mapping, and native serverless provisioning to provide real-time compliance and cost control across multi-account environments. Through tools like AWSCC and custom documentation extensions, it maintains high technical accuracy and developer visibility into supported cloud operations.