Serverless Deployment Infrastructure
Relevant source files
The following files were used as context for generating this wiki page:
Cloud Custodian utilizes serverless deployment infrastructure to execute policies as native cloud functions across AWS, Azure, and GCP environments. This infrastructure automates the provisioning of serverless functions and their associated event sources, such as CloudWatch Events, Azure EventGrid, and GCP AuditLogs.
The core system manages the lifecycle of these functions, including code packaging, IAM role assignment, and trigger configuration. It allows administrators to run policies in real-time response to environment changes or on a recurring schedule without maintaining dedicated server fleets.
Sources: README.md:32-37, README.md:90-95, tools/ops/policylambda.py:165-177
Architecture and Execution Modes
The infrastructure supports multiple execution modes that determine how and when a policy function triggers. The mu module (and provider-specific implementations) translates Custodian policy definitions into cloud-native resources.
Execution Mode Mapping
| Mode Type | Cloud Provider | Trigger Mechanism | Implementation Detail |
|---|---|---|---|
periodic | AWS / GCP / Azure | Scheduled Timer | Uses render_periodic to define cron/rate schedules. |
cloudtrail | AWS | Event Bus | Filters CloudWatch Events via render_cwe. |
config-rule | AWS | Compliance Change | Provisions AWS Config Rules with Lambda permissions. |
azure-event-grid | Azure | Resource Events | Configures Event Grid subscriptions for Azure Functions. |
gcp-audit-log | GCP | Log Entries | Binds Cloud Functions to Audit Log events. |
Sources: tools/ops/policylambda.py:44-115, README.md:90-95, tools/sandbox/c7n_sphere11/cfn/api.yml:153-165
Deployment Workflow
The deployment process follows a structured flow from policy definition to cloud resource provisioning.
This diagram illustrates the transformation of a YAML policy into a deployable zip archive and a Serverless Application Model (SAM) template.
Sources: tools/ops/policylambda.py:133-149, tools/ops/policylambda.py:165-180
Lambda and SAM Integration
For AWS deployments, the infrastructure can generate AWS Serverless Application Model (SAM) templates. This allows users to deploy Custodian policies using standard AWS DevOps tooling instead of the Custodian CLI's built-in capabilities.
Function Rendering
The policylambda.py utility extracts configuration from the mu.PolicyLambda class to populate SAM resource properties.
- Environment Variables: Translates policy settings into Lambda
Environmentvariables. - Tracing: Configures X-Ray
TracingConfig. - Dead Letter Queues (DLQ): Identifies SNS or SQS targets for failed executions.
- Permissions: Automatically creates
AWS::Lambda::Permissionresources for external triggers like AWS Config.
Sources: tools/ops/policylambda.py:30-41, tools/ops/policylambda.py:82-99
Event Source Transformation
Events are mapped to specific SAM event types based on the policy mode:
def render_cwe(p, policy_lambda, sam):
# Extracts CloudWatchEventSource from policy lambda
events = [e for e in policy_lambda.get_events(None)
if isinstance(e, mu.CloudWatchEventSource)]
# Maps to SAM CloudWatchEvent type
revents = {}
for idx, e in enumerate(events):
revents['PolicyTrigger%s' % string.ascii_uppercase[idx]] = {
'Type': 'CloudWatchEvent',
'Properties': {'Pattern': json.loads(e.render_event_pattern())}
}
return properties
Sources: tools/ops/policylambda.py:62-79
Automated Documentation and Reporting
The serverless infrastructure includes tools to document the deployed estate. c7n-autodoc crawls policy directories and generates HTML documentation, which it can upload to an S3 bucket configured for static website hosting.
Documentation Features
- Category Grouping: Policies are categorized (e.g., 'Security & Governance' vs 'Cost Controls') via regex matching on file paths.
- Environment Tagging: Uses policy tags to determine if a policy applies to 'Sandbox', 'Test', or 'Production'.
- S3 Integration: Automatically uploads the rendered documentation to a specified S3 path with
public-readACLs for easy access by business partners.
Sources: tools/sandbox/c7n_autodoc/c7n-autodoc.py:73-89, tools/sandbox/c7n_autodoc/README.md:7-17, tools/sandbox/c7n_autodoc/example_config_file.yml:45-56
Summary of Core Components
| Component | Responsibility | Relevant Files |
|---|---|---|
mu.PolicyLambda | Core logic for AWS Lambda resource management. | c7n/mu.py, tools/ops/policylambda.py |
SAM_RENDER_FUNCS | Mapping of execution modes to SAM/CloudFormation templates. | tools/ops/policylambda.py |
Archive Packager | Creates the zip file containing policy code and dependencies. | tools/ops/policylambda.py |
c7n-autodoc | Generates and uploads human-readable policy documentation. | tools/sandbox/c7n_autodoc/c7n-autodoc.py |
Sources: tools/ops/policylambda.py:102-115, tools/sandbox/c7n_autodoc/c7n-autodoc.py:99-108, README.md:215-220