Architecture Overview
Relevant source files
The following files were used as context for generating this wiki page:
Cloud Custodian (c7n) operates as a rules engine for managing public cloud accounts and resources. It enables users to define policies in YAML to ensure real-time compliance, security, and cost optimization across AWS, Azure, and GCP environments. Sources: README.md:19-32
The system consolidates ad-hoc scripts into a lightweight tool that supports arbitrary filtering, structured reporting, and automated provisioning of serverless functions for event-driven enforcement. Sources: README.md:23-26, README.md:66-71
Core Policy Engine Architecture
The engine processes YAML policy files to execute management logic against cloud resources. Each policy consists of four primary components:
- Resource: Specifies the target cloud service (e.g.,
aws.ec2,azure.cosmosdb,gcp.pubsub). - Filters: Define the criteria for matching specific resources using nested boolean conditions.
- Actions: Determine the operations to perform on matched resources (e.g., tag, stop, terminate).
- Mode: Controls the execution environment, such as periodic cron jobs or event-driven serverless functions.
Sources: README.md:36-40, README.md:104-109
Policy Execution Flow
The following diagram illustrates the flow from policy definition to resource action:
This diagram shows the sequence where a policy is parsed, validated against a schema, and passed to a resource manager that interacts with cloud APIs to filter and act upon resources. Sources: README.md:104-135, tools/c7n_sphinxext/c7n_sphinxext/docgen.py:34-45
Extension and Tooling Ecosystem
Cloud Custodian includes several specialized tools that extend core functionality for multi-account management and Infrastructure as Code (IaC) scanning.
| Tool | Purpose | Key Components |
|---|---|---|
| c7n-org | Multi-account/subscription execution | Parallel execution across accounts |
| c7n-left | IaC asset scanning | Support for Terraform files |
| c7n-mailer | Notification handling | Reference implementation for alerts |
| c7n-autodoc | Policy documentation | Jinja2 templates, S3/HTML output |
Sources: README.md:225-265, tools/sandbox/c7n_autodoc/README.md:5-18
Automated Documentation Architecture
c7n-autodoc automates the generation of policy documentation for business partners by crawling directories for policy files and rendering them into HTML. Sources: tools/sandbox/c7n_autodoc/c7n-autodoc.py:53-70
This diagram depicts the c7n-autodoc workflow: gathering policy data via regex, processing it through templates, and distributing it via S3. Sources: tools/sandbox/c7n_autodoc/c7n-autodoc.py:20-80
Project Governance and Area Division
The project is organized into functional areas to ensure scalable community leadership and vendor neutrality. Each area is managed by Area Maintainers who triage PRs and ensure the health of specific cloud providers or tools. Sources: GOVERNANCE.md:1-12
Organizational Structure
- Core Maintainers: Oversee project-wide changes, CNCF coordination, and security. Sources: GOVERNANCE.md:27-29
- Area Maintainers: Own specific providers (AWS, Azure, GCP, etc.) and managed tools like
c7n-org. Sources: GOVERNANCE.md:24-26 - Functional Areas: Divided by cloud provider and specific utilities like Mailing & Notifications or Releases & Tooling. Sources: GOVERNANCE.md:8-18
Testing and Development Architecture
Cloud Custodian utilizes a "Flight Data" recording pattern for provider testing, particularly in GCP development. This allows tests to run without active cloud credentials by replaying recorded API responses. Sources: tools/c7n_gcp/development.md:121-135
Test Infrastructure Setup
- Recording Mode: Real API calls are made and stored as JSON flight data. Sources: tools/c7n_gcp/development.md:154-158
- Playback Mode: Tests use recorded JSON files to simulate API interactions, enabling CI/CD without live infrastructure. Sources: tools/c7n_gcp/development.md:149-152
- Terraform Integration: The
@terraformdecorator manages real infrastructure during recording sessions. Sources: tools/c7n_gcp/development.md:179-195
Deployment and CI/CD Logic
Deployment workflows, particularly for Azure, utilize container hosts and synchronized storage. Policies are validated via custodian validate before being synchronized to blob storage containers. Sources: tools/ops/azure/container-host/devops-flow/README.md:3-9
This sequence shows the Azure DevOps workflow where policies are validated during PRs and synchronized to storage for consumption by containerized execution hosts. Sources: tools/ops/azure/container-host/devops-flow/README.md:18-50
Cloud Custodian provides a modular, policy-driven architecture that balances core engine stability with an extensive ecosystem of cloud-specific providers and developer-centric tools. Sources: README.md:19-32, GOVERNANCE.md:8-12