Miscellaneous Tools
Relevant source files
The following files were used as context for generating this wiki page:
Cloud Custodian includes a suite of specialized tools that extend the core policy engine's capabilities. These tools manage operational tasks such as log exporting, documentation generation, and Git-based policy change tracking.
The tools support multi-account environments and integrate with native cloud services like AWS CloudTrail, CloudWatch, and S3 to enhance governance and visibility.
Sources: README.md:214-239, tools/c7n_logexporter/c7n_logexporter/exporter.py
LogExporter
LogExporter transfers CloudWatch logs to S3 buckets. This tool automates the archival process, allowing for long-term storage and analysis of logs generated by Custodian policies or other cloud resources.
Process Flow
The exporter identifies log groups and initiates export tasks to a target S3 bucket.
The diagram shows the logical flow from log group discovery to the completion of the S3 export task.
Sources: tools/c7n_logexporter/c7n_logexporter/exporter.py:1-50
PolicyStream
PolicyStream treats Git history as a stream of logical policy changes. It analyzes commits in a repository to identify how Cloud Custodian policies evolve over time.
Component Logic
PolicyStream uses pygit2 to interface with Git repositories. It iterates through commits and parses YAML policy files to determine additions, deletions, or modifications to specific policies rather than simple file-level diffs.
The sequence shows how PolicyStream translates raw Git commits into structured policy updates.
Sources: tools/c7n_policystream/policystream.py:1-40, tools/c7n_policystream/pyproject.toml:9-12
TrailCreator
TrailCreator identifies resource creators by scanning CloudTrail events. It performs retroactive tagging to associate resources with the users who launched them.
Data Flow
- The tool queries CloudTrail for specific resource creation events (e.g.,
RunInstances). - It extracts the IAM identity of the creator.
- It applies tags to the identified resources if they are currently missing creator information.
Sources: tools/c7n_trailcreator/c7n_trailcreator/trailcreator.py:1-30, README.md:225-226
Autodoc and Sphinx Extensions
The documentation tools automate the creation of policy references for business partners and developers.
c7n-autodoc
c7n-autodoc.py generates HTML documentation from a directory of policy files. It groups policies by resource type and category (e.g., Cost Control vs. Security).
| Feature | Description |
|---|---|
| S3 Upload | Automatically uploads generated HTML to a configured S3 bucket. |
| Categorization | Uses regex to sort policies into "Security & Governance" or "Cost Controls". |
| Environment Tags | Displays which environments (Sandbox, Test, Prod) a policy applies to based on tags. |
Sources: tools/sandbox/c7n_autodoc/c7n-autodoc.py:12-65, tools/sandbox/c7n_autodoc/README.md:12-25
Sphinx Extensions
The c7n_sphinxext module provides custom directives for the Sphinx documentation generator. It allows developers to embed resource schemas and documentation directly into RestructuredText (RST) files.
CustodianResource: Renders documentation for a specific resource type including its filters and actions.CustodianSchema: Dumps the JSON schema for a policy element into the documentation.
Sources: tools/c7n_sphinxext/c7n_sphinxext/docgen.py:126-160
Tool Summary Table
The following table summarizes the primary miscellaneous tools and their functions.
| Tool Name | Primary File | Purpose |
|---|---|---|
| LogExporter | exporter.py | Exports CloudWatch log groups to S3. |
| PolicyStream | policystream.py | Tracks logical policy changes via Git history. |
| TrailCreator | trailcreator.py | Retroactively tags resource creators using CloudTrail. |
| Salactus | cli.py | Performs scale-out S3 scanning. |
| Autodoc | c7n-autodoc.py | Generates HTML policy reports for stakeholders. |
Sources: README.md:214-239, tools/c7n_logexporter/pyproject.toml:16
These miscellaneous tools facilitate the lifecycle management of Cloud Custodian deployments, from initial resource tagging and policy auditing to final documentation and log archiving.