CI/CD Integration

Relevant source files

The following files were used as context for generating this wiki page:

• hacking/azp/README.md
• hacking/azp/incidental.py
• AGENTS.md
• README.md
• packaging/release.py
• test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py

The Ansible project utilizes a robust CI/CD system primarily built on Azure Pipelines to ensure code quality, stability, and to manage the release process. The integration is deeply tied into GitHub pull requests, where a series of automated checks are triggered for every change. The core of the testing infrastructure is the ansible-test framework, a unified tool for running various checks, including static analysis (sanity), unit tests, and integration tests.

A significant feature of the CI process is the analysis of "incidental code coverage," which helps maintain test quality as code is migrated to collections. For continuous delivery, the project uses a set of Python scripts to semi-automate the release process, from version bumping and changelog generation to publishing artifacts on PyPI and creating GitHub releases. Developer interaction with the CI system is facilitated through the gh command-line tool and guided by comprehensive documentation in AGENTS.md.

Sources: hacking/azp/README.md, AGENTS.md, README.md, packaging/release.py

CI Architecture Overview

The Continuous Integration (CI) process is triggered on pull requests and centers around Azure Pipelines. The main project README.md links directly to the Azure Pipelines build status, confirming it as the primary CI platform. Developers primarily interact with CI results through GitHub.

The diagram below illustrates the high-level CI workflow.

This diagram shows the flow from a developer's push to a pull request through the automated testing stages in Azure Pipelines.

The main components of the CI system are:

• Azure Pipelines: The CI service that executes the test jobs.
• ansible-test: The command-line tool used within the CI environment to execute all categories of tests.
• gh CLI: The recommended tool for developers to check CI status and access detailed logs from Azure Pipelines.
• hacking/azp/ scripts: A collection of Python scripts for advanced CI tasks, such as downloading artifacts and analyzing code coverage.

Sources: README.md:5, AGENTS.md:38-44, hacking/azp/README.md:5-9

The ansible-test Framework

ansible-test is the unified testing tool for the Ansible project, used for both local development and CI runs. It provides a consistent way to execute different types of tests in isolated environments, primarily using Docker.

The framework supports several categories of tests:

Command	Description	Example
ansible-test sanity	Runs static analysis and linting checks (e.g., pylint, pep8).	ansible-test sanity -v --docker default
ansible-test units	Executes unit tests, which are typically written using pytest.	ansible-test units -v --docker default
ansible-test integration	Runs integration tests against specific targets, often requiring distro-specific Docker containers.	ansible-test integration -v --docker ubuntu2404
ansible-test coverage	Provides tools for analyzing code coverage data.	ansible-test coverage analyze targets combine ...

Test execution is isolated using Docker containers. Sanity and unit tests typically use the --docker default container, while integration tests require specific OS containers like --docker ubuntu2204 to test platform-dependent functionality.

Sources: AGENTS.md:21-44, hacking/azp/incidental.py:302-303

Debugging CI Failures

A standardized workflow exists for developers to diagnose and resolve CI failures on their pull requests.

The following diagram outlines the recommended debugging process.

This flowchart shows the steps for debugging a CI failure, from checking automated comments to reproducing the error locally.

The key steps are:

1. Check ansibot Comments: The ansibot user often posts comments on pull requests with direct links and summaries of test failures. This is the fastest way to get initial error details.
2. Get CI URLs: If ansibot comments are insufficient, the gh pr checks <number> command provides direct links to the Azure DevOps build results for detailed logs.
3. Reproduce Locally: Once the specific failing test is identified, it should be run locally using the same ansible-test command to facilitate debugging.

Sources: AGENTS.md:46-72

Incidental Code Coverage Analysis

A key practice in Ansible's CI strategy is the analysis of "incidental code coverage." This occurs when a test for one component unintentionally covers code in another. This analysis is crucial for ensuring that as modules are migrated to collections, no test coverage is accidentally lost. The process is managed by a set of scripts in the hacking/azp/ directory.

The workflow for this analysis is as follows:

This sequence diagram shows how a user interacts with the hacking/azp scripts to download and analyze coverage data from Azure Pipelines.

Sources: hacking/azp/README.md:12-42

The incidental.py Script

This script is the core of the analysis process. It takes downloaded test results from Azure Pipelines and generates reports on incidental coverage.

The script is composed of several key classes that work together to process the data.

This class diagram illustrates the main components of the incidental.py script and their relationships.

• CoverageData: Parses the run.json file from the downloaded Azure results to get metadata like the commit SHA and test outcome. It also locates all coverage-analyze-targets.json files.
• Git: A wrapper around the git command-line tool to retrieve file contents from specific commits.
• CoverageTool: A wrapper around ansible-test coverage analyze subcommands (combine, filter, missing, expand) to manipulate the coverage data.

The script can be used to find exclusive coverage from incidental_* tests or to find coverage gaps in a specific plugin's own tests. The output is a text report detailing the covered lines and execution arcs, with links to the relevant source code on GitHub.

Sources: hacking/azp/incidental.py:279-340, hacking/azp/README.md:60-91

Release Management (CD)

The release process for ansible-core is a semi-automated workflow driven by the packaging/release.py script. It standardizes the steps required to prepare and publish a new version. The process is divided into two main phases: prepare and complete.

Prepare Phase

This phase prepares the repository for a new release by versioning files and generating a changelog. It culminates in a pull request that must be merged before proceeding.

The flow of the prepare command is as follows:

This flowchart shows the sequence of functions called by the prepare command in packaging/release.py.

Sources: packaging/release.py:1017-1027

Complete Phase

After the "prepare" PR is merged, the complete phase is initiated. This phase handles building, testing, and publishing the release artifacts.

The flow of the complete command is as follows:

This flowchart shows the sequence of functions called by the complete command in packaging/release.py.

Sources: packaging/release.py:1078-1088

Key Data Structures

The release script uses dataclasses to manage the information needed to interact with GitHub.

Dataclass	Field	Description
GitHubRelease	user	The GitHub user or organization.
	repo	The repository name.
	tag	The git tag for the release (e.g., v2.12.0).
	target	The commit SHA to tag.
	title	The title of the release.
	body	The release notes content.
	pre_release	A boolean indicating if it's a pre-release.
PullRequest	upstream_user	The user/org of the upstream repository.
	upstream_repo	The name of the upstream repository.
	upstream_branch	The target branch for the PR (e.g., devel).
	user	The user/org of the fork.
	repo	The name of the forked repository.
	branch	The source branch in the fork.
	title	The title of the pull request.
	body	The body content of the pull request.

Sources: packaging/release.py:298-327