Testing, Tox, and CI/CD

Relevant source files

The following files were used as context for generating this wiki page:

• CONTRIBUTING.md
• pyproject.toml
• README.md
• tools/release_helper.py
• tests/core/test_dev_commands.py
• tests/core/test_version.py

Testing, Tox, and CI/CD represent the quality assurance and automation framework for the Red Discord Bot project. The system ensures that all code contributions meet specific style requirements, pass unit tests across multiple Python versions, and maintain consistent documentation standards. These automated checks are mandatory for merging pull requests and are integrated into the development workflow to identify errors early in the contribution process.

Sources: CONTRIBUTING.md:120-135, README.md:41-45

Local Test Automation with Tox

The project uses tox as the primary tool to manage and run test environments. Tox automates the creation of virtual environments to isolate testing dependencies and ensures that tests run in a clean, reproducible state.

Test Environments

Tox executes three main categories of checks, each within its own virtual environment:

Environment	Tool	Purpose
py38	pytest	Runs unit tests on Python 3.8.
docs	N/A	Validates that documentation builds without warnings and all links are valid.
style	black	Ensures the code adheres to the project's style guide.

Sources: CONTRIBUTING.md:125-133

Workflow execution

To execute all automated tests locally, you run the tox command in the project directory. You may also run a specific environment using tox -e <env>.

The diagram shows the parallel nature of Tox environments and the requirement for all checks to pass before a contribution is considered valid.
Sources: CONTRIBUTING.md:130-135

Unit Testing with Pytest

Unit tests are implemented using pytest to verify the functionality of core bot components and utilities. The project repository is configured as a pytest plugin to facilitate testing within the ecosystem.

Test Structures

The project includes tests for various core modules:

• Dev Commands: Validates code cleanup and REPL execution logic.
• Version Info: Ensures proper parsing and comparison of VersionInfo strings.
• Utilities: Tests helper functions like bounded_gather and pagify.

Sources: pyproject.toml:57-58, tests/core/test_dev_commands.py:10-20, tests/core/test_version.py:10-15

Pytest Configuration

The pyproject.toml file defines specific settings for the test runner:

• asyncio_mode = 'auto': Automatically handles asynchronous test cases.
• red-discordbot = "redbot.pytest": Registers the bot as a pytest entry point.

Sources: pyproject.toml:57-65

Code Style and Formatting

The project enforces a strict code style using black to maintain readability and reduce diff noise in version control.

• Formatter: All Python files must be formatted with Black.
• Line Length: The project sets a custom line length of 99 characters (overriding the default 88).
• Automation: The Makefile includes a make reformat command to automatically apply the style across the project.
• Exclusions: The redbot/vendored directory is excluded from style checks.

Sources: CONTRIBUTING.md:138-146, pyproject.toml:60-63

CI/CD and Release Automation

Continuous Integration is handled via GitHub Actions, which executes the test suite on every push and pull request.

Release Helper and Workflows

The project includes a tools/release_helper.py script that guides maintainers through the release lifecycle. This script interacts with GitHub Actions to trigger specialized workflows.

The diagram illustrates the interaction between the manual release script and automated GitHub Actions to manage release branches and PRs.
Sources: tools/release_helper.py:330-410, README.md:41-43

Release Stages

The release helper tracks the following stages to ensure process integrity:

1. Welcome/Info: Setting release type (Breaking, Standard, Maintenance, Hotfix).
2. Blocker Check: Ensuring no issues labeled "Release Blocker" remain open.
3. Changelog: Creating and pushing a dedicated changelog branch.
4. Workflow Spawn: Running the prepare_release.yml GitHub Action.
5. Merge: Merging the automatically generated pull requests.

Sources: tools/release_helper.py:53-70, tools/release_helper.py:545-560

Continuous Integration Status

The bot's stability is publicly tracked via GitHub Actions status badges. These badges indicate the current health of the V3/develop and stable branches, covering unit tests, build status, and documentation availability on ReadTheDocs.

Sources: README.md:41-45

Testing and CI/CD automation form the foundation of Red's "editable mode" development, allowing contributors to clone the repository and immediately verify changes using make newenv followed by tox.

Sources: CONTRIBUTING.md:105-115