CI/CD, Testing, and Releases
Relevant source files
The following files were used as context for generating this wiki page:
The Hermes Agent project implements a structured lifecycle for code verification, documentation auditing, and release promotion. This system ensures that all contributions, specifically new agent skills, undergo automated testing and style validation before reaching a stable state.
The core components of this lifecycle include a Pytest-based testing suite, Continuous Integration (CI) reports for performance tracking, automated documentation linters, and a dedicated release script for managing stable versioning.
Testing Infrastructure
The testing system utilizes pytest to validate agent logic and individual skill implementations. Global configurations and fixtures reside in tests/conftest.py to provide a consistent environment for all test modules.
Global Test Configuration
The project uses specific hooks and fixtures to manage test execution and environmental mocking:
- Flaky Test Management: The
pytest_collection_modifyitemshook automatically applies theflakymarker to tests within thetests/integration/directory, allowing for retries on non-deterministic failures. - Environment Mocking: Common fixtures provide mocked instances of agent tools to prevent side effects during testing. These include
mock_terminal,mock_agent, andmock_web_search. - Session Management: The
hermes_homefixture ensures a clean, isolated directory for Hermes data during test sessions.
Sources: tests/conftest.py:22-38, tests/conftest.py:55-70
Skill Testing
Individual agent skills are tested using a standardized pattern. Skill authors must include tests in tests/skills/test_<skill>_skill.py.
# Canonical command to run skill-specific tests
scripts/run_tests.sh tests/skills/test_<skill>_skill.py -q
Sources: skills/software-development/hermes-agent-skill-authoring/SKILL.md:129-130
Continuous Integration (CI) and Auditing
The CI pipeline executes various checks to maintain project quality, focusing on documentation integrity and test performance.
Documentation and Site Checks
Automated workflows, such as the Docs Site Checks workflow, validate the integrity of the documentation site.
| Tool | Purpose |
|---|---|
ascii-guard | Lints documentation for prohibited ASCII box diagrams, favoring Mermaid or tables. |
check_doc_links.py | Validates relative Markdown paths and anchors across the docs/ directory. |
generate-skill-docs.py | Automatically regenerates documentation pages from SKILL.md frontmatter. |
Sources: website/README.md:12-16, website/README.md:47-49, skills/software-development/hermes-agent-skill-authoring/SKILL.md:132-135
Performance Reporting
The timings_report.py script parses JUnit XML output from test runs to identify slow-running tests. It generates a Markdown summary that highlights the top N slowest tests, helping developers identify bottlenecks in the CI pipeline.
This diagram shows how test results are processed into performance reports.
Sources: scripts/ci/timings_report.py:10-52
Release Management
Stable releases are managed through the scripts/releases/stable.py script. This automation ensures version consistency and proper Git tagging.
Stable Release Workflow
The release process follows a specific sequence to promote the current codebase to a stable version:
- Worktree Check: The script verifies the repository has no uncommitted changes.
- Version Determination: It reads the current version from the source and identifies the next stable version.
- Branch Validation: It ensures the release is performed from the
mainbranch. - Tagging: The script creates a new Git tag and pushes it to the remote repository.
This diagram illustrates the automated steps taken to tag a stable release.
Sources: scripts/releases/stable.py:15-85
Versioning Logic
The system distinguishes between development and stable versions. A version is considered stable if it does not contain suffixes like dev, a (alpha), or b (beta).
| Function | Logic |
|---|---|
is_stable(version) | Returns true if the version string contains no development suffixes. |
get_next_version(current) | Increments the version number while removing development identifiers. |
Sources: scripts/releases/stable.py:20-45
The project manages CI/CD through specialized scripts and standardized testing fixtures, ensuring that code and documentation remain functional and performant across releases. Testing is primarily driven by Pytest with custom CI reporting, while releases are strictly controlled via Git tagging automation.
Sources: scripts/releases/stable.py, tests/conftest.py, scripts/ci/timings_report.py, skills/software-development/hermes-agent-skill-authoring/SKILL.md