Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• dev/guidelines/testing.md
• AGENTS.md
• pyproject.toml
• dev/guidelines/python.md
• dev/guides/local-development-setup.md
• tasks.py
• tests/integration/conftest.py

The nornir-infrahub project employs a multi-layered testing strategy to ensure the reliability of its Nornir plugins and their integration with Infrahub. The strategy distinguishes between fast, isolated unit tests and comprehensive, environment-dependent integration tests. It also incorporates automated quality gates for code formatting, linting, and type checking.

Sources: dev/guidelines/testing.md:1-10, AGENTS.md:52-60

Test Layers and Execution

The test suite consists of two fundamental layers: unit tests and integration tests. These layers differ in execution speed, environmental requirements, and verification scope.

Layer	Location	Network/Docker Required	Marker	Execution Command
Unit Tests	tests/unit/	No	None (Default)	pytest
Integration Tests	tests/integration/	Yes (Docker)	integration	pytest -m integration

Sources: dev/guidelines/testing.md:12-23, AGENTS.md:65-70, pyproject.toml:57-61

Unit Testing Strategy

Unit tests verify plugin logic, validation rules, and error paths in isolation. They use extensive mocking to avoid network dependencies or Docker requirements.

• Mocking SDK Objects: Logic involving InfrahubNode is tested by constructing real InfrahubNodeSync objects using dummy addresses and local data dicts.
• Task Plugin Mocking: Task plugins are tested by building fake Nornir tasks where task.host.data["InfrahubNode"] is a MagicMock.
• Inventory Mocking: InfrahubInventory is tested by patching the InfrahubClientSync class to simulate schema and node data.

Sources: dev/guidelines/testing.md:25-45, dev/guides/developing-a-task-plugin.md:36-60

Integration Testing Strategy

Integration tests validate end-to-end behavior against a live Infrahub API. This includes inventory loading, schema mapping resolution, and branch awareness.

• Infrastructure: Each test class spins up a fresh Infrahub stack using infrahub-testcontainers in Docker.
• Base Class: All integration tests must inherit from NornirInfrahubIntegration.
• Bootstrapping: A class-scoped bootstrap fixture loads schemas and populates test data (sites, devices, IP addresses).
• Authentication: Tests use a specific admin token fixture provided by the testcontainers helper.

Sources: dev/guidelines/testing.md:58-75, AGENTS.md:124-129, dev/guides/local-development-setup.md:38-48

The following diagram illustrates the lifecycle of an integration test class:

The diagram shows the sequence from container initialization through data bootstrapping to test execution and cleanup.
Sources: dev/guidelines/testing.md:58-75, tests/integration/conftest.py

Quality Gates and Automation

The project enforces strict quality gates through invoke tasks and CI workflows. These gates ensure code consistency and type safety before code is merged.

Automated Linters

The invoke lint command runs a comprehensive suite of checks:

• yamllint: Validates YAML file structure.
• ruff: Performs fast linting and code formatting checks.
• pylint: Conducts deep static code analysis.
• ty: Executes type checking on nornir_infrahub modules.
• rumdl: Validates and formats Markdown documentation.

Sources: tasks.py:34-86, AGENTS.md:43-50, dev/guidelines/python.md:9-17

CI Workflow

The GitHub Actions CI workflow triggers on pull requests and performs the following checks:

The sequence diagram details the parallel and sequential checks performed by CI, including linting, multi-version Python testing, and documentation validation.
Sources: dev/guidelines/git-workflow.md:65-75, pyproject.toml:7-15

Testing Conventions and Tools

Configuration

Testing behavior is defined in pyproject.toml using pytest markers and options.

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
addopts = "-vs -m 'not integration' -p no:pytest-infrahub-performance-test"
markers = ["integration: marks tests as integration tests (requires Docker)"]

Sources: pyproject.toml:52-61

Key Development Tasks

Developers interact with the testing and quality systems via invoke:

Task	Description
invoke format	Formats Python files (ruff) and Markdown (rumdl).
invoke lint	Runs all linters (yaml, ruff, pylint, ty, rumdl).
pytest	Runs unit tests only.
pytest -m integration	Runs integration tests (Docker required).

Sources: tasks.py:25-32, AGENTS.md:43-70, dev/guides/local-development-setup.md:20-48

Assertion Guidelines

• Nornir Results: Tests for task plugins should assert on result.failed, result.changed, and the content of result.result.
• Exceptions: Use pytest.raises for code that is expected to raise RuntimeError, ValueError, or Pydantic ValidationError.
• Mocks: Use assert_called_once_with to verify SDK client interactions.

Sources: dev/guidelines/testing.md:47-56, dev/guides/developing-a-task-plugin.md:52-60

Summary

The testing strategy for nornir-infrahub ensures software quality by separating logic verification from environmental integration. Unit tests provide rapid feedback through mocking, while integration tests use infrahub-testcontainers to verify real-world API interactions. This is supported by a robust automated linting and type-checking pipeline that maintains code standards across the project.