Testing and Validation Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• tests/test_core.py
• tests/test_workflow.py
• tests/test_github_automation.py
• agents.md
• README.md
• builder/builder/main.py
• builder/builder/download.py
• builder/builder/opsi.py

The Testing and Validation Strategy ensures the integrity of the automatic OPSI software repository by verifying package metadata, installer authenticity, and repository consistency. The strategy combines automated unit tests, continuous integration (CI) workflows, and runtime validation checks during the build cycle.

Sources: README.md:275-285, agents.md:73-81

Unit Testing Framework

The project uses pytest to execute unit tests that focus on core logic without making real network calls. These tests use mocks to simulate external responses and verify internal state transitions.

Test Categories

Category	Focus Area	Source File
Core Logic	Version normalization, URL validation, and ZIP path safety.	tests/test_core.py
Workflow	Package selection, duplicate prevention, and failure isolation.	tests/test_workflow.py
CI Automation	Verification of GitHub Action triggers and Dependabot config.	tests/test_github_automation.py

Sources: agents.md:24-28, tests/test_core.py, tests/test_workflow.py, tests/test_github_automation.py

Version Normalization Validation

Tests ensure that various upstream version formats (e.g., v1.4.2, 2026.03, 1.2.3-beta) normalize correctly to PEP 440 notation or standard semantic versions. This ensures consistent ordering where 10.0 follows 9.9.

Sources: tests/test_core.py:44-65, README.md:144-150

Runtime Validation Strategy

Validation occurs continuously during the repo-builder execution cycle. The builder performs multi-stage checks before publishing any package to the repository.

Build and Package Validation Flow

The following diagram illustrates the sequence of validation checks performed when a new version is detected.

Security and Integrity Checks

• URL Allowlisting: The system rejects any URL not matching the allowed_hosts defined in catalog/packages.yaml. It explicitly blocks URLs with credentials or non-HTTPS schemes.
  Sources: builder/builder/download.py:17-33, tests/test_core.py:68-87
• Checksum Verification: Every download is verified against a SHA256 hash. If the builder detects a hash change for a known version, it refuses to rebuild unless ALLOW_CHANGED_CHECKSUM is enabled.
  Sources: builder/builder/download.py:100-112, README.md:183-195
• ZIP Safety: During extraction of ZIP-based installers, the builder checks for path traversal attempts (e.g., ..), absolute paths, or symlinks pointing outside the destination.
  Sources: builder/builder/download.py:115-131, tests/test_core.py:103-125
• Signature Advisory: The system uses osslsigncode to log the signature status (SIGNED, UNSIGNED, or UNKNOWN) of EXE and MSI installers.
  Sources: builder/builder/download.py:155-171, README.md:197-202

Integration and Acceptance Testing

While the builder automates archive creation, Windows client acceptance testing is required to verify installer behavior.

• Dry Run Mode: Executing with --dry-run fetches metadata and reports proposed changes without modifying the state or repository.
  Sources: builder/builder/main.py:97-101, README.md:112-118
• Round-trip Validation: After building an .opsi archive, the builder extracts it back to a temporary directory. It compares the extracted control.toml and bundled installer SHA256 against the original source to ensure no corruption occurred during packaging.
  Sources: builder/builder/opsi.py:90-110
• Client Testing: Users must test the generated packages on a representative OPSI-managed VM to verify that silent arguments, upgrades, and uninstalls function correctly on Windows.
  Sources: README.md:79-85

Continuous Integration (CI)

The project uses GitHub Actions to maintain code quality and ensure that updates do not break existing validation logic.

Summary

The Testing and Validation Strategy establishes a layered defense against repository corruption and untrusted software. By combining rigid URL/checksum pinning at runtime with comprehensive unit tests for version and path logic, the system ensures that only verified, consistent packages are served to OPSI clients. Final validation remains with the administrator through dry runs and VM-based acceptance testing.