Maintenance Scripts and Tooling

Relevant source files

The following files were used as context for generating this wiki page:

• scripts/audit-repository.py
• builder/builder/main.py
• builder/builder/opsi.py
• builder/builder/repository.py
• README.md
• agents.md
• docker-compose.yml

Maintenance scripts and tooling manage the lifecycle of the OPSI repository by automating package building, metadata generation, and integrity auditing. These tools ensure the repository remains synchronized with upstream vendor releases while maintaining internal consistency through checksum verification and retention policies. The primary entry point for these operations is the repo-builder service, which executes periodic cycles or manual maintenance tasks.

Sources: README.md:14-22, agents.md:9-14, builder/builder/main.py:151-177

Core Builder Operations

The builder orchestrates the discovery, download, and packaging of software. It operates on a configurable cycle (default 6 hours) or as a one-off command. It utilizes a file lock (state/builder.lock) to prevent concurrent write operations that could corrupt repository metadata.

Execution Modes

The builder supports several flags to control its behavior:

Flag	Action
--once	Executes a single update cycle and exits.
--dry-run	Fetches metadata and reports proposed changes without modifying the filesystem.
--product <ID>	Limits the operation to a specific OPSI product ID.
--force	Rebuilds a specific product even if the version is unchanged.
--force-all	Rebuilds all enabled products in the catalog.

Sources: builder/builder/main.py:133-138, README.md:104-108

Lifecycle Flow

The following diagram illustrates the sequence of actions during a standard builder cycle:

The builder cleans temporary directories on startup to recover from previous crashes. It then iterates through the catalog, resolving the latest version for each product. If a new version is found, it downloads the installer, builds the .opsi package, and updates the repository metadata.

Sources: builder/builder/main.py:27-128, builder/builder/main.py:164-173

Repository Integrity and Auditing

Integrity tools verify that the physical .opsi archives match their recorded metadata and provenance records.

Audit Script

The scripts/audit-repository.py tool performs a read-only integrity check of all published packages. It performs the following validations:

• Computes the SHA256 hash of the .opsi archive and compares it to the package_sha256 field in the corresponding .provenance.json file.
• Verifies the MD5 hash against the .opsi.md5 sidecar.
• Ensures .opsi.zsync files are present and non-empty.
• Cross-references the physical files against the packages.json metadata index.

Sources: scripts/audit-repository.py:1-20

Provenance Tracking

Each package is accompanied by a .provenance.json file. This file acts as an immutable record of the package's origin, containing:

• upstream_version: The original version string from the vendor.
• sha256: The hash of the downloaded vendor installer.
• package_sha256: The hash of the final .opsi archive.
• source_url: The exact URL used for the download.
• signature_status: The result of osslsigncode verification (SIGNED, UNSIGNED, or UNKNOWN).

Sources: builder/builder/main.py:90-104, README.md:162-171

Repository Metadata Management

The builder/builder/repository.py module handles the organization and indexing of the repository. It relies on the official OPSI CLI tools to maintain compatibility.

Metadata Generation

Metadata is generated in a staging directory to prevent readers from seeing partial updates. The builder:

1. Creates a temporary directory within work/.
2. Creates hard links (or copies if cross-device) of all .opsi files into the stage.
3. Executes opsi-cli manage-repo metafile create and scan-packages.
4. Atomically replaces the live metadata files (packages.json, etc.) in the repository/ directory.

Sources: builder/builder/repository.py:35-55

Retention Policy

The retain function enforces the KEEP_VERSIONS environment variable. It identifies obsolete packages by grouping files by product ID and sorting them by version and revision.

Retention runs only after new metadata is successfully generated. It deletes the .opsi file along with its .md5, .zsync, and .provenance.json sidecars.

Sources: builder/builder/repository.py:24-32, builder/builder/repository.py:58-64

OPSI Packaging Tooling

The system interacts with the OPSI backend through the opsi-cli. The builder/builder/opsi.py module wraps these calls to handle package generation and extraction validation.

Build and Validation Logic

The build process includes a round-trip validation to ensure the produced archive is valid:

1. Generate: Templates are rendered into a source directory, including control.toml and OPSI scripts.
2. Build: opsi-cli package make creates the .opsi archive.
3. Extract: The builder immediately extracts the new archive to a verify directory.
4. Verify: It compares the extracted control.toml and installer hashes against the original source files.

Sources: builder/builder/opsi.py:75-103

Template Context

When generating packages, the following variables are provided to the Jinja2 templates:

Variable	Description
p	The complete package definition from the catalog.
version	The normalized OPSI-compatible version string.
revision	The integer package_revision from the catalog.
custom_detection	Rendered script block if detection.method is "custom".

Sources: builder/builder/opsi.py:30-35

Service Management

The maintenance infrastructure is deployed via Docker Compose, separating the builder logic from the web server.

Maintenance Commands

Common maintenance tasks are performed via docker compose:

# Perform a manual audit of all packages
docker compose run --rm --entrypoint python repo-builder scripts/audit-repository.py

# Repackage a specific product without checking upstream (Offline recovery)
docker compose run --rm --entrypoint python repo-builder -m builder.repackage auto-firefox

# Clean all temporary work files (Automatic on builder startup)
docker compose run --rm repo-builder --once

Sources: README.md:188-193, builder/builder/main.py:165-173, docker-compose.yml:1-35

Maintenance scripts and tooling provide the necessary automation to keep the OPSI repository current and reliable. By combining periodic background updates with manual audit and recovery tools, the system ensures that software installers are correctly packaged, verified, and indexed for OPSI clients.