Architecture Overview
Relevant source files
The following files were used as context for generating this wiki page:
The auto-opsi-repo system provides an automated pipeline for discovering, downloading, and packaging Windows software into OPSI 4.3 archives. It operates as a self-hosted Docker stack consisting of a Python-based builder and an Nginx-based web server to serve the repository data.
Sources: README.md:1-10, agents.md:7-15
System Components
The architecture divides responsibilities between two primary services and a centralized catalog.
repo-builder
The repo-builder is a Python application that utilizes official OPSI 4.3 CLI tools to manage the package lifecycle. It runs on a configurable interval (defaulting to 6 hours) and performs the following tasks:
- Discovers upstream releases via various source adapters.
- Downloads installers to a private workspace.
- Packages installers into
.opsiarchives using templates. - Generates repository metadata files (e.g.,
packages.json).
Sources: README.md:18-22, agents.md:17-18, docker-compose.yml:2-35
repo-web
The repo-web service uses Nginx to serve the generated .opsi archives and metadata over HTTP. It provides a read-only view of the repository directory and includes directory listing and basic security headers.
Sources: README.md:23-25, nginx/default.conf:1-16
Catalog
The catalog (catalog/packages.yaml) defines all supported products, their upstream sources, installer arguments, and detection methods. The builder uses this file as its source of truth for every build cycle.
Sources: README.md:26-28, catalog/packages.yaml:1-500
Data Flow and Build Process
The build process follows a linear progression from upstream discovery to repository publication.
The diagram shows the flow of data from upstream vendor sources through the builder's validation and packaging logic into the final distribution layer.
Sources: builder/builder/main.py:23-145, builder/builder/opsi.py:21-96
Source Adapters
The system supports multiple adapter types to resolve release metadata. Each adapter implements logic to identify the latest version, download URL, and SHA256 checksum.
| Adapter Type | Description |
|---|---|
github_release | Fetches releases from the GitHub REST API using regex for asset matching. |
winget_manifest | Queries the Microsoft WinGet manifest repository on GitHub. |
mozilla | Uses Mozilla's product-details JSON API for Firefox and Thunderbird. |
adoptium | Queries the Adoptium API for OpenJDK binaries. |
direct_url | Parses HTML or text files using regex to find versions and templates for downloads. |
microsoft | Uses the VS Code stable update API. |
Sources: builder/builder/sources/adapters.py:65-177, README.md:103-125
Security and Integrity
The architecture emphasizes integrity through multiple layers of verification:
- Host Validation: The builder verifies that redirect URLs match an explicit allowlist of hostnames or prefixes.
- Checksum Persistence: The system maintains a historical record of SHA256 hashes in
state/packages.jsonandprovenance.jsonfiles to prevent unexpected installer changes. - Atomic Operations: Files are staged in a private
work/directory and atomically renamed into the repository to prevent partial reads by Nginx. - Locking: A file lock (
state/builder.lock) ensures only one builder process modifies the repository at a time.
Sources: README.md:29-35, builder/builder/main.py:175-185, builder/builder/repository.py:61-70
Repository Structure
The repository storage contains the OPSI packages and various metadata sidecars used for integrity and OPSI compatibility.
| File Extension | Purpose |
|---|---|
.opsi | The actual OPSI package archive containing scripts and the vendor installer. |
.opsi.md5 | MD5 checksum for OPSI transfer compatibility. |
.opsi.zsync | Zsync file for efficient delta downloads. |
.opsi.provenance.json | Metadata including source URL, original SHA256, and build timestamp. |
packages.json | OPSI repository metadata used by the OPSI client/server. |
Sources: README.md:32-35, builder/builder/main.py:108-115, builder/builder/repository.py:61-70
Process Execution Sequence
The following sequence diagram illustrates the interaction during a successful build cycle for a single product.
The sequence illustrates the verification-heavy process of fetching and packaging a new software version.
Sources: builder/builder/main.py:68-150, builder/builder/opsi.py:80-96
Summary
The auto-opsi-repo architecture leverages official OPSI tooling within a containerized environment to automate the maintenance of a private software repository. By decoupling the discovery and packaging logic from the distribution layer, it ensures that repository metadata remains consistent even during upstream network failures or build errors.
Sources: README.md:18-35, agents.md:7-15