Asset Pipeline & Loading
Relevant source files
The following files were used as context for generating this wiki page:
The Asset Pipeline manages the discovery, extraction, and preparation of original game data from a Redguard installation. It provides a portable interface for both native and browser targets, ensuring game assets are converted into modern formats like GLB for models and PNG for textures without shipping any copyrighted data.
The system operates on a "Bring Your Own Game" (BYOG) principle. It validates a local installation, scans for supported file formats, and generates a portable cache of metadata and previews. In browser environments, a dedicated web worker handles the asynchronous decoding of original bytes using WASM-based parsers.
Sources: AGENTS.md:4-8, crates/cyrus-assets/README.md:104-108, README.md:12-15
Asset Discovery and Cataloging
The pipeline begins by locating and validating the game installation. It identifies original files, archives, and CD tracks to build a comprehensive Catalog.
Game Installation Mapping
Native applications use GameInstall::locate to resolve library folders and installation descendants. The system validates the root directory and discovery preserves files with duplicate names. The library indexes archives to expose their members as separate entries without loading all payloads into memory.
Sources: crates/cyrus-assets/src/lib.rs:250-255, crates/cyrus-assets/README.md:110-116
The Asset Catalog
The Catalog acts as a central registry for all discovered assets. It tracks content categories, file paths, and unique identities generated through content hashing.
| Structure | Description |
|---|---|
Catalog | Contains a list of Entry records, a list of Palette records, and a generation hash. |
Entry | Stores asset ID, name, path, category, format, and source information. |
Category | Enumerates types like Images, Models, Worlds, Audio, and Movies. |
Source | Identifies where the data resides (e.g., File, Archive, CdTrack, or World). |
The diagram shows the flow from the raw game installation to the generation of a portable catalog and metadata.
Sources: crates/cyrus-assets/README.md:110-120, crates/cyrus-assets/src/lib.rs:118-125
Preparation and Exporting
Preparation converts original Redguard formats into modern, renderer-friendly formats. This process occurs on-demand or during an initial startup preload.
Native and Browser Exporting
The pipeline uses shared math backends and parsers for both targets.
- Native Target: Writes converted files (GLB, PNG, WAV) to a local disk cache directory. It uses a generation hash to invalidate old caches when schema versions change.
- Browser Target: A background worker decodes original bytes in memory. The worker computes content fingerprints and transfers generated media to the main thread via WASM.
Sources: crates/cyrus-assets/README.md:122-132, crates/cyrus-assets-worker/README.md:4-10
Content Preparation Workflows
- Models: Original 3D files are exported as GLB files. They retain original bounds, palette RGB, and vertex mappings.
- Environments: The pipeline exports terrain height grids, static placements, and light inputs.
- Audio: Sound banks and dialogue are exported as WAV bytes.
- Menus/HUD: Images are decoded into PNG frames using original palettes.
Sources: crates/cyrus-assets/README.md:41-52, crates/cyrus-assets/README.md:183-190, crates/cyrus-assets/src/lib.rs:175-215
This sequence diagram illustrates the asynchronous loading of an asset in the browser viewer.
Sources: crates/cyrus-viewer/src/browser.rs:40-60, crates/cyrus-assets-worker/README.md:10-15
Web Integration and Loading
In web environments, asset loading is facilitated by a Service Worker and a dedicated background thread to prevent blocking the UI.
Web Loader and Asset Worker
The loader.js script initializes the application and requests a game folder from the user. Once the user selects a folder, the asset-worker.js handles all file reads and parsing.
- Service Worker: Registers
sw.jsto manage asset caching and HTTP requests. - Local Install: Uses local
Filehandles for native-like performance without uploading data. - Progress Reporting: The worker sends progress updates (
done,total,name) to the UI during large file reads or catalog generation.
Sources: dev/web/loader.js:48-75, crates/cyrus-assets-worker/README.md:1-8, crates/cyrus-viewer/src/browser.rs:105-125
Memory Asset Management
The browser target registers an in-memory asset source labeled cache://. When the worker finishes preparing an asset, it inserts the resulting bytes into a MemoryAssetReader root, making them available to the Bevy AssetServer.
Sources: crates/cyrus-viewer/src/browser.rs:150-165, crates/cyrus-assets/README.md:133-135
Startup Preload and Diagnostics
The startup phase includes an optional scan of the entire installation to verify format coverage and prepare essential HUD/Menu elements.
Preload Scanning
LoadingPlugin manages the startup sequence. On native targets, it spawns a "startup preparation" worker thread with an 8 MiB stack to handle complex parsers. This worker:
- Scans the install for all known files.
- Attempts to parse each file using the
rgprelibrary. - Generates a
preload-report.jsondetailing successful parses, failures, and unsupported formats.
Sources: crates/cyrus-startup/src/preparation.rs:35-65, crates/cyrus-startup/src/preparation.rs:125-135
Diagnostic Auditing
The cyrus --audit-worlds command runs the exporter with an audit observer. This captures skipped placement counts, texture fallback reasons, and runtime preparation diagnostics, storing them as artifacts for research.
Sources: crates/cyrus-assets/README.md:158-165
The Asset Pipeline & Loading system ensures a faithful reproduction of Redguard by strictly separating original data handling from the modern rendering engine. It provides high-performance, asynchronous asset conversion and robust diagnostic reporting across platforms.