Game Import System

Relevant source files

The following files were used as context for generating this wiki page:

• sofl/importer/importer.py
• sofl/importer/source.py
• sofl/importer/steam_source.py
• sofl/importer/lutris_source.py
• sofl/importer/heroic_source.py
• sofl/importer/bottles_source.py
• sofl/main.py
• sofl/preferences/import_sources.py
• sofl/game.py

The Game Import System is a core module in SOFL responsible for discovering and importing games from various third-party launchers and sources into the SOFL library. It operates asynchronously to avoid blocking the user interface, provides detailed progress feedback, and handles errors gracefully. The system is designed to be modular, allowing new sources to be added by implementing a common Source interface.

The import process can be triggered manually by the user or automatically on application startup. Once initiated, it scans enabled sources like Steam, Lutris, Heroic, and others, creates Game objects for discovered titles, and adds them to the central game store. The system also manages post-import tasks such as removing missing games and providing summary notifications.

Architecture

The import system is built around three main components: the Importer class, the Source abstract base class, and concrete Source implementations for each supported platform.

• Importer: The central orchestrator that manages the entire lifecycle of an import task. It initializes sources, runs them in parallel threads, monitors progress, and handles UI updates and post-import cleanup.
• Source: An abstract base class that defines the interface for all game sources. It standardizes how sources are identified, checked for availability, and iterated over.
• Source Implementations: Concrete classes (e.g., SteamSource, LutrisSource) that inherit from Source and contain the specific logic for finding and parsing game data from a particular launcher.

Sources: sofl/importer/importer.py, sofl/importer/source.py

High-Level Data Flow

The following diagram illustrates the high-level flow of the game import process, from user action to the final UI update.

Class Structure

The relationship between the Importer, Source hierarchy, and Game objects is shown below.

The Importer Class

The Importer class is the engine of the import system. It is instantiated when an import is triggered and manages the entire process from start to finish.

Sources: sofl/importer/importer.py

Core Responsibilities

• State Management: Manages the application state, setting it to AppState.IMPORT during the process and back to AppState.DEFAULT upon completion. It also disables UI actions like "Add Game" and "Preferences" to prevent conflicts.
• Asynchronous Execution: Uses Gio.Task to run the scanning process for each source in a separate thread, ensuring the UI remains responsive.
• Progress Tracking: Monitors the progress of both source scanning (sources_progress) and the subsequent processing pipelines for each game (pipelines_progress). This is reflected in a Gtk.ProgressBar.
• UI Feedback: Presents a modal dialog (Adw.Dialog) during the import and uses Adw.Toast notifications to summarize the results (games added/removed) or provide an "Undo" option.
• Error Aggregation: Collects all errors, including FriendlyError instances, from all sources and managers, and presents them to the user in a single dialog after the import is complete.
• Cleanup: Handles post-import tasks, including removing games that are no longer found (if configured) and resetting internal state.

Import Lifecycle

The following sequence diagram details the steps taken when Importer.run() is called.

The Source Abstraction

To support multiple game launchers, the system uses a Source abstract base class. Each supported launcher (Steam, Lutris, etc.) has its own concrete implementation of this class.

Sources: sofl/importer/source.py

Key Components

Class / Property	Description	Source File
Source	The abstract base class defining the contract for all game sources.	sofl/importer/source.py:38
source_id	A unique string identifier for the source (e.g., "steam", "lutris"). Used for settings and internal tracking.	sofl/importer/source.py:41
name	A human-readable name for the source (e.g., "Steam").	sofl/importer/source.py:42
is_available	A property that checks if the source is available on the current operating system.	sofl/importer/source.py:61-62
locations	A collection of Location objects that define the paths to necessary files or directories for the source to function.	sofl/importer/source.py:57
iterable_class	A reference to a SourceIterable class that contains the logic for discovering and yielding games.	sofl/importer/source.py:54
__iter__()	Resolves locations and returns an iterator from its iterable_class.	sofl/importer/source.py:73-84
SourceIterable	An abstract class whose __iter__ method must be implemented by concrete sources to yield Game objects.	sofl/importer/source.py:23
URLExecutableSource	A specialized subclass for sources that launch games via a URL protocol (e.g., steam://rungameid/...). It automatically creates the correct xdg-open, start, or open command.	sofl/importer/source.py:97

Location Objects

Each source defines its required file system locations using Location objects. A Location specifies candidate paths to search for a resource (like a config file or data directory) and can raise an UnresolvableLocationError if none are found. This mechanism allows the importer to gracefully skip sources that are not installed or configured correctly.

Sources: sofl/importer/location.py, sofl/importer/steam_source.py:107-126

Supported Import Sources

SOFL can import games from a variety of launchers. The import for each is controlled by a boolean key in the GSettings schema.

Source Name	source_id	GSettings Key	Data Source	Source File
Steam	steam	steam	Reads appmanifest_*.acf files from Steam library folders.	sofl/importer/steam_source.py
Lutris	lutris	lutris	Queries the pga.db SQLite database for installed games.	sofl/importer/lutris_source.py
Heroic	heroic	heroic	Reads JSON configuration files for GOG and Epic Games.	sofl/importer/heroic_source.py
Bottles	bottles	bottles	Parses library.yml to find games configured within Bottles.	sofl/importer/bottles_source.py
Others	various	various	The system also supports Flatpak, Desktop entries, Itch, Legendary, and RetroArch.	sofl/main.py:411-432

Example: LutrisSource Implementation

The LutrisSource demonstrates a typical implementation. It queries a SQLite database to find games.

1. Location: It defines a Location for pga.db, searching in common Lutris data directories.
2. Iteration: The LutrisSourceIterable copies the database to a temporary location to avoid locking issues.
3. Query: It executes an SQL query to select installed, non-hidden games. It can optionally exclude Steam and Flatpak games managed by Lutris based on user preferences (lutris-import-steam, lutris-import-flatpak).
4. Game Creation: For each row returned by the database, it constructs a Game object using GameFactory.create_game and yields it along with additional data like the path to the cover art.

# sofl/importer/lutris_source.py:35-56
class LutrisSourceIterable(SourceIterable):
    # ...
    def __iter__(self):
        # Query the database
        request = """
            SELECT
                games.id, games.name, games.slug, games.runner, ...
            FROM games ...
            WHERE
                ...
                AND (games.runner IS NOT "steam" OR :import_steam)
                AND (games.runner IS NOT "flatpak" OR :import_flatpak)
            ;
        """
        params = {
            "import_steam": shared.schema.get_boolean("lutris-import-steam"),
            "import_flatpak": shared.schema.get_boolean("lutris-import-flatpak"),
        }
        db_path = copy_db(self.source.locations.data["pga.db"])
        connection = connect(db_path)
        cursor = connection.execute(request, params)
        # ...
        for row in cursor:
            # Create game
            values = { ... }
            game = GameFactory.create_game(values)
            # ...
            yield (game, additional_data)

Sources: sofl/importer/lutris_source.py:32-84

Configuration

Users can configure the import system via the "Import Sources" section in the application's preferences.

• Enabling/Disabling Sources: Each source (Steam, Lutris, etc.) has a master switch that enables or disables it for import.
• Source-Specific Options: Some sources have additional options, such as Heroic's toggles for Epic, GOG, and Amazon games, or Lutris's toggles for importing its Steam and Flatpak entries.
• General Options: Global settings like auto-import (run import on startup) and remove-missing (remove games from SOFL if they are no longer found in the source) are also available.

These settings are managed via GSettings and are bound to the UI elements in the preferences dialog.

Sources: sofl/preferences/import_sources.py:48-70