Utility Functions

Relevant source files

The following files were used as context for generating this wiki page:

• superset_ext/utils.py
• superset_ext/extension.py
• superset_ext/main.py
• tests/test_superset_ext_utils.py
• pyproject.toml
• superset_ext/pass_through.py

The superset-ext utility functions provide the foundational logic for managing environment variables, configuring the Superset application, and ensuring the local filesystem is prepared for operation. These utilities are designed to bridge the Meltano environment with the specific requirements of Apache Superset, handling tasks such as prefix stripping, dynamic configuration file generation, and directory management.

These functions are primarily located in superset_ext/utils.py and are utilized by the main Superset extension class to initialize the environment before invoking Superset commands.

Sources: superset_ext/utils.py:1-5, superset_ext/extension.py:23-28

Environment Configuration and Parsing

The extension relies on environment variables prefixed with SUPERSET_ to configure both the extension itself and the underlying Superset application. The utility functions provide mechanisms to filter, trim, and transform these variables into a format compatible with Superset's configuration requirements.

Key Logic for Environment Handling

1. Prefix Trimming: Variables are often passed with a specific prefix (e.g., SUPERSET_) which needs to be removed for internal application use.
2. Configuration Loading: The system scans os.environ for keys matching a specific prefix and optionally returns them as a dictionary with the prefix removed.
3. Environment Preparation: A specialized function, prepared_env, constructs a comprehensive environment dictionary, handling mandatory defaults for FLASK_APP, SUPERSET_HOME, and SUPERSET_CONFIG_PATH.

Sources: superset_ext/utils.py:11-66

Environment Preparation Flow

This diagram illustrates how the prepared_env function processes raw environment variables into a structured configuration.

Configuration File Generation

A critical utility is the ability to write a physical Python configuration file (superset_config.py) that Superset consumes at runtime. This allows the extension to translate Meltano's key-value configuration into valid Python attributes.

write_config Functionality

The write_config function generates a script that iterates through the provided configuration dictionary and sets uppercase keys as module-level attributes.

Parameter	Type	Description
log	structlog.BoundLogger	Logger for status reporting.
config_path	Path	Target path for the generated file.
superset_config	dict	Key-value pairs of configuration.
force	bool	If True, overwrites existing files.

Sources: superset_ext/utils.py:88-100

# Implementation of config script generation
config_script_lines = [
    "import sys",
    "module = sys.modules[__name__]",
    f"config = {superset_config!r}",
    "for key, value in config.items():",
    "    if key.isupper():",
    "        setattr(module, key, value)",
    "",
    "# add any additional config here #",
    "",
]

Sources: superset_ext/utils.py:114-124

Integration with Extension Lifecycle

The utility functions are orchestrated by the Superset class during the initialization phase. The following sequence shows how the extension uses utilities to prepare the system.

Testing Utility Logic

The reliability of these utilities is verified through unit tests that mock the environment and filesystem.

• Prefix Trimming: Validates that trim_prefix("foo_", "foo_bar") correctly returns "bar".
• Env Loading: Ensures that load_config_from_env correctly filters variables based on the SUPERSET_ prefix.
• Config Writing: Uses tmp_path to verify that write_config creates files correctly and respects the force parameter.

Sources: tests/test_superset_ext_utils.py:16-27, tests/test_superset_ext_utils.py:75-84

Summary

The utility functions in superset-ext act as the glue between Meltano's environment-based configuration and Superset's file-based configuration. By automating the generation of superset_config.py and normalizing environment variables (such as mapping HOME to SUPERSET_HOME), these utilities ensure that Superset can run predictably within a Meltano-managed container or virtual environment.

Sources: superset_ext/utils.py:7-9, superset_ext/extension.py:34-40