Meltano.yml Configuration

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• superset_ext/extension.py
• superset_ext/utils.py
• superset_ext/main.py
• pyproject.toml
• superset_ext/pass_through.py

The meltano.yml configuration for the Superset extension defines how the Apache Superset utility is integrated into a Meltano project. It specifies the execution environment, including the virtual environment requirements via pip_url, the executable entry points, and the environment-specific configurations that drive the underlying Superset instance.

This configuration acts as the bridge between Meltano's orchestration and the superset-ext logic, allowing users to initialize the database, manage admin users, and run the Superset web interface directly through Meltano commands.
Sources: README.md:9-41, superset_ext/extension.py:23-38

Plugin Definition and Commands

The extension is typically configured as a utility within the utilities section of meltano.yml. It is recommended to use a unique name like ext_superset to avoid conflicts with existing Superset plugins.

Command Mappings

The configuration maps Meltano-specific commands to the superset_extension executable, which is the Typer-based CLI entry point for the extension.
Sources: README.md:11-30, superset_ext/main.py:19-22

Command	Executable	Description
describe	superset_extension	Returns the extension's capabilities and commands.
initialize	superset_extension	Generates the config file and performs db upgrade and init.
invoke	superset_extension	Primary entry point for passing arguments to the extension.
create-admin	superset_extension	Helper to create a Superset administrator user.
ui	superset_extension	Shortcut to run the web server on host 127.0.0.1 and port 8088.
(Default)	superset_invoker	The base invoker for direct passthrough to the superset CLI.

Sources: README.md:18-32, superset_ext/extension.py:151-177

The following diagram illustrates how Meltano commands are routed through the extension to the underlying Superset binary:

The superset_extension (main.py) provides high-level management, while superset_invoker (pass_through.py) provides a direct line to the Apache Superset CLI.
Sources: superset_ext/main.py:1-85, superset_ext/pass_through.py:1-18, superset_ext/extension.py:33-35

Environment and Path Configuration

The extension uses a specific prefix logic to extract configuration from the Meltano environment. By default, it looks for variables prefixed with SUPERSET_.
Sources: superset_ext/extension.py:21, superset_ext/utils.py:64-70

Key Configuration Parameters

These parameters are typically defined in the config block of the meltano.yml file:

Parameter	Default / Recommended Value	Description
HOME	$MELTANO_PROJECT_ROOT/.meltano/utilities/superset	Directory for Superset data and internal files.
CONFIG_PATH	$MELTANO_PROJECT_ROOT/superset/superset_config.py	Path where the generated Python config will be stored.
SQLALCHEMY_DATABASE_URI	sqlite:////.../superset.db	Connection string for the Superset metadata database.
WEBSERVER_PORT	8088	The port on which the Superset UI will listen.

Sources: README.md:33-41, superset_ext/utils.py:73-100

Internal Path Resolution Logic

The extension dynamically builds the execution environment by mapping these keys:

1. HOME in meltano.yml becomes SUPERSET_HOME.
2. CONFIG_PATH in meltano.yml becomes SUPERSET_CONFIG_PATH.
3. If FLASK_APP is not set, it defaults to superset.
   Sources: superset_ext/utils.py:84-96

Configuration Generation Lifecycle

When meltano invoke superset_ext:initialize is called, the extension executes a multi-step process to prepare the Superset environment.

The _write_config method converts the config dictionary from meltano.yml into a Python module (superset_config.py) that Superset can import. It iterates through the dictionary and uses setattr on the module for any key that is uppercase.
Sources: superset_ext/extension.py:46-91, superset_ext/utils.py:118-142

Configuration Script Snippet

The generated superset_config.py follows this structure:

import sys
module = sys.modules[__name__]
config = {'ROW_LIMIT': 42, 'SECRET_KEY': '...'}
for key, value in config.items():
    if key.isupper():
        setattr(module, key, value)

Sources: superset_ext/utils.py:129-135

Summary

The meltano.yml configuration for meltano-superset-ext provides a structured way to manage Apache Superset via Meltano. By defining custom commands and leveraging the extension's ability to translate YAML configuration into a functional superset_config.py file, it ensures that the Superset instance is correctly initialized and integrated into the Meltano project lifecycle.
Sources: README.md:1-60, superset_ext/extension.py:1-180