Invoking Native Commands

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• superset_ext/main.py
• superset_ext/extension.py
• superset_ext/pass_through.py
• superset_ext/utils.py
• pyproject.toml

The Meltano Superset extension provides a mechanism to invoke native Apache Superset commands through a wrapper layer. This system allows Meltano to manage Superset's lifecycle, configuration, and administrative tasks while still granting users full access to the underlying Superset CLI.

By utilizing the Meltano Extension Developer Kit (EDK), the project defines two primary entry points for command invocation: the superset_extension for managed commands and the superset_invoker for direct passthrough. This architecture ensures that environment variables are correctly mapped and configuration files are generated before the native superset binary is executed.

Sources: README.md:1-25, superset_ext/extension.py:30-40, pyproject.toml:56-58

Architecture of Invocation

The invocation system is built on the ExtensionBase class provided by the Meltano EDK. It uses an Invoker object to manage the execution of the native superset binary.

Components and Flow

The extension differentiates between managed commands (like initialize and create-admin) and generic pass-through commands. Managed commands often involve pre-requisite steps like writing configuration files or setting up database schemas.

The diagram shows how user requests flow through either the managed extension CLI or the direct invoker to eventually execute the native binary.
Sources: superset_ext/main.py:16-102, superset_ext/pass_through.py:12-21, superset_ext/extension.py:36-38

Core Invocation Methods

The extension implements several methods to handle different types of command execution.

The invoke Method

The invoke method in the Superset class is the primary interface for running subcommands. It accepts a command_name and variable command_args, passing them to the EDK Invoker.run_and_log method.

Component	Responsibility	Source
superset_bin	Defaults to "superset", the actual binary executed.	superset_ext/extension.py:35
superset_invoker	EDK Invoker instance initialized with specific environment config.	superset_ext/extension.py:37
run_and_log	Executes the command and pipes output to the extension's logs.	superset_ext/extension.py:127

Sources: superset_ext/extension.py:118-133, superset_ext/main.py:73-84

Passthrough Invocation

The superset_invoker script (mapped to superset_ext.pass_through:pass_through_cli) provides a lightweight wrapper that bypasses the Typer-based CLI of the extension for a more direct interaction with the native tool.

def pass_through_cli() -> None:
    pass_through_logging_config()
    ext = Superset()
    ext.pass_through_invoker(
        structlog.getLogger("superset_invoker"),
        *sys.argv[1:] if len(sys.argv) > 1 else []
    )

Sources: superset_ext/pass_through.py:12-21, pyproject.toml:58

Environment Preparation

Before any native command is invoked, the extension prepares the environment using superset_ext/utils.py. This ensures that Superset-specific environment variables are correctly prefixed and that paths are resolved.

Environment Mapping Logic

The extension looks for variables starting with SUPERSET_ and maps them to the appropriate configuration keys.

Feature	Description	File
Prefix Trimming	Removes SUPERSET_ from environment variables for internal use.	superset_ext/utils.py:13-26
Default Paths	Sets SUPERSET_HOME to .meltano/utilities/superset if not provided.	superset_ext/utils.py:74-82
Config Path	Sets SUPERSET_CONFIG_PATH to analyze/superset/superset_config.py.	superset_ext/utils.py:84-86

This sequence illustrates how environment variables are transformed before being passed to the native invoker.
Sources: superset_ext/utils.py:61-100, superset_ext/extension.py:36-37

Managed Command Initialization

Specific native commands are grouped under the initialize routine. This is a higher-level abstraction that runs a sequence of native Superset commands to ready the application.

1. db upgrade: self.superset_invoker.run("db", "upgrade", ...)
2. init: self.superset_invoker.run("init", ...)
3. create-permissions: self.superset_invoker.run("fab", "create-permissions", ...)

Sources: superset_ext/extension.py:59-92

Command Summary

The following table summarizes the available extension commands that wrap native functionality as defined in the describe method:

Command Name	Description	Source
initialize	Writes config, upgrades DB, and initializes Superset application.	superset_ext/extension.py:52-95
create-admin	Wraps fab create-admin to set up a new administrator user.	superset_ext/extension.py:97-116
invoke	Generic passthrough for any native Superset command.	superset_ext/extension.py:118-133
superset_invoker	Direct CLI invoker for raw command passthrough.	superset_ext/extension.py:155-157

Sources: superset_ext/extension.py:135-160, README.md:27-44

Conclusion

Invoking native commands in superset-ext is a structured process that leverages the Meltano EDK to ensure the environment is correctly configured. By providing both a managed Superset extension class and a direct passthrough invoker, the project maintains a balance between automated configuration management and raw access to the underlying Apache Superset CLI.