Database Configuration

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• superset_ext/utils.py
• superset_ext/extension.py
• superset_ext/main.py
• tests/test_superset_ext_utils.py

Database configuration in the superset-ext utility manages the setup, path resolution, and initialization of the Apache Superset metadata database within a Meltano environment. The extension automates the creation of necessary directories, handles SQLAlchemy URI mapping, and executes database migrations to ensure the environment is ready for the Superset application.

The system primarily relies on environment variables prefixed with SUPERSET_ to determine the database location and connection parameters. By default, it targets a SQLite database located within the Meltano project's utility storage.
Sources: superset_ext/extension.py:48-73, superset_ext/utils.py:65-98

Database URI and Path Management

The extension resolves the database connection string using the SQLALCHEMY_DATABASE_URI configuration. If no URI is provided in the environment, it defaults to a SQLite instance stored in the Meltano project root.

Default Path Resolution

When variables are not explicitly set, the extension constructs paths relative to MELTANO_PROJECT_ROOT. The default database path is sqlite:///$MELTANO_PROJECT_ROOT/.meltano/utilities/superset/superset.db. The initialization process automatically extracts the file path from this URI to ensure the parent directories exist before attempting to run migrations.
Sources: superset_ext/utils.py:75-96, superset_ext/extension.py:59-61

Configuration Mapping

Environment variables are processed and transformed to match Superset's expected internal configuration keys.

Meltano Config Key	Superset Internal Key	Default Value (relative to project root)
SQLALCHEMY_DATABASE_URI	SQLALCHEMY_DATABASE_URI	sqlite:///.../superset.db
HOME	SUPERSET_HOME	.meltano/utilities/superset
CONFIG_PATH	SUPERSET_CONFIG_PATH	analyze/superset/superset_config.py

Sources: superset_ext/utils.py:80-96, README.md:27-33

Initialization Logic

The database initialization is triggered via the initialize command. This process follows a specific sequence to ensure the schema is applied and permissions are correctly set in the Superset metadata store.

The flow ensures that the filesystem is prepared before the underlying Superset CLI tools are invoked. The superset db upgrade command is responsible for running the Alembic migrations that build the database schema.
Sources: superset_ext/extension.py:48-83, superset_ext/utils.py:101-133

Migration and Permission Handling

The extension uses an Invoker to execute the following database-related operations:

1. db upgrade: Synchronizes the database schema with the current version of Superset.
2. init: Initializes the Superset application, including default roles and permissions.
3. fab create-permissions: Specifically creates Flask-AppBuilder permissions required for access control.

Sources: superset_ext/extension.py:62-83

Configuration Persistence

The extension writes a Python configuration file (superset_config.py) that Superset loads at runtime. This file dynamically injects all environment-based configuration into the Superset module scope.

# Generated by write_config in superset_ext/utils.py
import sys
module = sys.modules[__name__]
config = {'SQLALCHEMY_DATABASE_URI': '...', 'ROW_LIMIT': 42}
for key, value in config.items():
    if key.isupper():
        setattr(module, key, value)

This mechanism ensures that the SQLALCHEMY_DATABASE_URI defined in Meltano is correctly passed to the Flask application.
Sources: superset_ext/utils.py:120-131, superset_ext/extension.py:42-46

Summary

The Database Configuration module provides a seamless bridge between Meltano's environment-driven configuration and Superset's requirement for a persistent metadata database. By automating directory creation, URI resolution, and the standard initialization suite (db upgrade, init, and create-permissions), the extension ensures a consistent and reproducible database state across different environments.
Sources: superset_ext/extension.py:59-86, tests/test_superset_ext_utils.py:34-56