Dependency Management
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Dependency management in the superset-ext project is handled through a dual-layered approach. The extension itself uses Poetry for internal package management and build configuration, while the deployment within Meltano projects relies on Meltano's pip_url mechanism to install the extension alongside specific versions of Apache Superset and its required compatibility shims.
This system ensures that the Meltano Extension SDK (EDK) is properly integrated while allowing end-users to define the specific versions of the core Superset application and its environment-specific dependencies (such as Flask or Werkzeug) at the utility level.
Build System and Core Dependencies
The project utilizes poetry-core as its build backend and defines requirements in pyproject.toml. Core dependencies focus on CLI interaction, configuration validation, and integration with the Meltano EDK.
The diagram above illustrates the primary libraries required to build and run the extension logic.
Sources: pyproject.toml:10-18, pyproject.toml:44-46
Primary Runtime Dependencies
The following table details the required libraries for the extension's execution:
| Dependency | Version Constraint | Purpose |
|---|---|---|
python | <3.11, >=3.7 | Base runtime environment |
meltano.edk | v0.1.0 (Git) | Extension Development Kit for Meltano integration |
structlog | ^21.2.0 | Structured logging |
pydantic | ^1.9.0 | Data validation and settings management |
typer | ^0.6.1 | CLI development |
PyYAML | ^6.0.0 | YAML configuration parsing |
Sources: pyproject.toml:10-18
Plugin Installation and External Dependencies
In a production Meltano environment, the extension acts as a wrapper. The README.md specifies that several external packages must be co-installed to ensure compatibility between Apache Superset 2.0.0 and its sub-dependencies.
Required Installation Stack
Meltano manages these dependencies via the pip_url in the meltano.yml configuration:
apache-superset==2.0.0flask==2.0.3werkzeug==2.0.3jinja2==3.0.1wtforms==2.3.3- `git+https://github.com/meltano/superset-ext.git%40main%60
Sources: README.md:14-16
Initialization Flow
The dependency on apache-superset is exercised during the initialize phase, where the extension invokes the superset binary to perform database migrations and permission creation.
Development and Quality Tooling
Development dependencies are managed separately to ensure a lean production footprint. These tools focus on testing and type safety.
| Tool | Version | Purpose |
|---|---|---|
pytest | ^7.2.1 | Unit and integration testing |
mypy | ^0.991 | Static type checking |
isort | (Profile: black) | Import sorting |
The project configuration enforces strict type checking through mypy, specifically disallowing untyped definitions and checking untyped decorators.
Sources: pyproject.toml:20-22, pyproject.toml:29-37
Executable Entry Points
The extension defines two primary entry points that allow Meltano to interact with the managed dependencies:
- superset_extension: Points to
superset_ext.main:app. This handles extension-specific logic likeinitializeandcreate-admin. - superset_invoker: Points to
superset_ext.pass_through:pass_through_cli. This acts as a shim for the underlyingsupersetCLI.
Sources: pyproject.toml:56-58, superset_ext/pass_through.py:12-19, superset_ext/main.py:22-26
Summary
Dependency management in superset-ext is designed to bridge the gap between the Meltano ecosystem and the complex requirements of Apache Superset. By utilizing Poetry for internal needs and Meltano's pip_url for deployment, the extension ensures that specific versions of web frameworks (Flask, Werkzeug) are pinned to prevent breaks in the core Superset application while maintaining access to the Meltano EDK.