Extension Entrypoint

Relevant source files

The following files were used as context for generating this wiki page:

• airflow_ext/main.py
• airflow_ext/wrapper.py
• airflow_ext/pass_through.py
• pyproject.toml
• README.md

The Extension Entrypoint serves as the primary interface for the Meltano Airflow extension, bridging the Meltano environment with the Apache Airflow CLI. It leverages the Meltano Extension Developer Kit (EDK) to provide standardized commands for initializing, describing, and invoking Airflow within a Meltano project context.

The entrypoint is split into two distinct CLI tools: airflow_extension, which handles extension-specific management tasks, and airflow_invoker, which provides a direct pass-through to the underlying Airflow binary.

Sources: pyproject.toml:34-36, airflow_ext/main.py:1-25

CLI Entrypoints

The project defines two main script entrypoints in pyproject.toml that allow users to interact with the extension via the command line.

Script Name	Function	Target
airflow_extension	Primary extension management CLI (initialize, describe, invoke)	airflow_ext.main:app
airflow_invoker	Direct shim for passing commands to the Airflow binary	airflow_ext.pass_through:pass_through_cli

Sources: pyproject.toml:34-36, airflow_ext/pass_through.py:11-20

Extension Management (airflow_extension)

The main management interface is built using typer and provides structured commands for lifecycle management. It initializes an instance of the Airflow wrapper class to perform its operations.

The diagram shows the mapping between CLI commands in main.py and the corresponding methods in the Airflow wrapper class.
Sources: airflow_ext/main.py:27-77, airflow_ext/wrapper.py:27-104

Core Command Logic

Initialization

The initialize command prepares the Airflow environment. It is responsible for creating the airflow.cfg file, initializing or migrating the metadata database, and deploying the Meltano DAG generator.

• Config Generation: Runs airflow config list --defaults to capture the default configuration and writes it to the designated AIRFLOW_CONFIG path.
• Database Setup: Detects the Airflow version. If version 2.x is detected, it runs db init. If version 3.x is detected, it runs db migrate.
• DAG Generation: Copies meltano_dag_generator.py (sourced from airflow_ext/files/orchestrate/meltano.py) into the Airflow DAGs folder.

Sources: airflow_ext/wrapper.py:61-90, airflow_ext/wrapper.py:106-151

Invocation and Pass-through

The extension provides two ways to invoke Airflow:

1. Managed Invoke: Through airflow_extension invoke, which uses the EDK's pass_through_invoker to execute Airflow commands while maintaining extension-level logging.
2. Direct Invoker: Through airflow_invoker, which acts as a lightweight shim using pass_through_logging_config before executing the command.

The sequence diagram illustrates how both entrypoints ultimately utilize the Airflow wrapper to execute the underlying binary.
Sources: airflow_ext/main.py:48-60, airflow_ext/pass_through.py:11-20, airflow_ext/wrapper.py:100-104

Environment Configuration

The entrypoint relies heavily on environment variables to locate Airflow components and configure its behavior.

Variable	Description	Source/Default
AIRFLOW_HOME	The base directory for Airflow	os.environ
AIRFLOW_CONFIG	Path to the airflow.cfg file	$AIRFLOW_HOME/airflow.cfg
AIRFLOW__CORE__DAGS_FOLDER	Path to the Airflow DAGs directory	$AIRFLOW_HOME/dags
LOG_LEVEL	Logging verbosity for the extension	Default: INFO

Sources: airflow_ext/wrapper.py:40-52, airflow_ext/main.py:82-89, README.md:27-35

Summary

The Extension Entrypoint provides the necessary scaffolding to run Apache Airflow as a Meltano utility. By separating management commands (initialize, describe) from execution shims (invoke), it ensures that users can easily set up the Airflow environment while retaining full access to the standard Airflow CLI via the airflow_invoker script.