Pre-Invoke Hooks

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• dbt_ext/extension.py
• dbt_ext/main.py
• dbt_ext/pass_through.py
• tests/test_dbt_ext.py
• dbt_ext/dbt_files/bundle/transform/dbt_project.yml

Pre-invoke hooks in the meltano-dbt-ext project are automated preparation steps executed before a primary dbt command is run. Their primary purpose is to ensure the dbt environment is clean and all necessary dependencies are up-to-date, thereby reducing manual maintenance and preventing execution failures caused by stale artifacts or missing packages.

Sources: README.md:12-18, dbt_ext/extension.py:53-58

Execution Logic and Architecture

The pre-invoke hook is implemented within the pre_invoke method of the dbt class. It acts as a wrapper that triggers supplemental dbt commands—specifically dbt clean and dbt deps—prior to the user-requested command.

Trigger Mechanism

The hook is automatically triggered whenever a command is passed through the extension's invoker. However, the system includes logic to prevent recursive loops and redundant executions. For instance, if the primary command being invoked is itself clean or deps, the pre-invoke steps are skipped.

Sources: dbt_ext/extension.py:65-71, tests/test_dbt_ext.py:21-42

Workflow Diagram

The following flowchart illustrates the decision logic applied during the pre-invoke phase:

The diagram shows the conditional checks performed before executing the core pre-invoke logic.
Sources: dbt_ext/extension.py:60-84, tests/test_dbt_ext.py:34-42

Configuration Options

Users can control the behavior of pre-invoke hooks through environment variables and Meltano plugin settings.

Setting Name	Environment Variable	Type	Default	Description
skip_pre_invoke	DBT_EXT_SKIP_PRE_INVOKE	boolean	false	Whether to skip the automatic execution of dbt clean and dbt deps.

Sources: README.md:14-18, dbt_ext/extension.py:46-48

Component Roles

Invoker Class

The dbt class utilizes a meltano.edk.process.Invoker instance to manage the execution of subprocesses. This invoker is initialized with the path to the dbt binary and the project directory.

Sources: dbt_ext/extension.py:45, dbt_ext/pass_through.py:15-18

Clean and Dependencies Management

1. dbt clean: Targeted at removing folders specified in the clean-targets list of dbt_project.yml, typically target/, dbt_packages/, and logs/.
2. dbt deps: Pulls the most recent versions of packages listed in packages.yml.

Sources: dbt_ext/dbt_files/bundle/transform/dbt_project.yml:25-28, README.md:89-94

Error Handling

If either dbt clean or dbt deps fails during the pre-invoke stage, the extension logs the error using log_subprocess_error and terminates the execution with the specific return code from the failed subprocess, preventing the primary command from running in an unstable environment.

Sources: dbt_ext/extension.py:76-79, dbt_ext/extension.py:84-87

Implementation Snippet

The following Python code represents the core logic for the pre-invoke sequence within the extension:

def pre_invoke(self, invoke_name: str | None, *invoke_args: ExecArg) -> None:
    if self.skip_pre_invoke:
        return

    if invoke_name in ["deps", "clean"]:
        return

    try:
        self.dbt_invoker.run_and_log("clean")
    except subprocess.CalledProcessError as err:
        sys.exit(err.returncode)

    try:
        self.dbt_invoker.run_and_log("deps")
    except subprocess.CalledProcessError as err:
        sys.exit(err.returncode)

Sources: dbt_ext/extension.py:53-87

Summary

Pre-invoke hooks provide a safety layer for dbt operations within Meltano by ensuring environment consistency. By automatically running clean and deps, the extension guarantees that every model run or test execution uses fresh artifacts and current package versions, unless explicitly disabled by the user via the DBT_EXT_SKIP_PRE_INVOKE setting.