Data Models and Schemas
Relevant source files
The following files were used as context for generating this wiki page:
The Meltano Extension Developer Kit (EDK) uses a structured set of data models and types to define how extensions describe themselves and interact with the Meltano ecosystem. These models ensure consistency across different extensions by providing a standardized interface for command discovery, execution arguments, and metadata reporting.
The primary purpose of these models is to support the describe and invoke capabilities of an extension. By using Python dataclasses and strict type aliases, the EDK provides a type-safe way to define extension capabilities that can be serialized into formats like JSON or YAML for consumption by Meltano.
Sources: meltano/edk/models.py, meltano/edk/extension.py:85-93
Core Command Models
The EDK defines a hierarchy of command models to represent different types of executable actions an extension can perform. These are implemented using Python dataclasses with slots=True for performance and memory efficiency.
Command Hierarchy
The base Command class provides the foundational structure, which is then specialized into ExtensionCommand for standard extension operations and InvokerCommand for pass-through CLI interactions.
The diagram above shows the inheritance relationship between the primary command types in the EDK.
Sources: meltano/edk/models.py:6-43
Model Field Definitions
| Field | Type | Description |
|---|---|---|
name | str | The unique identifier for the command. |
description | str | A human-readable summary of the command's purpose. |
commands | list[str] | A list of subcommands or methods supported by this command. |
pass_through_cli | bool | Indicates if the command should pass arguments directly to a wrapped binary. |
Sources: meltano/edk/models.py:9-14
Extension Description Schema
The Describe model is the top-level container used by an extension to report its available features. When the describe() method is called on an extension, it returns an instance of this model.
Serialization Flow
Extensions can export their description in multiple formats using the DescribeFormat enumeration.
The flow shows how an extension's internal description object is converted into various external formats.
Sources: meltano/edk/extension.py:22-28, meltano/edk/extension.py:95-121
Supported Formats
The DescribeFormat enum defines three valid output types:
- text: A human-readable representation using
devtools.prettier. - json: A standard JSON string, often used for machine-to-machine communication.
- yaml: A YAML representation, useful for Meltano configuration files.
Sources: meltano/edk/extension.py:22-28
Type System and Execution Arguments
To maintain consistency during subprocess execution, the EDK defines specific type aliases and interfaces for arguments passed to commands.
ExecArg Type
The ExecArg is a central type alias used throughout the EDK's ExtensionBase and Invoker classes.
# meltano/edk/types.py
ExecArg: TypeAlias = str | bytes
Sources: meltano/edk/types.py:7-9
Invoker Configuration
The Invoker class uses a specific schema for managing the execution environment of wrapped binaries.
| Component | Type | Description |
|---|---|---|
bin | str | Path or name of the binary to execute. |
cwd | str | None | The working directory for the subprocess. |
popen_env | dict[str, Any] | Dictionary of environment variables. Defaults to os.environ. |
Sources: meltano/edk/process.py:38-54
Project Initialization Schema
The EDK uses a template-based approach for generating new extensions. The schema for these projects is defined in the copier.yml configuration, which dictates the variables required to initialize an extension's data structure.
Template Variables
| Variable | Type | Description | Default |
|---|---|---|---|
extension_name | str | The name of the extension (e.g., MyExtension). | N/A |
extension_id | str | The unique ID for the wrapper. | {{ extension_name_lower }}-ext |
cli_prefix | str | The prefix for the CLI command. | {{ extension_name_lower }} |
wrapper_target_name | str | The name of the binary being wrapped. | N/A |
Sources: copier.yml:1-35
Summary of Data Structures
The data models in the Meltano EDK provide a rigid yet extensible framework for building Meltano-compatible extensions. By standardizing on the Command and Describe dataclasses, the EDK allows Meltano to programmatically understand the capabilities of any extension. The system is anchored by the ExtensionBase abstract class, which enforces the implementation of the describe() method, ensuring that all extensions adhere to the established metadata schemas.