API Endpoints & Interface
Relevant source files
The following files were used as context for generating this wiki page:
The FLEXT Meltano API provides a unified, enterprise-grade interface for orchestrating data integration pipelines involving Meltano, Singer, and DBT. It serves as a Go ↔ Python bridge, exposing high-level methods for project management, plugin discovery, and command execution while enforcing strict type safety and railway-oriented programming (ROP) patterns.
The public surface is primarily exposed through the meltano singleton facade, which abstracts complex internal services into a simplified programmatic interface for developers.
Sources: src/flext_meltano/init.py, src/flext_meltano/README.md, examples/01_simple_working.py:17-18
Public API Facade (meltano)
The meltano object is the primary entry point for all library interactions. It exposes methods for managing the lifecycle of data integration projects and executing underlying Meltano CLI operations.
Core Interface Methods
| Method | Description | Return Type |
|---|---|---|
fetch_version() | Retrieves the current version of the Meltano environment. | p.Result[str] |
discover_installed_plugins() | Scans the active project for installed plugins. | p.Result[t.Sequence] |
tap(name) | Factory method to initialize a Singer Tap component. | p.Result[Tap] |
target(name) | Factory method to initialize a Singer Target component. | p.Result[Target] |
dbt(name) | Factory method to initialize a DBT transformation component. | p.Result[Dbt] |
execute() | Executes the default service logic and returns a status payload. | p.Result[t.JsonMapping] |
Sources: examples/01_simple_working.py:21-42, src/flext_meltano/init.py:41-95, tests/unit/test_api.py:20-35
API Execution Flow
The following diagram illustrates the flow of a typical API request through the public facade into the service layer.
The API uses a Result pattern (p.Result) to ensure that failures are handled explicitly without unhandled exceptions.
Sources: src/flext_meltano/services/project_service.py:126-135, tests/unit/test_api.py:31-40
Project Management Interface
The FlextMeltanoProjectService provides the backend logic for project-level API calls. It handles the creation, validation, and initialization of Meltano project structures.
Project Creation & Validation
The API supports creating both permanent and temporary projects. Temporary projects are created in system temp directories with randomized prefixes to support isolated execution environments.
Execution Payloads
When the execute() method is called on the project service, it generates a normalized health and status payload.
| Field | Type | Description |
|---|---|---|
status | str | Current operation status (e.g., "ready"). |
service_type | str | Identifier for the service (e.g., "flext_meltano_project_service"). |
settings | dict | Dump of current validated project settings. |
Sources: src/flext_meltano/services/project_service.py:115-124
Data Configuration Models
API inputs and outputs for data sources are governed by Pydantic-based models that ensure schema compliance and protect sensitive information.
Tap & Target Configuration
The API uses TapConfig and TargetConfig to define the parameters for data extraction and loading.
class TapConfig(m.Entity):
tap_type: str
connection_config: t.JsonMapping
stream_config: t.JsonMapping
tap_version: str = "latest"
Sources: src/flext_meltano/_models/sources.py:18-35
Sensitive Data Protection
All configuration models implement a serialize_connection_config method. This ensures that sensitive credentials (like passwords or API keys) are masked when the API returns configuration details in logs or responses.
Sources: src/flext_meltano/_models/sources.py:53-56, src/flext_meltano/_models/sources.py:100-103
CLI Bridge Interface
The project exposes a Command Line Interface (CLI) that maps directly to the programmatic API. This is registered as the flext-meltano entry point.
| Command Option | Meltano CLI Mapping | Description |
|---|---|---|
--version | meltano --version | Displays version info. |
--list | meltano list | Lists project plugins. |
elt | meltano elt | Executes a legacy ELT pipeline. |
invoke | meltano invoke | Invokes a specific plugin. |
Sources: src/flext_meltano/_constants/base.py:52-66, pyproject.toml:96
Summary
The FLEXT Meltano API provides a robust, type-safe interface for managing data pipelines. By utilizing a public facade (meltano) and standardized service execution payloads, it bridges the gap between Go-based orchestrators and Python-based data tools. The interface ensures reliability through rigorous parameter validation, protected sensitive data serialization, and a consistent railway-oriented result pattern.