Type Definitions
Relevant source files
The following files were used as context for generating this wiki page:
Type Definitions in the FLEXT Meltano project provide a unified, enterprise-grade typing system designed to ensure strict type safety across Go-to-Python bridge operations, Meltano orchestrations, and Singer protocol implementations. This system centralizes domain-specific types, structural interfaces, and protocol contracts to prevent runtime errors and ensure consistency across the data integration platform.
The typing architecture is structured into a flattened namespace (t.Meltano) that removes legacy nested aliases in favor of canonical flext_cli mappings and specialized domain types. This includes explicit definitions for dbt manifests, Singer catalogs, and project lifecycle states.
Sources: src/flext_meltano/_typings/domains.py, tests/unit/test_typings.py:17-25
Domain-Specific Types
The project defines specific types for the various subsystems involved in the Meltano ecosystem, including dbt, Singer, and general pipeline processing.
Subsystem Mappings
Domain types utilize t.MappingKV and t.JsonMapping to define contracts for service payloads and configuration data.
| Type Name | Base Definition | Description |
|---|---|---|
NestedStrMapping | t.MappingKV[str, t.StrMapping] | Used for complex configuration nesting. |
EnvironmentInput | c.Meltano.Environment | c.Meltano.EnvironmentAlias | Validated environment identifiers. |
ServicePayload | t.JsonMapping | Standard JSON structure for service responses. |
DbtManifestData | t.JsonMapping | Raw data structure of a dbt manifest file. |
DbtProject | t.JsonMapping | Configuration structure for dbt projects. |
Sources: src/flext_meltano/_typings/domains.py:16-20
Flattened Namespace Architecture
The project enforces a flat namespace for types to avoid deeply nested imports. This architecture explicitly removes legacy aliases like MeltanoConfigDict or SingerCatalogEntry in favor of centralized primitives and models.
The diagram above shows the migration from legacy nested type aliases to modern flattened JSON mapping contracts and specific Pydantic models.
Sources: tests/unit/test_typings.py:64-83, tests/unit/test_plugin_protocols.py:44-52
Protocol Definitions
Protocols define the expected structural interfaces for core project components, enabling runtime checkable type safety without strict inheritance.
Project and Adapter Protocols
- Project: Represents a Meltano project capable of finding plugins and providing its root directory.
- Adapter: Interface for data components (taps/targets) that manage connection lifecycles.
- IndexedPlugin: Metadata interface for discovered plugins, including variants and logo URLs.
Data Models and Enumerations
Typed constants and Pydantic models define the lifecycle and structure of pipeline operations.
Operation Enums
The project uses StrEnum to define the single source of truth for statuses and categories.
| Enum Category | Example Values | Purpose |
|---|---|---|
PluginType | extractors, loaders, transforms | Categorization of Meltano plugins. |
OperationStatus | created, running, success, error | Lifecycle tracking of subprocesses. |
SingerMessageType | RECORD, SCHEMA, STATE | Singer protocol message identification. |
Sources: src/flext_meltano/_constants/enums.py:17-57, src/flext_meltano/_constants/enums.py:149-160
Project Configuration Models
The PipelineProjectModel uses these types to validate the maturity and environment consistency of a data project. It specifically checks for production environment markers and calculates project_maturity levels (BASIC, DEVELOPING, MATURE).
Sources: src/flext_meltano/_models/projects.py:45-98
dbt Manifest Structure
The system provides a typed model for dbt manifest nodes, ensuring that fully qualified names (FQN) are handled as sequences and string properties.
class DbtManifestNode(m.FlexibleModel):
name: str | None
path: str | None
fqn: t.StrSequence
resource_type: str = ""
@property
def fqn_string(self) -> str:
return ".".join(self.fqn)
Sources: src/flext_meltano/_models/projects.py:19-41
Summary
Type Definitions in FLEXT Meltano serve as the foundational contract for the Go-Python bridge. By leveraging flattened namespaces, structural protocols, and strict Pydantic models, the project ensures that data pipeline configurations, plugin discovery results, and transformation manifests are validated before execution, significantly reducing runtime failures in production environments.