Protocols Definition
Relevant source files
The following files were used as context for generating this wiki page:
Protocols in the flext-dbt-oracle-wms project define the structural contracts for integration points between Oracle Warehouse Management System (WMS) data extraction and DBT transformation workflows. These protocols leverage Python's typing.Protocol and @runtime_checkable decorator to ensure that various components—such as clients, transformers, and executors—adhere to expected interfaces without strict inheritance requirements.
The scope of these protocols covers the full lifecycle of data movement: connection validation, entity discovery, data extraction, and subsequent DBT model execution and testing. This architectural approach facilitates decoupling between the core logic and external dependencies like Meltano or specific Oracle WMS API versions.
Sources: src/flext_dbt_oracle_wms/protocols.py:1-15, src/flext_dbt_oracle_wms/_utilities/client.py:15-30
Core Protocol Architecture
The project organizes its interfaces under the FlextDbtOracleWmsProtocols namespace. This namespace inherits from both flext_meltano and flext_oracle_wms base protocols, creating a unified contract for the combined WMS-to-DBT integration.
Client Protocol
The DbtOracleWms.Client protocol defines the public surface for the Oracle WMS workflow. It includes methods for discovering entities, extracting data, and managing the end-to-end pipeline.
The diagram above illustrates the primary protocol definitions used to enforce structural typing across the integration layers.
Sources: src/flext_dbt_oracle_wms/protocols.py:18-72
Workflow Execution and Implementation
Implementations like FlextDbtOracleWmsClient use these protocols to orchestrate complex data flows. The workflow typically involves checking connectivity, fetching raw data via a WMS client, and passing that data to a DBT transformer.
Defined Protocols Summary
| Protocol Name | Purpose | Key Methods |
|---|---|---|
DbtOracleWms.Client | Workflow orchestration | discover_oracle_wms_entities, extract_oracle_wms_data, run_full_oracle_wms_to_dbt_pipeline |
DbtOracleWms.Dbt | DBT operation execution | run_dbt_models, test_dbt_models |
WmsIntegration | Data extraction stages | extract_wms_inventory_data |
Sources: src/flext_dbt_oracle_wms/protocols.py:22-72
Transformation and Business Logic
The transformation phase is governed by protocols that ensure data retrieved from Oracle WMS is mapped correctly to DBT models. The FlextDbtOracleWmsTransformer class provides a concrete implementation of these transformation requirements, handling items and business rule validation.
This sequence shows how the workflow facade interacts with protocol-compliant clients and services to execute a full pipeline run.
Sources: src/flext_dbt_oracle_wms/_simple_api_workflow.py:30-100, src/flext_dbt_oracle_wms/_utilities/client.py:65-95
Domain-Specific Protocols
In addition to the core integration protocols, the project defines specialized protocols for modeling and utilities.
Model Generator Protocol
The ModelGenerator is responsible for creating DBT model metadata dynamically. It produces DbtModel objects that contain SQL content, schema information, and WMS-specific business rules.
- Key Functionality:
generate_wms_staging_models(oracle_sources)returns a sequence of protocol-compliant model definitions. - Business Rule Validation: Protocols ensure that every record extracted undergoes a
validate_business_rulescheck before transformation.
Sources: src/flext_dbt_oracle_wms/models.py:133-178, src/flext_dbt_oracle_wms/_utilities/client.py:145-165
Test Protocol Extensions
The project extends these protocols into the testing domain via TestsFlextDbtOracleWmsProtocols. This allows for the creation of mock clients and services in unit tests that satisfy the same structural requirements as production code.
Sources: tests/protocols.py:15-25
Conclusion
Protocols Definition serves as the foundational contract for the flext-dbt-oracle-wms project. By defining clear interfaces for clients, DBT executors, and data transformers, the project ensures type safety and modularity. This allows the system to remain flexible, supporting various database backends (DuckDB, Postgres) and Oracle WMS entity types while maintaining a consistent operational workflow.