Simple API Architecture
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Simple API Architecture in flext-dbt-oracle-wms provides a unified, high-level facade for managing data workflows between Oracle Warehouse Management Systems (WMS) and DBT (Data Build Tool) transformations. It abstracts complex orchestration logic, including entity discovery, data extraction, validation, and DBT execution, into a streamlined interface designed for developer consumption.
The architecture is built upon a tiered inheritance structure where base logic, metadata operations, and workflow orchestration are separated into distinct modules. This ensures clear separation of concerns while providing a single public entry point for executing end-to-end data pipelines.
Sources: src/flext_dbt_oracle_wms/_simple_api_base.py, src/flext_dbt_oracle_wms/_simple_api_workflow.py, src/flext_dbt_oracle_wms/_utilities/client.py
Core Components
The architecture consists of three primary layers that form the internal API hierarchy, supported by a specialized orchestration client.
Component Hierarchy
The following class diagram illustrates the inheritance and relationship between the core API components.
The hierarchy ensures that high-level workflow methods have access to both metadata utilities and base extraction functions.
Sources: src/flext_dbt_oracle_wms/_simple_api_base.py:17-25, src/flext_dbt_oracle_wms/_simple_api_metadata.py:16-18, src/flext_dbt_oracle_wms/_simple_api_workflow.py:16-18
Key Classes and Descriptions
| Class | Responsibility | Source |
|---|---|---|
FlextDbtOracleWmsBase | Manages shared runtime dependencies (settings, client, service) and low-level entity access. | _simple_api_base.py:17 |
FlextDbtOracleWmsMetadata | Handles direct Oracle WMS entity lookups and metadata extraction for items and shipments. | _simple_api_metadata.py:16 |
FlextDbtOracleWmsWorkflow | Orchestrates end-to-end execution, tracking, and service lifecycle operations. | _simple_api_workflow.py:16 |
FlextDbtOracleWmsClient | The underlying driver that interacts with the real Oracle WMS API and Meltano runners. | _utilities/client.py:20 |
Service | A helper class within the utilities namespace for tracking, logging, and workflow recommendations. | utilities.py:53 |
Data Workflow Logic
The primary workflow facilitated by this architecture is the transformation of Oracle WMS data into DBT models. This process involves multiple stages of validation and coordination.
Workflow Execution Flow
The run_oracle_wms_to_dbt_workflow method coordinates the sequence of operations from initial tracking to final transformation.
Client Orchestration
The FlextDbtOracleWmsClient acts as the engine for the architecture. It performs the following technical steps during a full pipeline run:
- Discovery: Fetches available entity types from the Oracle WMS client.
- Extraction: Pulls records for specific entities (e.g., items, shipments).
- Validation: Ensures records meet required field criteria defined in settings.
- Transformation: Uses
FlextDbtOracleWmsTransformerandFlextMeltanoLibraryRunnerto execute DBT transformations.
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:65-103
Configuration and Settings
The architecture relies on FlextDbtOracleWmsSettings for environment configuration. These settings are injected or fetched globally across all components.
| Setting | Description | Relevant Usage |
|---|---|---|
oracle_wms_base_url | The endpoint for the Oracle WMS instance. | _utilities/client.py:192 |
required_fields_per_entity | Mapping of entities to their mandatory fields for validation. | _utilities/client.py:155 |
dbt_threads | Number of threads to use during DBT transformation. | utilities.py:84 |
dbt_target | The DBT target environment (e.g., dev, prod). | utilities.py:85 |
Sources: src/flext_dbt_oracle_wms/_utilities/client.py, src/flext_dbt_oracle_wms/utilities.py
Interface Protocols
The system uses Python Protocols to define strict interfaces for integration points, ensuring that components like the Client or WmsIntegration handlers remain interchangeable and testable.
@runtime_checkable
class Client(Protocol):
"""Protocol for the public Oracle WMS workflow client surface."""
def discover_oracle_wms_entities(self) -> p.Result[t.StrSequence]: ...
def extract_oracle_wms_data(
self,
entity_name: str,
filters: t.ConfigurationMapping | None = None,
) -> p.Result[Sequence[t.ConfigurationMapping]]: ...
def run_full_oracle_wms_to_dbt_pipeline(
self,
entity_names: t.StrSequence | None = None,
filters: t.ConfigurationMapping | None = None,
model_names: t.StrSequence | None = None,
) -> p.Result[m.Dict]: ...
Sources: src/flext_dbt_oracle_wms/protocols.py:22-54
Summary
The Simple API Architecture provides a robust, layered framework for integrating Oracle WMS data into modern analytics stacks via DBT. By separating concerns into base management, metadata discovery, and workflow orchestration, the system maintains high modularity. The use of strict protocols and a centralized client ensures that complex operations like data validation and multi-entity extraction are handled consistently across the entire package.