Workflow & WMS Patterns

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_target_oracle_wms/_utilities/helpers.py
• src/flext_target_oracle_wms/_utilities/client.py
• tests/unit/test_workflow.py
• tests/unit/test_wms_patterns.py
• tests/unit/test_target.py
• tests/integration/test_oracle.py

The Workflow & WMS Patterns in flext-target-oracle-wms define the end-to-end processing logic for ingesting Singer protocol messages into an Oracle Warehouse Management System (WMS) cloud environment. This system manages the transition from raw Singer SCHEMA, RECORD, and STATE messages to transformed, Oracle-compliant data structures.

The core objective is to provide a robust, production-grade integration that handles field mapping, data type conversion, and stream-to-table registration while maintaining high performance and security standards.

System Architecture & Data Flow

The architecture follows a standard Singer Target pattern, enhanced with WMS-specific transformation and mapping components. The Target runtime orchestrates various managers to process lines from standard input (stdin) or lists.

The diagram above illustrates how different Singer message types are routed through specialized handlers and utility classes to produce the final Oracle payload.
Sources: src/flext_target_oracle_wms/_utilities/client.py:126-173, src/flext_target_oracle_wms/_utilities/helpers.py:15-155

Key Components

Component	Description	File Path
Target	Orchestrates the lifecycle (setup, process, cleanup) of the target.	client.py
CatalogManager	Stores stream metadata and registered Singer schemas.	client.py
WMSTableManager	Manages mappings between Singer stream names and Oracle table names (uppercased).	helpers.py
WMSDataTransformer	Handles the transformation of record keys (uppercasing) and value conversions.	helpers.py
WMSTypeConverter	Converts Singer scalar types to Oracle-friendly JSON or string formats.	helpers.py

Sources: src/flext_target_oracle_wms/_utilities/client.py:24-110, src/flext_target_oracle_wms/_utilities/helpers.py:44-142

Workflow Lifecycle

The target execution follows a strict lifecycle of initialization, message processing, and resource cleanup.

1. Initialization and Setup

During setup(), the target validates the provided configuration, including required fields like base_url, username, and password. It initializes internal state for schemas and table mappings.

2. Message Processing Loop

The process_lines method iterates through input lines, parsing them as JSON and dispatching them based on the type field.

• SCHEMA Messages: Registers the stream in the CatalogManager and maps the stream name to an uppercase table name in the WMSTableManager.
• RECORD Messages: Requires a previously registered schema. The record is transformed via the WMSDataTransformer.
• STATE Messages: Logs the current state for bookmarking and progress tracking.

# Example of processing sequence in tests
def test_schema_then_record_then_state() -> None:
    target = u.TargetOracleWms.Target(_valid_config())
    lines = [
        _schema_line("orders", {"id": {"type": "string"}}),
        _record_line("orders", {"id": "1", "name": "test"}),
        _state_line({"bookmarks": {"orders": "1"}}),
    ]
    result = target.process_lines(lines)
    assert result.success

Sources: src/flext_target_oracle_wms/_utilities/client.py:180-218, tests/unit/test_target.py:141-153

WMS Pattern Implementations

Data Transformation & Uppercasing

Oracle WMS typically requires uppercase identifiers for table and column names. The WMSTableManager and WMSDataTransformer enforce this by default.

• Stream to Table: A stream named inventory is registered as the table INVENTORY.
• Field Mapping: A record field qty is transformed to QTY during processing.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:139-141, src/flext_target_oracle_wms/_utilities/helpers.py:102-104

Type Conversion Rules

The WMSTypeConverter ensures data compatibility between Singer types and Oracle requirements:

Singer Type	Oracle Target Format	Transformation Logic
string	str	Direct string representation.
integer	int	Numeric parsing; falls back to string on error.
number	float	Handled as float if a decimal is present.
boolean	string	Converted to literal "True" or "False".
object/array	JSON string	Serialized to a UTF-8 encoded JSON string.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:44-64, tests/unit/test_wms_patterns.py:53-90

Multi-Stream Handling

The system supports interleaved messages from multiple streams. The CatalogManager maintains a registry of all schemas encountered during the run, allowing the StreamProcessor to retrieve the correct transformation context for any record.

Summary

The Workflow & WMS Patterns module provides the core logic for Singer-to-Oracle data integration. By centralizing transformation rules—such as automatic field uppercasing and robust type conversion—it ensures that incoming data streams are normalized for the Oracle WMS Cloud environment while adhering to the standard Singer protocol for interoperability.