High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_target_oracle_wms/models.py
• src/flext_target_oracle_wms/_utilities/helpers.py
• src/flext_target_oracle_wms/constants.py
• src/flext_target_oracle_wms/api.py
• src/flext_target_oracle_wms/utilities.py
• tests/unit/test_sinks.py
• tests/unit/test_target.py

The flext-target-oracle-wms project is a Singer-compliant target designed to load data into Oracle Warehouse Management System (WMS) Cloud. It acts as the destination in an ETL/ELT pipeline, consuming structured Singer messages (SCHEMA, RECORD, STATE) and transforming them into formats compatible with Oracle WMS REST APIs. The architecture leverages the flext-* ecosystem, specifically flext-core for result patterns, flext-meltano for Singer protocol structures, and flext-oracle-wms for entity definitions.

Sources: README.md:1-10, src/flext_target_oracle_wms/models.py:16-25

Core Component Structure

The system is organized into specialized managers and helpers that handle specific stages of the Singer message lifecycle. The primary entry point is the Target class (accessed via u.TargetOracleWms.Target), which coordinates these components.

Architectural Components

Component	Responsibility
Catalog Manager	Manages stream definitions and ensures schemas are registered before records are processed.
WMSTableManager	Maintains in-memory mappings between Singer stream names and target Oracle WMS table/entity names.
WMSDataTransformer	Sanitizes and converts Singer record payloads into Oracle-compatible formats (e.g., uppercasing keys).
WMSTypeConverter	Handles low-level scalar type conversions between JSON/Singer types and Oracle-friendly values.
WMSSchemaMapper	Translates Singer JSON schemas into normalized catalog entries for Oracle DDL or API payloads.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:17-126, tests/unit/test_sinks.py:53-70

Component Interaction Diagram

The following diagram illustrates how the Target orchestrates internal helpers when receiving messages.

Data Transformation Flow

The transformation logic ensures that data adheres to Oracle WMS requirements, such as uppercased field names and specific data type serializations.

Record Processing Sequence

When a RECORD message is received, the WMSDataTransformer iterates through the fields, using the WMSTypeConverter to resolve values based on the registered schema.

Transformation Rules

• Key Normalization: All keys in the record are converted to uppercase (e.g., name becomes NAME).
• Complex Types: object and array types are serialized to JSON strings using the DEFAULT_ENCODING (UTF-8).
• Numeric Handling: Numeric strings are attempted to be cast to float or int; if casting fails, they remain as strings.
• Schema Enforcement: Records received for streams without a previously registered SCHEMA message result in a failure.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:45-63, tests/unit/test_target.py:126-135, tests/unit/test_sinks.py:90-95

Configuration and Models

The target is configured using a structured WmsTargetConfig model, which includes authentication, batching, and load strategy settings.

Configuration Model Table

Field	Type	Default	Description
wms_auth	WmsAuthenticationConfig	N/A	Contains base_url, username, password, and auth_method.
batch_size	int	1000	The number of records collected before performing a write.
load_method	str	APPEND_ONLY	Strategy for writing (Valid values: APPEND_ONLY, UPSERT, MERGE).
company_code	str	DEFAULT	WMS organizational identifier.

Sources: src/flext_target_oracle_wms/models.py:48-75, src/flext_target_oracle_wms/constants.py

Validation Logic

The architecture includes built-in validation for both static configuration and runtime constraints:

1. Field Presence: Requires base_url, username, and password.
2. Business Rules: Ensures batch_size is positive and load_method is supported.
3. Authentication: Supports oauth2, basic, and api_key methods.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:22-38, src/flext_target_oracle_wms/models.py:77-85

Integration and Lifecycle

The target follows a strict lifecycle managed by the FlextTargetOracleWmsCli.

Execution Flow

1. Setup: Initializes internal buffers and validates configuration.
2. Process Lines: Reads Singer messages from stdin line-by-line.
3. Message Routing:
   • SCHEMA: Passed to WMSSchemaMapper and registered in WMSTableManager.
   • RECORD: Transformed by WMSDataTransformer and buffered.
   • STATE: Triggers a flush of current buffers (if applicable) and emits the state to stdout.
4. Cleanup: Finalizes any remaining buffers and closes connections.

Sources: tests/unit/test_target.py:100-115, tests/integration/test_oracle.py:48-60, tests/unit/test_workflow.py:110-125

Conclusion

The high-level architecture of flext-target-oracle-wms provides a robust, typed bridge between Singer data streams and Oracle WMS Cloud. By decoupling schema mapping, data transformation, and connection management into dedicated helper classes, the system ensures data integrity and consistency while maintaining compliance with Oracle's strict API requirements.