Error Handling Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• examples/03_error_handling.py
• tests/unit/test_features.py
• examples/README.md
• src/flext_target_oracle_wms/_utilities/helpers.py
• tests/unit/test_target.py
• tests/unit/test_workflow.py

The error handling strategy in flext-target-oracle-wms is designed for mission-critical Oracle WMS Cloud SaaS integrations. It leverages the flext-core result pattern (r pattern) and flext-observability to ensure resilience, data integrity, and clear diagnostic paths during ETL operations.

The strategy encompasses validation of configurations, transformation safety, and graceful degradation during network or data-level failures. It utilizes explicit Result objects to handle expected failures without relying solely on exception bubbling, while still employing standard Python try-except blocks for unexpected system-level errors.

Sources: examples/README.md:95-102, src/flext_target_oracle_wms/_utilities/helpers.py:22-38

Core Error Handling Mechanics

The Result Pattern (r Pattern)

The primary mechanism for handling expected failures is the Result object. Instead of throwing exceptions for anticipated issues (like a missing stream registration), methods return a p.Result which contains either a success value or an error description. This allows the caller to check .success or .failure properties and handle the outcome programmatically.

The diagram shows the logical flow of handling operations that return Result objects instead of raising exceptions.

Sources: examples/03_error_handling.py:26-30, src/flext_target_oracle_wms/_utilities/helpers.py:36-38, tests/unit/test_target.py:126-130

Multi-Layered Validation

Errors are caught at various stages of the data pipeline:

1. Configuration Validation: Ensures all required credentials and settings (e.g., base_url, username) are present before initialization.
2. Schema Validation: Validates that RECORD messages are preceded by valid SCHEMA messages; orphan records trigger a failure result.
3. Data Transformation Safety: The WMSTypeConverter handles data type mismatches (e.g., non-numeric values in numeric fields) by safely falling back to string representations or returning failure results.

Sources: src/flext_target_oracle_wms/_utilities/helpers.py:25-40, tests/unit/test_target.py:126-130, tests/unit/test_workflow.py:108-112

Resilience Patterns

The system implements several high-level patterns to manage failures in production environments:

Pattern	Description	Implementation Details
Circuit Breaker	Prevents the system from repeatedly trying an operation likely to fail.	Enabled via configuration circuit_breaker_enabled.
Exponential Backoff	Retries failed network operations with increasing wait times.	Configurable via max_retries and exponential_backoff.
Rollback on Error	Ensures data consistency by rolling back partial batches.	Controlled by rollback_on_error setting.
Auto-Recovery	Attempts to re-establish connections or session states.	Configurable via auto_recovery flag.

Sources: examples/README.md:120-136

Implementation Components

Resilience Workflow

The workflow involves monitoring, logging, and structured error propagation.

This diagram illustrates the interaction between decorators, result-based helpers, and the logging system.

Sources: examples/03_error_handling.py:23-35, examples/README.md:120-125

Critical Functions and Methods

• validate_wms_target_config(settings): Validates presence of base_url, username, and password. Returns r[bool].fail if missing.
  • Sources: src/flext_target_oracle_wms/_utilities/helpers.py:25-38
• convert_singer_to_oracle(singer_type, value): Wrapped in try-except for SINGER_SAFE_EXCEPTIONS to prevent transformation crashes on malformed numbers.
  • Sources: src/flext_target_oracle_wms/_utilities/helpers.py:53-65
• process_lines(lines): Stops processing immediately and returns a failure result if malformed JSON or invalid message sequences (e.g., Record before Schema) are detected.
  • Sources: tests/unit/test_target.py:145-149, tests/unit/test_workflow.py:101-105
• run_error_handling_example(): Demonstrates the use of @flext_monitor_function to wrap business logic and provide observability into errors.
  • Sources: examples/03_error_handling.py:22-35

Configuration for Error Resilience

Resilience is highly configurable through the target settings.

Key	Type	Purpose
max_retries	int	Number of retry attempts for transient errors.
exponential_backoff	bool	Whether to increase delay between retries.
circuit_breaker_enabled	bool	Enables/Disables the circuit breaker mechanism.
rollback_on_error	bool	Determines if the batch should be rolled back on a single record failure.

Sources: examples/README.md:120-130

Conclusion

The error handling strategy in this project prioritizes SSOT (Single Source of Truth) and fail-fast principles for data integrity while providing robust recovery mechanisms for infrastructure instability. By combining the Result pattern for logic flow with monitoring decorators for observability, the architecture ensures that errors are not just caught, but are handled in a way that provides clear evidence for troubleshooting without crashing the entire pipeline.