Troubleshooting Guide
Relevant source files
The following files were used as context for generating this wiki page:
This guide provides technical reference and procedures for diagnosing and resolving issues within the flext-dbt-oracle-wms integration. It covers connectivity verification, data quality validation, and pipeline execution monitoring. The scope includes the orchestration layer between Oracle WMS APIs and the DBT transformation environment.
Sources: README.md:10-12, src/flext_dbt_oracle_wms/_utilities/client.py:9-25
Connectivity and Integration Health
Connectivity issues typically arise at the interface between the FlextDbtOracleWmsClient and the Oracle WMS domain client. Troubleshooting begins by verifying the environment configuration and the health of the WMS endpoint.
Connection Validation Flow
The client utilizes a health check endpoint to verify connectivity to the Oracle WMS base URL defined in the project settings.
The diagram shows the validation path for Oracle WMS connectivity.
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:92-114
Connectivity Configuration
| Parameter | Source | Description |
|---|---|---|
oracle_wms_base_url | FlextDbtOracleWmsSettings | The target API endpoint for WMS data extraction. |
oracle_wms_environment | FlextDbtOracleWmsSettings | Identifies the environment context (e.g., dev, prod). |
DBT_SOURCE_SCHEMA | Env Var | The raw schema where WMS data is initially landed. |
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:109-113, dbt_project_migrated.yml:61
Pipeline Execution Failures
Pipeline failures often occur during the automated flow of discovery, extraction, and transformation. The run_full_oracle_wms_to_dbt_pipeline method acts as the primary orchestrator.
Pipeline Logic Flow
The pipeline follows a strict sequence where failure at any stage halts subsequent operations.
Pipeline stage transitions and error handling paths.
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:48-83
Common Failure Points
- Discovery Failure: Triggered when the WMS domain client is unavailable or returns an error during entity mapping. Sources: src/flext_dbt_oracle_wms/_utilities/client.py:56
- Validation Failure: Occurs when extracted records lack required fields specified in
required_fields_per_entity. Sources: src/flext_dbt_oracle_wms/_utilities/client.py:157-164 - Transformation Failure: Errors during the execution of
FlextMeltanoLibraryRunner.run_dbt_transformation, often due to SQL errors or profile misconfiguration. Sources: src/flext_dbt_oracle_wms/_utilities/client.py:133-145
Data Quality and Consistency
Data quality issues are identified via specialized DBT tests that validate referential integrity and business logic across staging and mart layers.
Business Rule Violations
The system enforces several constraints on WMS allocation data. Violations of these rules indicate upstream data entry errors or extraction logic defects.
| Test Case | Description | Severity |
|---|---|---|
| Negative Quantities | allocated_quantity must be >= 0. | Critical |
| Future Timestamps | picked_timestamp cannot exceed current_timestamp. | Warning |
| Orphaned Allocations | Allocations must reference a valid order_dtl_id. | Critical |
| Inventory Mismatch | allocated_quantity cannot exceed available_quantity. | Critical |
Sources: tests/staging/test_stg_wms__allocation_business_rules.sql:6-12, tests/data_quality/test_cross_model_consistency.sql:7-16
Data Freshness
Data freshness is monitored via mod_ts timestamps in the source tables. Warnings are triggered if data is older than 12 hours, and errors at 24 hours.
Sources: dbt_project.yml:76-78
Performance Optimization
High latency or timeouts during execution can be addressed by adjusting threading and batching strategies.
Batch Processing Recommendations
The Service helper provides recommendations when the number of entities to process exceeds a specific threshold (default: 20).
Sources: src/flext_dbt_oracle_wms/utilities.py:22-68
DBT Profile Tuning
DBT performance is managed via the profiles.yml configuration:
- Dev: 4 threads (DuckDB).
- Staging: 8 threads (PostgreSQL).
- Prod: 16 threads (PostgreSQL/Snowflake).
Sources: profiles.yml:9-37
Summary
Troubleshooting in the flext-dbt-oracle-wms project requires a two-step approach: first, validating the API connectivity via the FlextDbtOracleWmsClient; and second, verifying data integrity using the suite of cross-model consistency tests. Monitoring the logs for tracking_id generated during track_workflow_execution provides the necessary context for identifying specific failed pipeline runs.
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:82, src/flext_dbt_oracle_wms/utilities.py:88-92