Constants & Base Utilities
Relevant source files
The following files were used as context for generating this wiki page:
The Constants & Base Utilities module provides the foundational infrastructure for the flext-dbt-oracle-wms package. It centralizes project-wide configuration values, materialization types, and shared helper classes that facilitate the extraction of Oracle WMS data and its subsequent transformation via DBT. This module acts as a bridge between the core flext-meltano logic and the specific domain requirements of Oracle Warehouse Management Systems (WMS).
This module ensures consistency across the 5-layer architecture by defining standard schemas, materialization strategies, and workflow instrumentation helpers. It integrates tightly with the project settings to provide a unified interface for both the CLI and programmatic access.
Sources: src/flext_dbt_oracle_wms/constants.py, src/flext_dbt_oracle_wms/utilities.py, dbt_project.yml
Global Constants and Enums
The project utilizes a hierarchical constant structure that inherits from both flext-meltano and flext-oracle-wms base domains. This allows the package to leverage universal data engineering constants while specializing for WMS-to-DBT workflows.
DbtOracleWms Constants
The FlextDbtOracleWmsConstants class defines core project identifiers and DBT-specific enumerations.
| Constant / Enum | Value / Type | Description |
|---|---|---|
PROJECT_NAME | "flext_dbt_oracle_wms" | The canonical name of the DBT project. |
Materialization.TABLE | "table" | DBT table materialization strategy. |
Materialization.VIEW | "view" | DBT view materialization strategy. |
Materialization.INCREMENTAL | "incremental" | DBT incremental materialization for large datasets. |
Sources: src/flext_dbt_oracle_wms/constants.py:14-30
Heritage and Inheritance
The following diagram illustrates how constants are aggregated from multiple upstream domains:
The project employs dual inheritance to satisfy both data integration (Meltano) and source system (Oracle WMS) requirements.
Sources: src/flext_dbt_oracle_wms/constants.py:11-14
Base Utilities and Workflow Services
The utility layer provides specialized services for metadata extraction, workflow monitoring, and performance recommendations. These are organized under the FlextDbtOracleWmsUtilities (aliased as u) namespace.
Workflow Service Helper
The u.DbtOracleWms.Service class handles the instrumentation and logging of WMS-to-DBT pipelines.
- Workflow Tracking: Generates tracking payloads that include the workflow name, type, and involved entities.
- Performance Recommendations: Evaluates the number of entities being processed against a threshold (
PERFORMANCE_RECOMMENDATION_THRESHOLD = 20). If the batch size is exceeded, it recommends smaller batches to optimize DBT execution. - Completion Logging: Provides standardized logging for success and failure states of long-running workflows.
Sources: src/flext_dbt_oracle_wms/utilities.py:16-86
Data Flow for Workflow Instrumentation
The following sequence diagram shows how the utility service tracks a typical execution:
Client and Runtime Orchestration
The FlextDbtOracleWmsClient is the primary orchestrator that coordinates between the Oracle WMS source client and the DBT transformation runner.
Client Responsibilities
- Entity Discovery: Interrogates the Oracle WMS domain to find available data objects (e.g.,
allocation,inventory,shipment). - Data Extraction: Retrieves records based on filters and maps them to standard configuration mappings.
- Pipeline Coordination: Executes the full "Discover -> Extract -> Validate -> Transform" sequence.
- Health Checks: Validates connectivity to the Oracle WMS base URL and checks the environment status.
Sources: src/flext_dbt_oracle_wms/_utilities/client.py:21-125
Transformation Workflow
The client manages the transition from raw records to DBT-managed tables through FlextMeltanoLibraryRunner.
Project Configuration Defaults
The dbt_project.yml file defines the default behavior for the transformation layer, which is influenced by the constants defined in Python.
| Feature | Default Setting | Source |
|---|---|---|
| Project Profile | flext_oracle_wms | dbt_project.yml:6 |
| Base Materialization | table | dbt_project.yml:25 |
| Staging Materialization | view | dbt_project.yml:30 |
| Audit Logging | Enabled (true) | dbt_project.yml:95 |
| Schema (Base) | wms | dbt_project.yml:26 |
| Schema (Staging) | wms_staging | dbt_project.yml:31 |
Sources: dbt_project.yml:1-115
Implementation Example: Utility Usage
Utilities are typically accessed via the u alias throughout the package for concise implementation of logging and monitoring.
# Example of workflow tracking in the utility service
def track_workflow_execution(
self,
workflow_name: str,
workflow_type: str,
entity_names: t.StrSequence | None = None,
additional_data: t.ConfigValueMapping | None = None,
) -> m.Dict:
"""Return tracking payload for workflow instrumentation."""
u.fetch_logger(__name__).info("Tracking workflow execution")
return m.Dict.model_validate({
"workflow_name": workflow_name,
"workflow_type": workflow_type,
"entity_names": ",".join(entity_names or []),
"additional_data": str(additional_data or {}),
"tracking_id": f"{workflow_name}:{workflow_type}",
"status": "running",
})
Sources: src/flext_dbt_oracle_wms/utilities.py:68-86
Conclusion
The Constants & Base Utilities module provides the necessary glue to standardize Oracle WMS data operations within a DBT context. By centralizing materialization strategies, workflow service helpers, and orchestration clients, it ensures that data flows predictably from extraction to business-ready marts while maintaining high standards for observability and performance.
Sources: src/flext_dbt_oracle_wms/constants.py, src/flext_dbt_oracle_wms/utilities.py, src/flext_dbt_oracle_wms/_utilities/client.py