Oracle dbt Models
Relevant source files
The following files were used as context for generating this wiki page:
Oracle dbt Models provide the core data structures and logic for integrating Oracle Database with dbt (data build tool) workflows within the FLEXT ecosystem. This module facilitates the generation, configuration, and execution of dbt pipelines specifically optimized for Oracle environments, leveraging Pydantic for strict typing and validation.
The system acts as a bridge between the flext-db-oracle database layer and the flext-meltano orchestration layer, ensuring that model metadata, connection parameters, and SQL materializations are handled consistently across the data engineering lifecycle.
Sources: src/flext_dbt_oracle/models.py:11-15, rules.json:1-15, src/flext_dbt_oracle/base.py:16-20
Core Data Models
The FlextDbtOracleModels class serves as the primary namespace for all dbt-related domain objects. It inherits from both FlextMeltanoModels and the base Oracle models (m) to provide a unified schema for staging and transforming data.
Dbt Model Metadata
The DbtOracle.Model class defines the structure for dbt model definitions, including classification, materialization strategies, and dependencies.
| Field | Type | Description |
|---|---|---|
name | str | The unique dbt model name. |
dbt_model_type | str | Classification (e.g., staging, mart). |
materialization | Enum | Strategy (table, view, incremental, snapshot). |
sql_content | str | The actual SQL body of the model. |
columns | Sequence | Normalized column metadata. |
dependencies | Sequence | Upstream dbt model dependencies. |
Sources: src/flext_dbt_oracle/models.py:18-65, rules.json:43-52
Model Generation Logic
To automate the creation of staging layers, the ModelGenerator provides deterministic generation of dbt models based on source tables.
The generator creates a staging model for each provided source table, automatically naming the table with a stg_ prefix and assigning the appropriate source reference.
Sources: src/flext_dbt_oracle/models.py:67-91
Connection and Configuration
Connection management is handled through a combination of OracleConnectionConfig for instance-level parameters and FlextDbtOracleSettings for environment-driven runtime configuration.
Connection Parameters
The system supports both Service Name and SID-based connections to Oracle databases, utilizing the tcp or tcps protocols.
DSN Generation Logic
The DSN is computed dynamically based on whether a SID or a Service Name is provided:
- SID:
protocol://username:***@host:port:sid - Service Name:
protocol://username:***@host:port/service_name
Sources: src/flext_dbt_oracle/models.py:93-145, src/flext_dbt_oracle/settings.py:17-105
SQL Dialect and Materialization
The adapter strictly enforces Oracle-specific SQL dialects and materialization strategies as defined in the project rules.
Supported Materializations
The system supports the following dbt materialization types for Oracle:
- Table: Full table recreation.
- View: Virtual table creation.
- Incremental: Supporting
mergeanddelete+insertstrategies. - Snapshot: For capturing historical changes.
Data Type Mapping
dbt types are mapped to Oracle-native types to ensure schema consistency:
string->VARCHAR2(4000)integer->NUMBER(38,0)float->BINARY_DOUBLEboolean->NUMBER(1,0)timestamp->TIMESTAMP
Sources: rules.json:43-85
Pipeline Execution Utilities
The FlextDbtOracleUtilities provide a high-level Client for managing the execution of dbt pipelines and data discovery.
The client includes methods for testing connectivity, discovering metadata, and running the actual transformation pipeline for selected tables.
Sources: src/flext_dbt_oracle/utilities.py:15-62
Summary
The Oracle dbt Models module provides a robust, typed framework for managing dbt transformations on Oracle Database. By centralizing model definitions in models.py and configuration in settings.py, it allows for automated generation of staging layers and consistent connection management across different environments. The integration of rules.json ensures that all SQL generated and all data types used are strictly compliant with Oracle's dialect and performance requirements.
Sources: src/flext_dbt_oracle/models.py:11-15, src/flext_dbt_oracle/settings.py:17-25, rules.json:100-115