Database Connection Lifecycle
Relevant source files
The following files were used as context for generating this wiki page:
The Database Connection Lifecycle in the flext-dbt-oracle project manages the transition from environment configuration to active database sessions. It encompasses the ingestion of connection parameters, the validation of connection strings (DSNs), the establishment of connection pools, and health monitoring through diagnostic testing. This system ensures that dbt models can interact with Oracle databases reliably using modern oracledb drivers and specific Oracle identifiers like Service Names and SIDs.
The lifecycle is primarily driven by FlextDbtOracleSettings, which serves as the single source of truth for connection parameters, and OracleConnectionConfig, which provides the low-level data structure for session establishment. These components work together to provide a robust foundation for dbt-oracle materializations.
Sources: src/flext_dbt_oracle/settings.py:17, src/flext_dbt_oracle/models.py:75, rules.json:71
Connection Configuration and Validation
The lifecycle begins with the definition and validation of connection parameters. The system utilizes Pydantic-based models to enforce type safety and logical constraints on connection inputs, such as ensuring that the connection pool's maximum size is greater than or equal to its minimum size.
Data Structures for Connectivity
Two primary classes manage connection metadata:
FlextDbtOracleSettings: Handles global settings, environment variable mapping (using theFLEXT_DBT_ORACLE_prefix), and conversion to secondary configuration formats.OracleConnectionConfig: A specialized model representing the core technical parameters required to reach an Oracle instance.
Sources: src/flext_dbt_oracle/settings.py:17-30, src/flext_dbt_oracle/models.py:75-103
Connection Parameters Table
| Parameter | Type | Default | Description |
|---|---|---|---|
oracle_host | str | localhost | The hostname or IP of the Oracle server. |
oracle_port | int | 1521 | The listener port for connection. |
oracle_username | str | oracle | Database user identity. |
oracle_service_name | str | XEPDB1 | The Oracle service name identifier. |
sid | str | None | None | Optional System Identifier (legacy Oracle ID). |
protocol | Literal | tcp | Connection protocol (tcp or tcps). |
Sources: src/flext_dbt_oracle/settings.py:31-60, rules.json:115-122
Validation Logic
The system enforces validation rules during the instantiation of settings. For example, it prevents invalid pool configurations and ensures that passwords provided as strings are converted into secure SecretStr types.
@u.model_validator(mode="after")
def validate_pool_sizes(self) -> FlextDbtOracleSettings:
"""Validate pool upper bound against minimum."""
if self.pool_max_size < self.pool_min_size:
msg = "Pool max size must be >= pool min size"
raise ValueError(msg)
return self
Sources: src/flext_dbt_oracle/settings.py:116-121, src/flext_dbt_oracle/models.py:105-109
Connection String Generation
Oracle requires specific formatting for connection strings depending on whether a Service Name or a SID is used. The lifecycle logic automatically determines the correct separator (/ for Service Name, : for SID) and constructs the Data Source Name (DSN).
The logic for this is implemented in both the settings and the core connection models to ensure consistency across different service layers.
Sources: src/flext_dbt_oracle/settings.py:145-155, src/flext_dbt_oracle/models.py:122-135, tests/unit/_config_parts/connection.py:16-43
Connection Pooling and Performance
To handle high-concurrency dbt workloads, the lifecycle includes managed connection pooling. This allows the adapter to reuse established sessions rather than incurring the overhead of a new TCP handshake and Oracle session creation for every query.
Performance Configuration
The lifecycle manages several performance-tuning parameters that influence how the Oracle driver interacts with the database.
| Feature | Field | Default | Purpose |
|---|---|---|---|
| Min Pool | pool_min_size | 1 | Minimum number of idle connections. |
| Max Pool | pool_max_size | 10 | Maximum allowed concurrent connections. |
| Timeout | query_timeout | 300 | Seconds before a long-running query is aborted. |
| Batch Size | fetch_size | 1000 | Records retrieved per network round-trip. |
Sources: src/flext_dbt_oracle/settings.py:84-114, rules.json:107-112
Diagnostic Lifecycle and Pipeline Integration
The final stage of the connection lifecycle is validation and health checking. The DbtOracle.Client provides a test_connection method to verify that the configured parameters result in a successful handshake with the Oracle instance.
Diagnostic testing returns a payload confirming the host and database identity, ensuring the lifecycle has successfully reached a "ready" state for SQL execution.
Sources: src/flext_dbt_oracle/utilities.py:65-71, src/flext_dbt_oracle/base.py:34-47
Implementation in Services
The connection lifecycle is consumed by FlextDbtOracleServiceBase, which generates a dbt-compatible connection profile. This profile translates the internal FlextDbtOracleSettings into the JSON structure expected by dbt core for Oracle-backed workflows.
Sources: src/flext_dbt_oracle/base.py:34-47, rules.json:113-124
Summary
The Database Connection Lifecycle in flext-dbt-oracle provides a structured path from configuration to execution. By centralizing validation in Pydantic models, automating DSN formatting for SID/Service Name variations, and providing built-in pooling and diagnostic checks, the system ensures stable and performant interactions with Oracle Database for dbt-driven data transformations.