Type System and Protocols
Relevant source files
The following files were used as context for generating this wiki page:
The Type System and Protocols within flext-tap-oracle define the structural and behavioral contracts necessary for extracting data from Oracle databases following the Singer specification. This system relies on a Method Resolution Order (MRO) facade that composes types and protocols from flext-core, flext-db-oracle, and flext-meltano to ensure consistency across the FLEXT ecosystem.
The primary purpose of this architecture is to provide a strictly typed domain model for Oracle-specific data extraction, discovery, and performance monitoring. By utilizing Python's Protocol and runtime_checkable features, the system enforces static analysis and runtime validation for critical components like database connections and Singer stream metadata.
Sources: src/flext_tap_oracle/typings.py:1-20, src/flext_tap_oracle/models.py:18-35
Domain Type System
The type system is organized into a hierarchical namespace that distinguishes between core framework types and Oracle-specific domain types. The FlextTapOracleTypes class serves as the primary entry point, inheriting from both flext_meltano and flext_db_oracle type definitions.
Type Composition Hierarchy
The following diagram illustrates how Oracle-specific types are composed from parent FLEXT libraries.
Core Oracle Types
The system defines several domain-specific types used for data mapping and system summaries.
| Type Name | Base Type | Description |
|---|---|---|
SummaryData | t.JsonMapping | Standardized format for system and execution summaries. |
OracleValue | t.JsonValue | None | A union type representing any valid value returned from an Oracle query. |
TestOraclePort | Literal[1521, 10521, 1522] | restricted ports for test environments. |
MockOracleRecord | t.MappingKV[str, Scalar] | Key-value mapping representing a single Oracle row in tests. |
Sources: src/flext_tap_oracle/typings.py:19-21, tests/typings.py:38-40
Protocol Specifications
Protocols define the required interfaces for service interactions without enforcing strict class inheritance. This project leverages TestsFlextTapOracleProtocols to combine generic test protocols with domain-specific Oracle requirements.
Test Protocols and Mocking
To facilitate robust testing, the project defines protocols for mock database interactions and data providers.
Sources: tests/protocols.py:40-100
Functional Interface: MockOracleConnection
This protocol ensures that mock database clients provide the necessary methods for query execution during unit and integration tests.
connect(): Establishes a mock connection.disconnect(): Tears down the mock connection.execute_query(query, parameters): Simulates SQL execution, returning a sequence of JSON mappings.
Sources: tests/protocols.py:47-62
Data Model Schemas and Validation
The type system is further reinforced by Pydantic models and a JSON rule schema (rules.json). These define the constraints for connection parameters and data type mappings between Oracle and Singer.
Oracle to Singer Data Mapping
The system maps internal Oracle types to Singer-compliant JSON types to ensure compatibility with destination targets.
| Oracle Type | Singer/JSON Type | Conversion Rule |
|---|---|---|
VARCHAR2, CLOB, RAW | string | UTF-8 Encoded string |
NUMBER, BINARY_FLOAT | number | Precision preserved |
DATE, TIMESTAMP | string | RFC3339 / ISO format |
JSON | t.JsonValue | Native JSON parsing |
Sources: src/flext_tap_oracle/rules.json:208-235
Validation Rules
Validation is handled via flext_core integration using Pydantic V2. The system enforces specific constraints on Oracle connection parameters:
- Service Selection: At least one of
service_nameorsidmust be provided. - Port Range: Must be an integer between 1 and 65535.
- Batch Size: Minimum of 1, Maximum of 50,000.
- Parallel Streams: Must not exceed the
connection_pool_size.
Sources: src/flext_tap_oracle/rules.json:76-130
Execution and Performance Models
The FlextTapOracleModels class defines entities used for tracking the state and performance of the tap during operation.
Metrics and Statistics Model
The OracleTapExecutionStats model inherits from a _MetricsBase and provides detailed telemetry for extraction jobs.
Key statistical fields include:
oracle_connection_time: Latency observed during initial handshake.oracle_query_time: Total duration spent executing SQL statements.oracle_result_processing_time: Time spent converting Oracle C-types to Python/Singer types.success_rate: Computed field based onstreams_processedvsfailed_streams.
Sources: src/flext_tap_oracle/models.py:133-165
Summary
The Type System and Protocols in flext-tap-oracle provide a unified framework for Oracle data integration. By combining strict Pydantic models with flexible Python Protocols and a comprehensive JSON-based rule engine, the project ensures that Oracle database nuances are correctly translated into the Singer ecosystem while maintaining high performance and type safety.