Protocols & Interfaces
Relevant source files
The following files were used as context for generating this wiki page:
The Protocols and Interfaces within flext-target-oracle define the structural contracts and behavioral expectations for the Oracle-specific Singer target implementation. By utilizing Python's typing.Protocol and runtime checkable classes, the system ensures a clean separation of concerns between the core Singer message handling, data loading logic, and Oracle database interactions.
These interfaces facilitate a "Clean Architecture" approach, allowing for modular development where services such as connection management, schema validation, and batch processing can be implemented and tested independently while adhering to established MRO (Method Resolution Order) facade patterns.
Sources: src/flext_target_oracle/_protocols/base.py:1-20, CONTRIBUTING.md:120-135
Core Domain Protocols
The domain protocols represent the primary functional units of the Oracle target. These are organized within the FlextTargetOracleProtocolsBase namespace and are marked as @runtime_checkable to support structural subtyping at runtime.
Technical Service Interfaces
The following table summarizes the core service protocols defined for the Oracle target environment:
| Protocol Name | Purpose | Key Methods/Properties |
|---|---|---|
Target | Main interface for processing Singer records. | process_record() |
Connection | Manages Oracle database connectivity. | connect_target() |
Schema | Handles DDL operations based on Singer schemas. | create_table_from_schema() |
Batch | Orchestrates bulk or batch data operations. | execute_batch_target() |
Record | Handles transformations of Singer records for Oracle. | transform_record_target() |
Security | Validates credentials and access levels. | validate_target_credentials() |
Sources: src/flext_target_oracle/_protocols/base.py:38-100
Connection and Result Handling
Connectivity is governed by the ConnectionOperationResult and ConnectionService protocols. The system uses a Result-oriented approach (via the p.Result type) to ensure error handling is explicit rather than exception-based.
The ConnectionService contract specifically provides methods for retrieving effective connection info and validating active connectivity.
Sources: src/flext_target_oracle/_protocols/base.py:24-36, src/flext_target_oracle/_protocols/base.py:112-124
Service Layer Implementation
The utility services implement the protocols defined in the base layer, providing concrete logic for batching, schema verification, and record transformation.
Batch and Record Services
The FlextTargetOracleBatchService manages in-memory buffers for records before they are flushed to the Oracle database. Simultaneously, the FlextTargetOracleRecordService handles the "Railway-Oriented" transformation logic, such as filtering ignored columns or applying field mappings.
The transformation logic in FlextTargetOracleRecordService.transform_record iterates through record items, skipping keys defined in stream.ignored_columns and applying renames found in stream.column_mappings.
Sources: src/flext_target_oracle/_utilities/services.py:65-98, src/flext_target_oracle/_utilities/services.py:113-138
Schema Management
The SchemaService ensures that destination tables exist before any data loading occurs. The protocol defines ensure_table_exists, which is called during the processing of SCHEMA messages from the Singer stream.
Sources: src/flext_target_oracle/_protocols/base.py:126-135, src/flext_target_oracle/_utilities/services.py:53-62
Client Facade Interface
The FlextTargetOracle class serves as the public facade, coordinating between the various protocol implementations to satisfy the Singer target lifecycle.
Message Processing Flow
The client implements a dispatcher pattern (via the process_singer_message method) that routes incoming messages to specific handlers based on their type.
The FlextTargetOracle client maintains internal state for schemas (a mapping of stream names to schema messages) and the current state_message.
Sources: src/flext_target_oracle/_utilities/client.py:18-35, src/flext_target_oracle/_utilities/client.py:133-146
Implementation Metrics and Settings
Target behavior is configured through a FlextTargetOracleSettings object, which is validated against business rules via the OracleSettingsProtocol.
| Feature | Protocol/Method | Source File |
|---|---|---|
| Batch Size | ImplementationMetrics.batch_size | client.py |
| Bulk Ops | ImplementationMetrics.use_bulk_operations | client.py |
| Business Rules | validate_business_rules() | base.py |
| Config Validation | validate_configuration() | client.py |
Sources: src/flext_target_oracle/_protocols/base.py:167-173, src/flext_target_oracle/_utilities/client.py:118-123, src/flext_target_oracle/_utilities/client.py:185-188
Conclusion
The use of Protocols in flext-target-oracle provides a strict yet flexible framework for Oracle data loading. By defining clear interfaces for ConnectionService, SchemaService, BatchService, and RecordService, the architecture supports complex operations like bulk merging and flattened storage while maintaining testability and adherence to the FLEXT ecosystem's design principles.