Core API Reference
Relevant source files
The following files were used as context for generating this wiki page:
The Core API of flext-tap-oracle provides a modern, standardized interface for extracting data from Oracle databases using the Singer protocol. It leverages the flext-core and flext-db-oracle frameworks to provide high-performance stream management, automated schema discovery, and robust connection handling.
The API is structured around several primary namespaces including FlextTapOracleModels for domain entities, FlextTapOracleStreams for data extraction logic, and FlextTapOracleSettings for runtime configuration. These components work together to ensure Singer compliance (v1.5.0) while optimizing Oracle-specific operations like connection pooling and type mapping.
Sources: README.md:9-12, src/flext_tap_oracle/models.py:23-35, src/flext_tap_oracle/rules.json:10-20
Architecture Overview
The system follows a layered architecture where the Tap Oracle API acts as a facade over database-specific operations and Singer-specific protocols.
This diagram illustrates the high-level flow from CLI entry points to the underlying Oracle database via the specialized stream and utility layers.
Sources: src/flext_tap_oracle/api.py, src/flext_tap_oracle/streams.py:19-28, src/flext_tap_oracle/_utilities/client.py:21-25
Stream Management API
The FlextTapOracleStreams class serves as the unified container for stream-related logic. It implements the OracleStream class, which handles row count estimation, record fetching, and data type transformation.
OracleStream Implementation
The OracleStream integrates directly with FlextDbOracleApi to perform optimized queries without raw SQLAlchemy calls.
- Row Estimation: Uses Oracle system views and
COUNT(*)queries with validated table names. - Record Extraction: Implements
get_recordswhich yields processed rows as JSON mappings. - Type Transformation: Maps Oracle-specific types (CLOB, BLOB, DATE, TIMESTAMP) to Singer-compatible formats using ISO8601 for dates and string representation for large objects.
Sources: src/flext_tap_oracle/streams.py:32-60, src/flext_tap_oracle/streams.py:84-110, src/flext_tap_oracle/streams.py:165-200
Stream Factory
The StreamFactory provides static methods to instantiate streams either manually or automatically from Oracle metadata.
| Method | Description | Parameters |
|---|---|---|
create_oracle_stream | Creates a stream with explicit schema. | tap, name, table_name, schema, oracle_api |
create_oracle_stream_from_table | Generates a stream automatically from metadata. | tap, table_metadata, oracle_api, stream_prefix |
Sources: src/flext_tap_oracle/streams.py:212-255
Client Utilities and Discovery
Client utilities handle the lifecycle of the tap, including connection testing and table discovery. These are implemented as a Mixin to provide reusable logic across service components.
Discovery Flow
The discovery process ensures that the connection is viable before attempting to introspect the schema.
Sources: src/flext_tap_oracle/_utilities/client.py:28-56, src/flext_tap_oracle/_utilities/client.py:86-118
Configuration and Rules
Configuration is managed via FlextTapOracleSettings, which utilizes Pydantic for validation and environment variable injection (prefix FLEXT_TAP_ORACLE_).
Required Connection Parameters
| Field | Type | Default | Description |
|---|---|---|---|
oracle_host | NonEmptyStr | localhost | Database hostname |
oracle_port | PortNumber | 1521 | Listener port |
oracle_service_name | str | XE | Service Name or SID |
oracle_user | SecretStr | N/A | Authentication username |
oracle_password | SecretStr | N/A | Authentication password |
Sources: src/flext_tap_oracle/settings.py:34-60, src/flext_tap_oracle/rules.json:57-70
Business Logic Validation
The API enforces specific business rules during configuration:
oracle_hostmust not be empty.oracle_service_namemust be provided.max_parallel_streamsmust not exceed theconnection_pool_size.
Sources: src/flext_tap_oracle/settings.py:65-72, src/flext_tap_oracle/rules.json:115-120, src/flext_tap_oracle/rules.json:344-350
Data Models
FlextTapOracleModels provides standardized entities for tracking tap execution and performance.
Execution Statistics
The OracleTapExecutionStats model tracks the health and performance of an extraction job.
| Metric Field | Type | Description |
|---|---|---|
total_records | NonNegativeInt | Total rows extracted |
oracle_query_time | NonNegativeFloat | Time spent executing SQL |
errors_encountered | NonNegativeInt | Count of failures during sync |
failed_streams | MutableSequence | List of streams that did not complete |
Performance and Error Handling
The API implements specific rules for performance optimization and resilience as defined in the rules.json configuration.
- Extraction Limits: Recommended
batch_sizeranges from 100 for small tables to 10,000 for large tables.max_parallel_streamsis capped at 4 by default for production environments. - Circuit Breaker: Enabled by default with a failure threshold of 5 and a timeout of 60 seconds.
- Retry Strategy:
- Connection errors: 3 attempts with exponential backoff.
- Query errors: 2 attempts with linear backoff.
Sources: src/flext_tap_oracle/rules.json:173-190, src/flext_tap_oracle/rules.json:237-255, src/flext_tap_oracle/rules.json:265-275
Conclusion
The flext-tap-oracle Core API provides a robust framework for Oracle data integration. By centralizing stream logic in FlextTapOracleStreams and using strictly validated models in FlextTapOracleModels, the system ensures high data quality and operational transparency. Developers can extend the tap by utilizing the StreamFactory for custom table mappings while relying on the built-in ClientMixin for standardized connection and discovery workflows.
Sources: src/flext_tap_oracle/api.py, src/flext_tap_oracle/models.py:23-35, src/flext_tap_oracle/streams.py:19-25