Introduction
Relevant source files
The following files were used as context for generating this wiki page:
The flext-tap-oracle-oic project is a Singer Tap specifically designed for extracting data from Oracle Integration Cloud (OIC). It facilitates enterprise-grade ETL processes by providing a typed, robust interface to OIC's various API surfaces, including Design, Runtime, Monitoring, and B2B components. The tap is built upon the flext-core and flext-meltano frameworks, ensuring standardized patterns for authentication, pagination, and stream management.
Sources: README.md:1-12, src/flext_tap_oracle_oic/models.py:116-126, src/flext_tap_oracle_oic/tap.py:126-130
Core Architecture
The tap architecture is centered around the FlextTapOracleOic facade class, which orchestrates the discovery of streams, management of API clients, and execution of Singer protocol commands. It leverages a specialized FlextTapOracleOicClient for authenticated communication and a comprehensive set of models for data validation and schema definition.
High-Level Component Interaction
The following diagram illustrates the relationship between the main components of the tap and the external OIC API.
The tap uses an OAuth2 client credentials flow to obtain access tokens from the Oracle Identity Cloud Service (IDCS), which are then used by the API client for all subsequent resource requests.
Sources: src/flext_tap_oracle_oic/tap.py:32-60, src/flext_tap_oracle_oic/tap.py:126-160
Stream Management
The tap categorizes data into different stream types based on the OIC API endpoint categories. These are primarily divided into core (design-time), infrastructure, and monitoring streams.
Stream Categories
| Category | Streams Included | Description |
|---|---|---|
| Core | integrations, connections, packages, lookups, libraries | Design-time metadata for OIC resources. |
| Infrastructure | certificates, adapters | Security and connectivity configuration data. |
| Extended | projects | Resource grouping and folder structure. |
| Monitoring | executions, metrics | Runtime performance and status information. |
Sources: src/flext_tap_oracle_oic/constants.py:58-65, src/flext_tap_oracle_oic/_models/streams.py:816-827
Stream Implementation Logic
Every stream inherits from OICBaseStream, which provides standardized logic for:
- Adaptive Pagination: Adjusting page sizes based on response times to optimize throughput.
- Endpoint Discovery: Constructing regional URLs for Design vs. Runtime APIs.
- Incremental Extraction: Using replication keys like
lastUpdatedorstartTime.
Sources: src/flext_tap_oracle_oic/models.py:116-150, src/flext_tap_oracle_oic/_models/streams.py:31-40
Data Extraction Flow
The extraction process follows the standard Singer Tap lifecycle, beginning with configuration validation and discovery, followed by record extraction using the configured replication method.
Records are enriched with tap-specific metadata, such as _tap_extracted_at and _tap_stream_name, to ensure traceability in the target destination.
Sources: src/flext_tap_oracle_oic/models.py:228-255, src/flext_tap_oracle_oic/tap.py:207-250
Configuration and Settings
The tap is configured primarily through environment variables or a configuration JSON, managed by FlextTapOracleOicSettings.
Required Parameters
| Parameter | Env Variable | Description |
|---|---|---|
oauth_client_id | FLEXT_TAP_ORACLE_OIC_OAUTH_CLIENT_ID | OAuth2 client identifier. |
oauth_client_secret | FLEXT_TAP_ORACLE_OIC_OAUTH_CLIENT_SECRET | OAuth2 client secret (SecretStr). |
oauth_token_url | FLEXT_TAP_ORACLE_OIC_OAUTH_TOKEN_URL | OIC/IDCS token endpoint. |
oic_url | FLEXT_TAP_ORACLE_OIC_BASE_URL | Base URL of the OIC instance. |
Sources: src/flext_tap_oracle_oic/settings.py:31-50, src/flext_tap_oracle_oic/tap.py:321-335
Conclusion
The flext-tap-oracle-oic provides a highly structured and scalable way to sync Oracle Integration Cloud data. By strictly adhering to Singer and FLEXT standards, it offers reliable connectivity, comprehensive metadata extraction, and sophisticated handling of OIC's complex API ecosystem.