Data Models & Typings
Relevant source files
The following files were used as context for generating this wiki page:
The Data Models & Typings module within flext-tap-oracle-oic provides a standardized, Pydantic-driven framework for modeling Oracle Integration Cloud (OIC) resources, API responses, and stream extraction logic. It leverages the flext-core and flext-meltano base classes to implement Singer-compliant data structures and complex OIC domain entities.
This system ensures type safety and rigorous validation across the project, covering everything from core entity metadata (Integrations, Connections) to performance metrics and standardized API envelope parsing.
Architecture and Core Models
The project follows a hierarchical MRO (Method Resolution Order) pattern where FlextTapOracleOicModels extends both legacy flext_oracle_oic models and FlextMeltanoModels. This structure centralizes domain logic, validation helpers, and stream base classes.
Sources: src/flext_tap_oracle_oic/models.py:53-56, src/flext_tap_oracle_oic/_models/init.py:11-57
Hierarchy and Namespace
The architecture uses a nested namespace pattern to organize models by functional domain:
- TapOracleOic: Contains validation utilities and the
OICBaseStreamfor Singer extraction. - OracleOic: Contains pure domain entities migrated from
flext-oracle-oic, such asOICConnectionandOICIntegration.
The diagram represents the relationship between core Meltano/Singer models and the specific OIC implementations. Sources: src/flext_tap_oracle_oic/models.py:53-65
OIC Stream Modeling
The OICBaseStream class serves as the professional foundation for all OIC API interactions. It handles intelligent endpoint discovery, URL construction for various OIC API categories (Design, Runtime, Monitoring, etc.), and standardized pagination.
Base Stream Components
Key functionalities included in OICBaseStream:
- Endpoint Discovery: Automatically differentiates between
design.integrationandruntime.integrationendpoints based on OCI regions. - Response Parsing: Uses a standardized
OicEnvelopeto extract records from OIC-specific JSON structures (e.g.,itemsordataarrays). - Metric Tracking: Hooks for monitoring response status codes and record counts.
Sources: src/flext_tap_oracle_oic/models.py:100-145, src/flext_tap_oracle_oic/models.py:214-250
Standardized Stream Definitions
The FlextTapOracleOicModelsStreams class contains inner classes for each specific OIC resource. Each stream defines its own Singer schema using SingerPropertiesList.
| Stream Name | Primary Key | Replication Key | API Category |
|---|---|---|---|
| integrations | id | lastUpdated | core (Design API) |
| connections | id | lastUpdated | core (Design API) |
| packages | id | lastUpdated | core |
| lookups | name | lastUpdated | core |
| metrics | metricId, timestamp | timestamp | monitoring |
Sources: src/flext_tap_oracle_oic/_models/streams.py:34-451
Entity Models and Health Monitoring
Entity models like OicIntegrationEntity and OicMetricsRecord extend FlextMeltanoModels.Entity. These models include computed fields to generate real-time health and performance summaries.
OicIntegrationEntity
Represents an integration with complete metadata, including temporal information and execution statistics. It includes a integration_health_summary computed field that calculates error rates.
The flow shows how raw data is transformed into a validated entity with a calculated health status. Sources: src/flext_tap_oracle_oic/_models/_integration.py:19-125
Performance Metrics Modeling
The OicMetricsRecord model captures resource utilization and business metrics.
| Field | Type | Description |
|---|---|---|
metric_id | str | Unique metrics record identifier |
cpu_usage_percent | float | CPU usage (validated 0-100) |
throughput_mps | float | Messages per second |
latency_ms | float | Average latency in milliseconds |
Sources: src/flext_tap_oracle_oic/_models/_metrics.py:19-90
API Response and Error Handling
All API interactions are wrapped or parsed using the OicApiResponse model and OicEnvelope.
OIC API Envelope
The OicEnvelope (referenced as _OicEnvelope in the base stream) is used to detect the standard OIC pagination metadata:
totalSize: Total records available.count: Records in the current page.hasMore: Boolean indicating further pages.items/data: The actual record payload.
Sources: src/flext_tap_oracle_oic/models.py:326-340, src/flext_tap_oracle_oic/_models/_api_response.py:20-65
Error Context and Classification
The OicApiResponse model categorizes errors using error_code, error_message, and error_details. The system maps HTTP status codes to specific exceptions (e.g., 401 to AuthenticationError, 429 to RateLimitError).
Sources: src/flext_tap_oracle_oic/models.py:342-369, src/flext_tap_oracle_oic/_models/_api_response.py:67-100
Implementation Snippet: Base Stream Parsing
The following snippet illustrates how the OICBaseStream handles response normalization and record enrichment:
# From src/flext_tap_oracle_oic/models.py:269-278
def _enrich_record(
self,
record: t.JsonMapping,
) -> t.JsonMapping:
"""Enrich record with tap metadata for traceability."""
enriched = t.json_dict_adapter().validate_python(record)
enriched["_tap_extracted_at"] = u.generate_datetime_utc().isoformat()
enriched["_tap_stream_name"] = self.name
return enriched
Sources: src/flext_tap_oracle_oic/models.py:269-278
Summary
The Data Models & Typings module provides a robust, typed abstraction layer over the Oracle Integration Cloud APIs. By combining strict Pydantic validation with Singer-compliant stream definitions, it enables the tap to handle complex OIC scenarios including multi-region endpoint discovery, adaptive pagination, and automated health monitoring of integration instances. Significance lies in its ability to transform inconsistent API responses into structured, traceable data for downstream consumers.