Metrics & Telemetry
Relevant source files
The following files were used as context for generating this wiki page:
Metrics and Telemetry within the FLEXT Tap Oracle OIC project provide critical observability into Oracle Integration Cloud (OIC) performance, health, and resource utilization. This system is designed to extract, process, and analyze high-fidelity data regarding integration executions, system throughput, and resource consumption to ensure operational stability and data integrity.
The telemetry framework is built upon Pydantic-based models that validate and structure performance data retrieved via the OIC Monitoring API. It supports real-time monitoring of CPU usage, memory consumption, latency, and business-centric metrics such as success and error rates.
Sources: src/flext_tap_oracle_oic/_models/_metrics.py:12-16, src/flext_tap_oracle_oic/models.py:100-112
Core Metrics Data Model
The OicMetricsRecord serves as the primary entity for performance monitoring. It captures both infrastructure-level telemetry and business-level execution statistics.
Performance and Resource Metrics
The system tracks several key performance indicators (KPIs) to monitor the health of the OIC environment:
- CPU & Memory: Percentage-based CPU utilization and Megabyte-based memory usage.
- Throughput & Latency: Messages per second (MPS) and average latency in milliseconds.
- Resource Counters: Active database connections, thread counts, and message queue depths.
Business Metrics
In addition to resource usage, the model tracks integration-specific outcomes:
- Success/Error Counts: Tracking the number of successful and failed messages.
- Retry Count: Monitoring the frequency of retry attempts for specific integrations.
The class diagram above illustrates the relationship between metrics records and integration entities, showing the specific telemetry fields captured for each.
Sources: src/flext_tap_oracle_oic/_models/_metrics.py:22-132, src/flext_tap_oracle_oic/_models/_integration.py:18-125
Metrics Analysis Logic
The project includes built-in logic to transform raw telemetry into sectioned summaries for easier consumption. This analysis calculates derived values such as the Error Rate.
| Category | Metric | Description |
|---|---|---|
| Identity | Metric ID | Unique identifier for the telemetry snapshot. |
| Identity | Integration ID | Reference to the associated OIC integration. |
| Performance | Throughput | Calculated as messages per second. |
| Business | Error Rate | Ratio of error_count to total messages (success + error). |
| Resource | Queue Depth | Number of messages currently waiting in the OIC queue. |
Sources: src/flext_tap_oracle_oic/_models/_metrics.py:98-132
Summary Generation Flow
The metrics_analysis_summary property aggregates data into distinct logical blocks: performance, business_metrics, and resource_utilization.
This flowchart describes how raw telemetry data is validated and structured into a multi-faceted analysis summary.
Sources: src/flext_tap_oracle_oic/_models/_metrics.py:98-154
Extraction and Monitoring Streams
Telemetry is extracted through specialized Singer streams that interface with the OIC Monitoring API.
Metrics Stream
The MetricsStream is responsible for extracting time-series performance data.
- Path:
/monitoring/v1/metrics - Primary Keys:
metricId,timestamp - Replication: Incremental based on the
timestampkey.
Executions Stream
The ExecutionsStream captures telemetry related to individual integration runs.
- Path:
/monitoring/v1/integrations - Replication: Incremental based on
startTime. - Data Points: Duration (ms), payload size, and processed record counts.
Sources: src/flext_tap_oracle_oic/_models/streams.py:649-742
Adaptive Telemetry Collection
The system employs an adaptive pagination strategy within the FlextTapOracleOicPaginator to optimize data retrieval based on network telemetry. It tracks response times and adjusts page sizes dynamically:
- Slow Responses: If average response time exceeds 5.0 seconds, the page size is reduced by 20%.
- Fast Responses: If response time is under 1.0 second, the page size may increase by 20% (up to a limit of 1000).
Sources: src/flext_tap_oracle_oic/tap_streams.py:84-100, src/flext_tap_oracle_oic/constants.py:73-75
The sequence diagram shows the feedback loop where API latency (telemetry) influences the pagination behavior (operation) of the metrics extraction process.
Sources: src/flext_tap_oracle_oic/tap_streams.py:33-100
Conclusion
Metrics & Telemetry in the FLEXT Tap Oracle OIC are foundational for maintaining the health of integration pipelines. By capturing both system-level resource utilization and business-level execution outcomes, the tap provides a comprehensive view of the integration ecosystem's performance. The implementation of adaptive pagination and structured analysis ensures that telemetry collection is both efficient and informative.