E2E & Sink Testing
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The E2E & Sink Testing framework for flext-target-oracle-oic provides a comprehensive validation suite for Singer target implementations and Oracle Integration Cloud (OIC) API interactions. The testing strategy encompasses two primary layers: specialized Sink testing, which focuses on the processing logic of individual data streams (Connections, Integrations, Packages, and Lookups), and End-to-End (E2E) testing, which validates the entire lifecycle of data loading, including authentication and real API communication.
Sources: tests/unit/test_e2e.py:1-20, tests/unit/test_e2e_sinks.py:1-12
This testing infrastructure ensures that the target can correctly interpret Singer messages, validate configurations, and securely communicate with OIC endpoints using OAuth2. Unlike standard unit tests, these functional tests are designed to operate without mocks in specific scenarios to verify real-world interoperability.
Sources: tests/unit/test_e2e.py:14-16, tests/unit/test_target.py:1-8
Sink Architecture and Processing
The project utilizes specialized sink classes derived from a common base to handle specific OIC entities. Each sink is responsible for processing records and managing batch operations for its respective stream.
Sink Components
The following sink classes are defined to handle different OIC data types:
FlextTargetOracleOicConnectionsSink: Manages connection metadata and adapter types.FlextTargetOracleOicIntegrationsSink: Handles integration flows, including base64 encoded archive content.FlextTargetOracleOicPackagesSink: Processes OIC package bundles.FlextTargetOracleOicLookupsSink: Manages lookup tables and versioning.
Sources: src/flext_target_oracle_oic/target.py:35-59, tests/unit/test_e2e_sinks.py:22-27
Data Flow within Sinks
The processing flow involves initializing the sink with a Singer target instance, defining a JSON schema for validation, and invoking process_record for incoming data.
The diagram shows how incoming Singer messages are passed through the sink's processing logic before being sent to OIC.
Sources: src/flext_target_oracle_oic/target.py:18-32, tests/unit/test_e2e_sinks.py:38-51
E2E Testing Framework
The E2E suite validates the full integration path, from target initialization to resource teardown.
Configuration Loading
Tests utilize environment variables or a .env file to load real OIC credentials. This ensures that tests run against actual OIC environments when available, skipping execution if required keys are missing.
| Configuration Key | Description |
|---|---|
base_url | The OIC instance base URL. |
oauth_client_id | OAuth2 Client ID for authentication. |
oauth_client_secret | OAuth2 Client Secret. |
oauth_token_url | The URL used to request access tokens. |
oauth_client_aud | Optional audience for the OAuth2 request. |
Sources: tests/unit/test_e2e.py:34-56, src/flext_target_oracle_oic/utilities.py:84-100
Authentication Lifecycle
The TargetOracleOic.Authenticator class manages the OAuth2 lifecycle. It builds the token request payload and retrieves access tokens required for API headers.
The sequence diagram illustrates the automated authentication flow handled by the utility layer during E2E execution.
Sources: src/flext_target_oracle_oic/utilities.py:81-140, tests/unit/test_target.py:91-110
Target Initialization and Mapping
The FlextTargetOracleOic class acts as the central entry point, mapping stream names to their corresponding sink implementations.
Sink Mapping Table
| Stream Name | Sink Class |
|---|---|
connections | FlextTargetOracleOicConnectionsSink |
integrations | FlextTargetOracleOicIntegrationsSink |
packages | FlextTargetOracleOicPackagesSink |
lookups | FlextTargetOracleOicLookupsSink |
| Unknown | FlextTargetOracleOicBaseSink |
Sources: src/flext_target_oracle_oic/target.py:65-80, tests/unit/test_e2e.py:73-89
Service Runtime Creation
The FlextTargetOracleOicServiceRuntime provides a factory method for creating sinks dynamically within the service facade. This ensures that the target configuration is normalized to a standard JSON mapping before the sink is instantiated.
# src/flext_target_oracle_oic/_utilities/service_runtime.py:27-46
@classmethod
def create_sink(
cls,
*,
stream_name: str,
schema: t.JsonMapping,
target_config: t.JsonMapping,
) -> FlextTargetOracleOicBaseSink:
normalized_target_config = u.normalize_to_json_mapping(target_config)
runtime_target = FlextTargetOracleOic()
sink_class = runtime_target.fetch_sink_class(stream_name)
return sink_class(
target=cls.Target(
config=t.json_dict_adapter().validate_python(normalized_target_config),
validate_config=False,
),
stream_name=stream_name,
schema=t.json_dict_adapter().validate_python(schema),
key_properties=[],
)
Sources: src/flext_target_oracle_oic/_utilities/service_runtime.py:27-46, src/flext_target_oracle_oic/api.py:30-41
Conclusion
E2E & Sink Testing in this project ensures the reliability of data movement into Oracle Integration Cloud by strictly validating sink processing logic and authentication protocols. By combining granular sink tests with comprehensive lifecycle E2E tests, the system guarantees that data schemas are respected and OIC communication remains secure and robust across all supported stream types.