Benchmarking

Relevant source files

The following files were used as context for generating this wiki page:

• tests/unit/test_benchmarks.py
• examples/README.md
• tests/examples/test_examples.py
• pyproject.toml
• tests/unit/test_quality.py
• tests/unit/test_features.py

Introduction

Benchmarking within the flext-target-oracle-wms project focuses on measuring the performance, scalability, and efficiency of Singer protocol data processing into Oracle WMS Cloud. The system utilizes unit tests to track component-level execution time and provides documented performance metrics for batch processing, memory consumption, and concurrent stream handling. These benchmarks ensure that the target can handle high-volume enterprise data loads while remaining within defined performance thresholds.

Sources: tests/unit/test_benchmarks.py:1-12, examples/README.md:184-187

Performance Metrics and Thresholds

The benchmarking suite utilizes iterative execution to measure the time elapsed for critical operations. Components are expected to complete operations within a specific PERF_THRESHOLD_SEC across a defined number of PERF_ITERATIONS.

Performance Indicators

Metric Type	Measurement	Typical Range
Single Record	Processing speed	~100-500 records/second
Batch Mode (1K)	Throughput	~2,000-5,000 records/second
Batch Mode (5K)	Throughput	~8,000-15,000 records/second
Concurrent Streams	Total throughput	~20,000+ records/second
Memory (Basic)	Baseline usage	~50-100 MB
Memory (Batch)	Batch processing usage	~200-500 MB

Sources: examples/README.md:189-204, tests/unit/test_benchmarks.py:41-45

Component-Level Benchmarks

The project benchmarks individual internal components to identify bottlenecks in the data transformation and management pipeline.

Data Transformation Performance

The WMSTypeConverter and WMSDataTransformer are tested for their ability to rapidly convert Singer types to Oracle formats and transform record keys into the required uppercase format.

The logic involves iterating through thousands of records to ensure transform_record and convert_singer_to_oracle operate below the performance threshold.

Sources: tests/unit/test_benchmarks.py:41-55, tests/unit/test_benchmarks.py:104-112, tests/unit/test_features.py:64-83

Registry and Lookup Performance

The WMSTableManager and CatalogManager are benchmarked for their efficiency in stream registration and table name lookups. High-performance lookup is critical as these operations occur for every batch and record processed.

Sources: tests/unit/test_benchmarks.py:57-64, tests/unit/test_benchmarks.py:75-82

Batch and Concurrent Processing

The target is optimized for high-performance batch operations. Benchmarking includes assessing the impact of different batch sizes and the use of concurrent streams.

Scalability Flow

The benchmarked flow demonstrates that increasing batch sizes (e.g., from 1K to 5K) significantly improves record-per-second throughput, while parallel processing of successful streams allows the system to exceed 20,000 records/second.

Sources: examples/README.md:104-120, examples/README.md:189-196

Tooling and Automation

Benchmarking is integrated into the development workflow through standard testing tools and custom measurement logic.

• Execution Tools: The project uses pytest and poetry for running benchmark tests. Although pytest-benchmark is noted as a preferred tool, the current suite uses time.time() for manual measurement.
• Dependency Management: Benchmark-related dependencies like pytest-benchmark and pytest-cov are managed via pyproject.toml.
• Quality Gates: Performance and benchmark tests are marked as a specific category (performance) within the pytest configuration to allow for isolated execution.

Sources: pyproject.toml:96-100, pyproject.toml:343-345, tests/unit/test_benchmarks.py:1-12

Summary

Benchmarking in flext-target-oracle-wms serves as a critical quality gate, ensuring component efficiency and system-wide scalability. By measuring record transformation speeds and throughput across various batch configurations, the project maintains a high-performance baseline suitable for mission-critical enterprise integrations.

Sources: examples/README.md:237-240, tests/unit/test_benchmarks.py:14-16