Integration Tests Suite

Relevant source files

The following files were used as context for generating this wiki page:

• tests/aggressivehpctesting/aggressivetesting.md
• tests/aggressivehpctesting/phase_1_crud_and_indexing.rs
• tests/aggressivehpctesting/phase_2_search_and_query.rs
• tests/aggressivehpctesting/phase_3_soak_test.rs
• tests/client.rs
• README.md
• Cargo.toml

The Integration Tests Suite for the Milvus Rust SDK is designed to validate the stability, performance, and correctness of the SDK when interacting with a live Milvus server. The suite ranges from basic functional tests of client operations to "Aggressive HPC Testing" involving high concurrency and long-duration soak tests. These tests ensure the SDK is production-ready by simulating real-world workloads, including large data volumes and complex multi-threaded interactions.

The suite utilizes tokio for asynchronous execution and docker-compose to manage the underlying Milvus infrastructure during testing. It specifically targets high-performance computing (HPC) scenarios to identify bottlenecks such as gRPC timeouts, rate-limiting, and server-side stability under load.

Sources: tests/aggressivehpctesting/aggressivetesting.md:3-10, README.md:27-37, Cargo.toml:34-70

Testing Infrastructure and Execution

The test suite requires a running Milvus environment, typically orchestrated via Docker. The Cargo.toml file defines numerous integration test targets, including a dedicated suite for aggressive testing.

Test Execution Flow

The testing process involves setting up a Milvus cluster using docker-compose -f ./docker-compose.yml up -d and executing cargo test. Debugging is supported via RUST_BACKTRACE=1.
Sources: README.md:32-45, Cargo.toml:34-75

Functional Integration Tests

Basic integration tests focus on individual component reliability, covering collection management, aliases, and property manipulation.

Test Category	Key Operations Tested	Source File
Client	Connection, Wrong URL Handling, Timeouts	tests/client.rs
Collection	Create, Has, Drop, Rename, Alter Properties	tests/client.rs
Alias	Create, Alter, Drop Alias	tests/client.rs
Schema	Builder Pattern, Primary Key, Auto-ID	tests/client.rs

Functional tests verify that the Client correctly maps Rust structures to Milvus gRPC requests. For example, create_has_drop_collection ensures that a schema built with CollectionSchemaBuilder is correctly recognized by the server.
Sources: tests/client.rs:20-100, tests/client.rs:170-200

Aggressive HPC Testing Suite

The Aggressive HPC (High-Performance Computing) suite is a structured three-phase plan to stress the SDK.

Phase 1: High-Concurrency CRUD and Indexing

This phase utilizes tokio::spawn to initiate multiple task groups:

• Writers: 20 tasks inserting 10,000 entities each in batches of 1,000.
• Deleters: 5 tasks performing random deletions.
• Indexer: Tasks to create and re-create IVF_FLAT indexes during mutation operations.

Verification is performed by comparing the logical count (via count(*)) against the expected count (inserts minus deletes).
Sources: tests/aggressivehpctesting/aggressivetesting.md:15-35, tests/aggressivehpctesting/phase_1_crud_and_indexing.rs:38-110

Phase 2: High-Volume Search and Query

Phase 2 focuses on read-stability while the collection is under "churn" (active modification).

• Searchers: 50 concurrent tasks performing vector searches for 60 seconds.
• Queriers: 20 concurrent tasks executing complex filter expressions.
• Churn: 5 tasks continuing background inserts and deletes to ensure data volatility.

Phase 3: Soak Test

The soak test is designed to run for 60 minutes to identify memory leaks or performance degradation. It cycles through a full lifecycle for each iteration:

1. Create Collection & Partition.
2. Insert 1,000 entities and Flush.
3. Create Index and Load Collection.
4. Perform Search and Query.
5. Release Collection and Drop Index.
6. Delete entities and Drop Partition/Collection.

Sources: tests/aggressivehpctesting/aggressivetesting.md:54-70, tests/aggressivehpctesting/phase_3_soak_test.rs:20-50

Lessons from Aggressive Testing

Aggressive testing revealed several critical environmental and SDK requirements:

• Rate-Limiting: High-concurrency tests require custom Milvus server configurations to avoid RateLimit errors.
• gRPC Timeouts: Default timeouts are insufficient for high load; the ClientBuilder must implement timeout() for stability.
• Data Consistency: Testing identified that concurrent inserts to multiple unique partitions could lead to silent data loss in certain Milvus versions, leading the suite to prefer writing directly to the collection during high-concurrency phases.
• Cleanup Dependencies: The suite found that drop_collection might hang if the collection is not explicitly flushed beforehand.

Sources: tests/aggressivehpctesting/aggressivetesting.md:80-95, tests/aggressivehpctesting/phase_3_soak_test.rs:60-70

Conclusion

The Integration Tests Suite serves as the final gateway for ensuring the Milvus Rust SDK's reliability. By combining standard functional verification with aggressive, multi-threaded stress tests, the suite identifies edge cases in gRPC handling, concurrency bottlenecks, and long-term stability issues that unit tests might miss. The successful execution of over 290 cycles in the soak test provides high confidence for production deployment.

Sources: tests/aggressivehpctesting/aggressivetesting.md:105-112