Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• tests/memory_db_tests.rs
• tests/performance_tests.rs
• src/memory_db.rs
• README.md
• src/main.rs
• Cargo.toml

LogUtil employs a multi-tiered testing strategy designed to ensure the reliability, performance, and security of its real-time Nginx log analysis capabilities. The strategy focuses heavily on the MemoryDB component, which serves as the core data engine, using a combination of functional unit tests, large-scale performance benchmarks, and memory pressure simulations.

The testing architecture is divided into discrete suites that validate data integrity (insertion and indexing), analytical accuracy (top IP/URL extraction), and system stability under high-traffic scenarios. This ensures that the tool can handle the 10,000+ entries mentioned in the project overview while maintaining minimal CPU and memory overhead.

Sources: README.md, tests/performance_tests.rs:7-12

Functional Unit Testing

Functional tests focus on the correctness of the MemoryDB indexing system and its ability to retrieve and categorize log data accurately. These tests utilize a helper function, create_test_record, to generate consistent LogRecord instances for verification.

Core Validation Areas

• Data Retrieval: Tests verify that records can be found by specific fields such as IP address and URL.
• Statistical Accuracy: Ensures that get_stats() correctly reports total records, unique IPs, and unique URLs.
• Ranking Logic: Validates that get_top_ips and get_top_urls correctly sort and truncate results based on request frequency.
• Security Heuristics: Verifies the detection of attack patterns (e.g., "admin", "union select") and the identification of suspicious IPs.

Sources: tests/memory_db_tests.rs:4-106

The flow shows how functional tests move from record creation to specific analytical assertions.
Sources: tests/memory_db_tests.rs:24-40, tests/memory_db_tests.rs:136-160

Performance and Scalability Testing

Performance tests are designed to measure the system's behavior when processing large datasets, ranging from 10,000 to 10,000,000 records. These tests measure execution time for insertions, search operations, and statistical calculations.

Test Case	Dataset Size	Metrics Tracked
Large Dataset Stress	1,000,000 records	Insertion time, Search latency, Top N retrieval time
Memory Usage Optimization	10k to 10M records	VmRSS (Linux) or RSS (macOS), Records per second
Cache Performance	50,000 records	First call vs. Cached call latency, Speedup factor

Sources: tests/performance_tests.rs:7-130

Caching Efficiency

Testing includes specific benchmarks for get_top_ips and get_top_urls to verify that the internal caching mechanism provides significant speedups (measured in 'x' factor) for repeated analytical queries.

Sources: tests/performance_tests.rs:175-212

Resource Management and Memory Pressure

LogUtil implements automatic cleanup and eviction policies to prevent memory exhaustion. Testing validates these policies by setting artificial record limits and monitoring the system's response.

Eviction Logic Verification

Tests use set_max_records to establish a ceiling (e.g., 1,000,000 entries) and then flood the database to ensure the evict_old_records logic triggers correctly, maintaining the record count within bounds.

Sources: tests/performance_tests.rs:142-171, src/memory_db.rs:88-109

This diagram illustrates the interaction between the test suite and the database's internal eviction cycle.
Sources: tests/performance_tests.rs:150-171, src/memory_db.rs:112-132

Specialized Feature Testing

Beyond standard CRUD operations, the strategy covers specialized analytical modules:

• Security Analysis: Tests detect SQL injection, XSS, and Path Traversal patterns by searching log lines for specific substrings like admin, union select, or ../.
• Response Time Analysis: Verifies the calculation of average, minimum, and maximum response times, as well as the identification of "slow requests" exceeding a 1.0s threshold.
• User Agent Tracking: Benchmarks the efficiency of counting and ranking unique user agent strings.

Sources: src/tabs/security.rs:250-300, tests/performance_tests.rs:215-245, src/memory_db.rs:288-305

Testing Infrastructure

The project uses standard Rust testing tools and custom environment monitoring:

• Cargo Test: Primary execution engine for unit and performance tests.
• System Integration: Custom logic in get_memory_usage reads /proc/self/status on Linux and executes ps on macOS to capture real-time memory footprints during tests.
• Simulated Loads: Performance tests generate synthetic IPs and URLs (e.g., 192.168.x.x and /api/v1/resource/x) to simulate realistic traffic patterns.

Sources: Cargo.toml:68-71, tests/performance_tests.rs:250-280, tests/performance_tests.rs:15-25

LogUtil's testing strategy ensures that the application remains stable as a real-time monitor by strictly validating the MemoryDB's performance constraints and the accuracy of its security and performance analytics across varying scales of log data.