Performance Tuning
Relevant source files
The following files were used as context for generating this wiki page:
Performance tuning in EdgeVec involves optimizing the tradeoff between search speed, memory footprint, and recall accuracy within a resource-constrained WebAssembly (WASM) environment. The project establishes strict performance budgets, such as sub-10ms P99 search latency for 100k vectors, and provides various mechanisms—ranging from HNSW parameter adjustment to SIMD acceleration and binary quantization—to meet these targets.
Sources: GATE_1_COMPLETE.md:144-150, README.md:103-130
HNSW Configuration Parameters
The core of EdgeVec's performance tuning lies in configuring the Hierarchical Navigable Small World (HNSW) index. Key parameters influence the graph density and search depth, directly impacting search recall and latency.
Core Parameters
- M (Max Neighbors): Defines the maximum number of outgoing connections for each node in the graph layers. Recommended values typically range around 16. Higher values increase recall and graph connectivity but consume more memory and increase search/insertion time.
- ef_construction: Controls the size of the dynamic candidate list during index building. Higher values lead to a more accurate graph at the cost of significantly longer indexing time.
- ef_search: Controls the size of the dynamic candidate list during search. Increasing
ef_searchimproves recall but increases search latency.
Sources: CONTRIBUTING.md:73-83, wasm/examples/stress-test.html:565-568
| Parameter | Type | Default/Standard | Impact on Performance |
|---|---|---|---|
M | usize | 16 | Higher = Higher Recall, Higher Memory, Slower Search |
ef_construction | usize | 200 | Higher = Higher Recall, Slower Indexing |
ef_search | usize | (Dynamic) | Higher = Higher Recall, Slower Search |
Sources: wasm/examples/stress-test.html:566-567, CONTRIBUTING.md:78-83
HNSW Performance Flow
This diagram illustrates how parameters influence the search and insertion logic within the HNSW layers.
The diagram shows the relationship between configuration parameters and their specific performance outcomes.
Sources: GATE_1_COMPLETE.md:88-92, wasm/examples/stress-test.html:565-570
SIMD Acceleration (v0.7.0)
EdgeVec utilizes WASM SIMD128 instructions to accelerate vector operations like Dot Product, L2 Distance, and Hamming distance. SIMD provides a 2x+ speedup on supported browsers by processing 4 float32 values simultaneously.
Throughput and Browser Support
- SIMD Throughput: Reaches approximately 2.1 to 2.3 Gelem/s (Giga-elements per second) for Dot Product and L2 distance.
- Hamming Distance: Optimized SIMD implementation provides up to 8.75x speedup for Binary Quantization operations.
- Scalar Fallback: For browsers that do not support SIMD (such as iOS Safari), EdgeVec automatically falls back to scalar implementations, which are functional but ~2x slower.
Sources: README.md:106-125, wasm/examples/simd_benchmark.html:362-378
SIMD Lane Architecture
SIMD processes four 32-bit floats in a single instruction cycle.
Sources: wasm/examples/simd_benchmark.html:370-385
Memory Management and Quantization
Tuning memory usage is critical for browser-based deployments where heap limits (often around 1GB) apply.
Binary Quantization (BQ)
Binary Quantization provides a 32x memory reduction by representing vector dimensions as single bits.
- Raw BQ: ~85% recall, ~5x faster search.
- BQ + Rescore: Using BQ for initial candidate retrieval followed by a rescoring phase with full-precision vectors can achieve ~95% recall while remaining ~3x faster than full F32 search.
Sources: README.md:151-165, wasm/examples/v060_cyberpunk_demo.html:150-165
Scalar Quantization (SQ8)
SQ8 reduces memory by approximately 3.6x (e.g., reducing 3.03 GB to 832 MB for 1M vectors) by quantizing 32-bit floats into 8-bit integers.
Sources: README.md:213-217
Memory Pressure API
Developers can monitor WASM heap usage and trigger compaction when memory levels reach thresholds (e.g., "warning"). Compaction reclaims space by physically removing vectors marked with a "soft-delete" tombstone bit.
Sources: README.md:188-198, GATE_1_COMPLETE.md:104-106
Build Optimization
The Cargo.toml configuration is tuned for maximum performance and minimum bundle size through specific release profiles.
[profile.release]
opt-level = "z" # Optimize for size
lto = true # Link-time optimization
codegen-units = 1 # Maximum optimization
strip = true # Remove symbols
[profile.bench]
opt-level = 3 # Maximum speed for benchmarks
debug = true # Required for profiling
Sources: Cargo.toml:193-210
Performance Budgets
EdgeVec architecture is governed by specific targets verified through the benchmark suite.
| Operation | Target | Actual (Reported) | Status |
|---|---|---|---|
| Search (100k vectors) | < 10ms P99 | ~0.94ms (10k @ 768D) | ✅ PASS |
| Single Insertion | < 1ms mean | - | ✅ PASS |
| Index Load (100k) | < 500ms | - | ✅ PASS |
| Bundle Size (Gzipped) | < 500KB | 217KB | ✅ PASS |
Sources: GATE_1_COMPLETE.md:144-150, README.md:118-121, README.md:138-142
Summary
Performance tuning in EdgeVec is a multi-layered approach starting from high-level HNSW parameter adjustment (M, ef_search) down to hardware-level SIMD optimizations. Memory footprint can be tuned using Binary or Scalar Quantization, while the Memory Pressure API and Compaction system ensure the WASM heap remains efficient during long-running sessions. Accuracy and speed are verified against strict performance budgets to maintain "Nvidia Grade" software standards.
Sources: README.md:12-25, CONTRIBUTING.md:16-25, GATE_1_COMPLETE.md:124-128