Embedding Model Integrations
Relevant source files
The following files were used as context for generating this wiki page:
Embedding model integrations in EdgeVec provide the mechanism to bridge high-dimensional vector data produced by machine learning models with a high-performance, WebAssembly-based vector database. The system is designed to handle vectors typically generated by modern Transformer-based models, facilitating operations like similarity search, metadata filtering, and batch processing directly within client-side environments such as browsers or mobile applications.
The integration layer focuses on standardizing the input format—predominantly utilizing Float32Array or typed metadata—to ensure compatibility with various embedding providers. By offloading the compute-heavy index search and distance calculations to a Rust-backed WASM engine, EdgeVec maintains low latency (P50 < 0.2ms) even when integrated with large embedding dimensions ranging from 128 to 2048.
Sources: wasm/examples/index.html:1052-1056, wasm/examples/batch_insert.js:22-35, wasm/examples/simd_benchmark.html:1008-1015
Integration Architecture
The architecture of EdgeVec integration relies on a clear boundary between the JavaScript application layer (where embedding models reside) and the high-performance WASM core. Application code typically fetches or generates embeddings using external libraries (like Transformers.js or OpenAI API) and passes the resulting numeric arrays to the EdgeVec instance.
Data Flow Overview
The following diagram illustrates the flow from a raw input query to the retrieval of contextually relevant results using an embedding model integration.
The diagram shows how the application layer handles the model inference while EdgeVec manages the computational geometry and persistence.
Sources: wasm/examples/v070_filter_playground.html:268-285, wasm/examples/index.html:1037-1045
Supported Model Dimensions and Configurations
EdgeVec integrations are initialized with a specific dimensionality that must match the output of the chosen embedding model. The EdgeVecConfig class allows developers to tune the internal HNSW (Hierarchical Navigable Small World) parameters to optimize for the specific characteristics of the model's vector space.
Dimensionality Presets
While EdgeVec supports arbitrary dimensions up to 2048, standard integrations often target specific model outputs:
| Model Type Example | Typical Dimensions | Config Use Case |
|---|---|---|
| Fast/Mobile Embeddings | 128 - 384 | Mobile compatibility, low memory |
| Standard Transformers | 768 | Desktop RAG applications |
| High-Fidelity Models | 1024 - 1536 | Professional search, high precision |
| Large Encoders | 2048 | Maximum allowed capacity |
Sources: wasm/examples/batch_insert.js:28-35, wasm/examples/batch_insert.html:488-510, wasm/examples/simd_benchmark.html:625-635
Core Configuration Parameters
Integrations utilize the EdgeVecConfig constructor to define the environment:
// Initialization for a standard 768-dimension model
const config = new EdgeVecConfig(768);
const db = new EdgeVec(config);
Sources: wasm/examples/v070_filter_playground.html:273-274, tests/mobile/index.html:697-700
SIMD Acceleration for Vector Operations
For compute-intensive embedding integrations, EdgeVec utilizes WASM SIMD128 to accelerate distance metrics. This is particularly relevant for models with high dimensionality (e.g., 768 or 1536), where scalar operations can become a bottleneck.
The system automatically detects browser support for f32x4 registers. When available, it executes four floating-point multiplications or additions simultaneously, providing throughput often exceeding 2 Gelem/s (Giga-elements per second).
The diagram represents the parallel processing of vector lanes enabled by SIMD integration.
Sources: wasm/examples/simd_benchmark.html:15-30, wasm/examples/simd_benchmark.html:600-610, Cargo.toml:193-195
Metadata-Augmented Integrations
Integrated embedding models often require context beyond the raw vector. EdgeVec supports inserting vectors alongside JSON-compatible metadata, which allows for SQL-like filtering on the results of the model search.
Integration Workflow with Metadata
- Generate Embedding: Obtain a
Float32Arrayfrom the model. - Define Metadata: Create a JS object containing attributes (category, price, etc.).
- Insert with Context: Use
insertWithMetadatato bind the vector to the data.
const vector = model.embed("Product Description");
const metadata = { category: "electronics", price: 299.99 };
db.insertWithMetadata(vector, metadata);
Sources: wasm/examples/v070_filter_playground.html:278-283, wasm/examples/filter-playground.html:1255-1265
Filter Operators for Model Results
The integration supports over 15 operators to refine results retrieved by the embedding model:
| Operator Category | Examples |
|---|---|
| Comparison | =, !=, >, <, >=, <= |
| Logical | AND, OR, NOT |
| Range/Collection | BETWEEN, IN [...], ANY [...] |
| String Patterns | CONTAINS, STARTS_WITH, LIKE |
Sources: wasm/examples/v070_filter_playground.html:167-185, wasm/examples/filter-playground.html:1050-1080
Batch Processing and Performance
Integrations often involve bulk data ingestion. EdgeVec provides optimized insertBatch and insertBatchWithProgress methods to reduce the overhead of the JavaScript-to-WebAssembly boundary transition.
For an integration involving 1,000 vectors at 128 dimensions, batch operations can be significantly faster than sequential calls due to minimized context switching. The integration layer also includes error handling for common model output issues such as NaN values, Infinity, or dimension mismatches.
Sources: wasm/examples/batch_insert.js:335-360, wasm/examples/batch_insert.js:460-500
Summary
Embedding model integrations in EdgeVec facilitate the deployment of advanced AI search features directly in client-side environments. By combining flexible dimensionality support (128 to 2048) with SIMD-accelerated distance calculations and robust metadata filtering, the system ensures that high-quality embeddings produced by external models can be indexed and queried with minimal latency and high accuracy. The integration remains platform-agnostic, supporting standard Float32Array inputs from any machine learning inference engine.