Embeddings & Reranking

Relevant source files

The following files were used as context for generating this wiki page:

• core/src/features/embeddings/rac_embeddings_service.cpp
• core/src/features/rerank/rac_rerank_service.cpp
• idl/embeddings_options.proto
• idl/rerank.proto
• core/tests/test_rerank.cpp
• core/AGENTS.md

Embeddings and Reranking are core AI primitives within the RunAnywhere commons library that enable semantic search and information retrieval. The Embeddings service transforms text into high-dimensional vectors, while the Reranking service refines search results by scoring the relevance of documents relative to a specific query. These features are implemented as single-backend capabilities within the C++ core, following the Service and Component architectural split.

Sources: core/README.md, core/AGENTS.md:73-85

Architecture Overview

The Embeddings and Reranking systems utilize the RunAnywhere plugin registry to dispatch requests to specific hardware-accelerated backends. The architecture separates the public API surface from the backend-specific implementation through a two-layer pattern consisting of a Service layer for dispatch and a Component layer for lifecycle management.

Data Flow and Dispatch

1. Service Layer: Dispatches calls to the appropriate backend. It looks up the model in the registry and calls rac_plugin_find() to retrieve the engine vtable.
2. Plugin Registry: Performs an ABI-versioned handshake. It routes requests to backends like llamacpp or onnx based on priority.
3. Backend Implementation: Executes the actual inference. For example, llamacpp serves both embeddings and reranking, while onnx serves embeddings.

Embeddings Service

The Embeddings service converts text strings into numerical vectors. These vectors represent the semantic meaning of the text, allowing for mathematical comparison of different inputs.

Embedding Options

The service supports various normalization methods to ensure the resulting vectors are suitable for similarity calculations (e.g., Cosine Similarity).

Option	Type	Description
normalize	EmbeddingNormalization	Specifies the normalization algorithm (NONE, L2).
dimensions	int32	Optional target dimension for the output vector.

Sources: idl/embeddings_options.proto:5-15

Implementation Details

The rac_embeddings_service.cpp file manages the lifecycle of the embedding service. It instantiates the backend through the engine's create operation and wraps it in a rac_embeddings_service_t structure.

• Create: rac_embeddings_service_create initializes the service by finding the highest-priority plugin for RAC_PRIMITIVE_EMBED.
• Embed: rac_embeddings_service_embed executes the vector generation against the selected backend.

Sources: core/src/features/embeddings/rac_embeddings_service.cpp:15-35, core/AGENTS.md:108

Reranking Service

Reranking is a cross-encoder primitive (wire value 11) that scores a list of documents based on their relevance to a query. Unlike embeddings, which compare vectors, reranking models analyze the query and document pairs directly for higher accuracy.

Reranking Process

The Reranking component typically follows an initial retrieval step. It takes a query and a list of "candidate" documents, returning a sorted list of scores.

Data Structures (IDL)

The Reranking service uses Protobuf messages for requests and results.

Message	Field	Type	Description
RerankRequest	query	string	The search query.
RerankRequest	documents	repeated string	The list of documents to score.
RerankResult	items	repeated RerankScoredItem	The scored and ranked results.
RerankScoredItem	relevance_score	float	The relevance score of the document.
RerankScoredItem	index	uint32	The original index of the document in the request.

Sources: idl/rerank.proto:11-30

Key Functions

• rac_rerank_service_create: Finds the backend and allocates the service implementation.
• rac_rerank_service_rerank: Proxies the rerank request to the engine's vtable.
• rac_rerank_component_load_model: Pins the service to a specific model path via the LifecycleManager.

Sources: core/src/features/rerank/rac_rerank_service.cpp:18-40, core/tests/test_rerank.cpp:132-135

Backend Support

Embeddings and Reranking are supported by different engines depending on the platform and model format.

Primitive	Engine	Model Format
RAC_PRIMITIVE_EMBED	llamacpp	GGUF
RAC_PRIMITIVE_EMBED	onnx	ONNX
RAC_PRIMITIVE_RERANK	llamacpp	Rank-pooling GGUF

Sources: core/AGENTS.md:108-113, core/README.md

Summary

The Embeddings & Reranking modules provide the necessary tools for building sophisticated on-device search capabilities. The Embeddings service enables efficient vector-based retrieval, while the Reranking service provides a precision layer to ensure the most relevant information is prioritized. By leveraging a plugin-based architecture, RunAnywhere ensures these operations are performed with optimal hardware acceleration across all supported platforms.