Plugin Registry & Engine Routing

Relevant source files

The following files were used as context for generating this wiki page:

• engines/AGENTS.md
• core/AGENTS.md
• AGENTS.md
• core/README.md
• core/include/rac/plugin/rac_engine_vtable.h
• core/include/rac/plugin/rac_plugin_entry.h

The Plugin Registry and Engine Router form the core dispatch mechanism of the RunAnywhere SDK. This system allows the C++ commons library to discover, register, and route AI requests to specific inference backends (engines) such as llamacpp, sherpa, or qhexrt. It uses a stable vtable-based ABI to ensure that different engines can provide modular support for various AI primitives like text generation, speech-to-text, and image diffusion.

Sources: core/README.md, engines/AGENTS.md:16-23

Engine Architecture

An engine serves as an op-table adapter for specific AI modalities. Each engine implements a single rac_engine_vtable_t structure. This structure contains 10 active primitive slots and 7 reserved slots. If an engine does not support a specific primitive, it sets the corresponding slot to NULL.

The Valid Engine Contract

To be considered valid by the registry, an engine must provide:

• Manifest: A rac_engine_manifest_t containing the engine name, supported primitives, hardware runtimes, and priority.
• Vtable: A populated rac_engine_vtable_t where supported primitive slots are non-NULL.
• Capability Check: A function to determine if the engine can run on the current hardware/OS.
• Registration: A defined entry point using RAC_PLUGIN_ENTRY_DEF(<name>).

Sources: engines/AGENTS.md:16-23, engines/AGENTS.md:104-124

Model Lifecycle

Engines must follow a uniform model lifecycle for every primitive they implement:

1. create: Allocate the implementation shell.
2. initialize: Load weights and model files.
3. use: Perform inference (e.g., generate, transcribe).
4. cleanup: Unload weights but keep the implementation shell alive.
5. destroy: Free the implementation memory.

Sources: engines/AGENTS.md:124-135

Engine Routing Logic

The routing system selects engines based on a plain priority order. When a component requires an AI primitive, it calls the plugin registry to find the appropriate engine.

Selection Process

The registry performs selection in two ways:

1. Automatic Selection: Callers invoke rac_plugin_find(primitive). The registry returns the engine with the highest priority value that serves that specific primitive. No complex scoring is performed.
2. Explicit Pinning: Callers invoke rac_plugin_find_for_engine(primitive, engine_name). This allows the system to bypass priority and use a specific backend, which is often used for hardware-specific optimizations (e.g., pinning neurt for Apple Neural Engine models).

Sources: engines/AGENTS.md:25-28, core/AGENTS.md:112-117

Routing Flow Diagram

The following diagram illustrates how a request for an LLM (Large Language Model) primitive is routed from the Service Layer to a specific Engine Plugin.

The Service Layer requests a primitive, and the Plugin Registry selects the engine with the highest priority (in this case, qhexrt).
Sources: core/AGENTS.md:112-117, engines/AGENTS.md:25-28

Plugin ABI (v9)

The Plugin ABI version is currently 9u. The registry rejects any plugin where the metadata.abi_version does not match RAC_PLUGIN_API_VERSION. This strict handshake ensures binary compatibility between the core library and dynamically loaded engines.

Primitive Slots

The registry manages routing for the following first-class primitives defined in the vtable:

Primitive	vtable Field	Purpose
RAC_PRIMITIVE_GENERATE_TEXT	llm_ops	Text generation (LLM)
RAC_PRIMITIVE_TRANSCRIBE	stt_ops	Speech-to-Text (STT)
RAC_PRIMITIVE_SYNTHESIZE	tts_ops	Text-to-Speech (TTS)
RAC_PRIMITIVE_DETECT_VOICE	vad_ops	Voice Activity Detection (VAD)
RAC_PRIMITIVE_EMBED	embedding_ops	Vector embeddings
RAC_PRIMITIVE_VLM	vlm_ops	Vision Language Models (VLM)
RAC_PRIMITIVE_DIFFUSION	diffusion_ops	Image generation/diffusion
RAC_PRIMITIVE_DIARIZE	diarization_ops	Speaker diarization
RAC_PRIMITIVE_SEGMENT	segmentation_ops	Image segmentation
RAC_PRIMITIVE_RERANK	rerank_ops	Cross-encoder reranking

Sources: core/AGENTS.md:154-180, engines/AGENTS.md:16-23

Registration Mechanisms

Engines can be registered using two primary modes depending on the platform and build configuration.

Static Registration

Used primarily on iOS and WebAssembly (WASM) where dynamic loading is restricted.

• The engine sources are compiled directly into the rac_commons library.
• The RAC_STATIC_PLUGIN_REGISTER(name) macro registers the engine before main() executes.
• Linker flags like -force_load (Apple) or --whole-archive (Linux) are required to prevent the linker from stripping the unreferenced registration code.

Dynamic Loading (dlopen)

Default on Android, Linux, macOS, and Windows.

• Engines are built as standalone shared libraries (e.g., librunanywhere_llamacpp.so).
• The host calls rac_registry_load_plugin().
• The loader uses dlsym to find the rac_plugin_entry_<name> symbol based on the filename.

Sources: engines/AGENTS.md:213-234, core/AGENTS.md:112-117

Registration Sequence

This diagram shows the dynamic loading sequence where the registry locates and validates the engine entry point.
Sources: engines/AGENTS.md:225-234, core/AGENTS.md:154-159

Engine-Runtime Relationship

Engines declare their dependencies on hardware runtimes (like CUDA, Metal, or ONNXRT) in their manifest.

• Metadata Only: Declaring a runtime is advisory. It informs the registry which hardware is required, but the registry does not automatically reject an engine if a runtime is missing unless the capability_check fails.
• Pattern 1 (Bundled): Engine (e.g., llamacpp) compiles its runtime (ggml) directly.
• Pattern 2 (External Library): Engine (e.g., onnx) calls into a separate runtime library (ONNX Runtime) for compute.

Sources: engines/AGENTS.md:176-200

Conclusion

The Plugin Registry and Engine Routing system provides the modular foundation for the RunAnywhere SDK. By decoupling AI capabilities into specific primitives and using a priority-based routing system, the SDK can support diverse hardware and AI models while maintaining a stable, versioned interface. This architecture allows developers to add new backends or hardware support by simply implementing the rac_engine_vtable_t contract and registering it with the core library.