Apple MLX & NeuRT Engines

Apple MLX and NeuRT are high-priority inference engines within the RunAnywhere ecosystem specifically optimized for Apple hardware. These engines allow the SDK to perform low-latency AI tasks, such as Large Language Model (LLM) generation and Stable Diffusion, by leveraging the Apple Neural Engine (ANE) and Metal-accelerated compute.

The engines integrate with the cross-platform C++ core via the rac_engine_vtable_t plugin interface. While they serve overlapping modalities like LLM and STT, they target different runtime paths: MLX acts as a Swift-based bridge to the MLX framework, whereas NeuRT drives Core ML and ANE-specific model graphs. Sources: engines/AGENTS.md, core/README.md

Architecture Overview

Both engines function as backend plugins registered within the core's plugin registry. They fill specific slots in the engine vtable to declare support for AI primitives. The core router selects these engines based on a combination of priority and explicit engine pins.

The diagram shows the dispatch flow from the high-level SDK through the C++ core to the Apple-specific engines. Sources: engines/AGENTS.md, core/AGENTS.md

Capabilities and Modalities

Sources: engines/AGENTS.md, core/README.md

The MLX Engine

The MLX engine wraps the Apple MLX framework through a Swift bridge (mlx-swift-lm). It is designated as a "Pattern 1" engine, meaning it bundles or directly manages its own runtime compute rather than calling into a separate runtime adapter.

• Priority: 110 (Highest priority for Apple platforms).
• Availability: Public, but restricted to Apple hardware via mlx_capability_check.
• Execution: Physical device only; registration returns false or silent-rejects in the iOS Simulator.
• Registration: Dynamic registration occurs via the linked Swift MLX runtime.

Sources: engines/AGENTS.md, bindings/react-native/AGENTS.md

MLX Data Flow

MLX uses a Swift callback bridge to facilitate communication between the C++ core and the Swift-based MLX implementation.

The sequence illustrates how the MLX engine bridges the C++ request to the Swift-based MLX framework. Sources: core/AGENTS.md, bindings/react-native/AGENTS.md

The NeuRT Engine

NeuRT (Neural RunTime) is an identity-named engine that manages Core ML models and ASR (Automatic Speech Recognition) graphs. It is consumed as prebuilt archives from the private neurun repository and pinned via SHA-256 in core/VERSIONS.

• Priority: 100 (Kept below MLX to allow explicit name-pinning).
• Specialization: Highly optimized for the Apple Neural Engine (ANE).
• Modality-Specific Drivers: Includes a bidirectional-encoder driver for embeddings and cross-encoder reranking.
• Diffusion: Implements Stable Diffusion pipelines using MLModel.

Sources: engines/AGENTS.md, core/README.md

Implementation Patterns

NeuRT follows a "Pattern 5" shell, where it can exist in two states based on build-time macros:

1. ROUTABLE: Full manifest and vtable filled with the private archive implementation.
2. STUB: An all-NULL shell used when private archives are absent, allowing public CI to build without access to private source code.

Sources: engines/AGENTS.md

Registration and Initialization

Both engines utilize a standardized registration lifecycle. For static hosts like iOS, they use the RAC_STATIC_PLUGIN_REGISTER macro to prevent the linker from stripping the engine code.

1. Phase 1 (Sync): Register the engine vtable and manifest with the ModuleRegistry.
2. Phase 2 (Async): Discover downloaded model artifacts located in {base_dir}/RunAnywhere/Models/.
3. Dispatch: The core calls vtable->{primitive}_ops->create(model_id, config_json, **impl) to instantiate a specific model session.

Sources: engines/AGENTS.md, core/AGENTS.md, bindings/react-native/AGENTS.md

Summary

Apple MLX and NeuRT engines provide the performance backbone for RunAnywhere on iOS and macOS. MLX serves as the primary high-priority engine for a wide range of tasks including LLM and Vision-Language Models (VLM). NeuRT provides specialized support for Core ML and the Apple Neural Engine, particularly for Diffusion and ASR tasks. Together, they ensure that Apple users receive highly optimized, hardware-accelerated on-device AI. Sources: engines/AGENTS.md, core/README.md