Web SDK & WebAssembly
Relevant source files
The following files were used as context for generating this wiki page:
The Web SDK provides a Swift-aligned TypeScript facade over the cross-platform C++ core, enabling on-device AI inference within modern web browsers. It utilizes Emscripten-compiled WebAssembly (WASM) artifacts to execute high-performance inference logic for various modalities, including Large Language Models (LLM), Speech-to-Text (STT), and Text-to-Speech (TTS).
The architecture prioritizes a "commons-first" approach where the C++ core manages business logic, modality pipelines, and streaming state machines. The TypeScript layer remains thin, focusing on browser-specific I/O coordination, BackendWorker hosting, and providing a public API that mirrors the project's mobile SDKs. Sources: bindings/web/AGENTS.md:14-25, bindings/web/AGENTS.md:37-43
Architecture and Package Structure
The Web SDK is organized into three primary NPM packages. Each package contains self-contained WASM artifacts that embed the required C++ commons code. Applications only load the specific backend modules needed for their requested AI features. Sources: bindings/web/AGENTS.md:5-12
Component Packages
| Package | Purpose | Primary Artifacts |
|---|---|---|
@runanywhere/web | Public core facade and generated proto types. | racommons.{js,wasm} |
@runanywhere/web-llamacpp | LLM, VLM, and LoRA backend. | racommons-llamacpp.{js,wasm} (CPU), racommons-llamacpp-webgpu.{js,wasm} |
@runanywhere/web-onnx | STT, TTS, and VAD backend. | racommons-onnx-sherpa.{js,wasm} |
Sources: bindings/web/AGENTS.md:7-12, bindings/web/tests/browser/multi-wasm-init.spec.ts:8-19
Execution Environment
The SDK executes all inference and model mutation within a BackendWorker to keep the UI thread responsive. The system prioritizes WebGPU acceleration (navigator.gpu) when supported by the device; otherwise, it falls back to a multi-threaded CPU worker. Sources: bindings/web/AGENTS.md:45-50
The diagram shows the relationship between the browser application, the TypeScript facade, and the underlying WASM modules running within a worker. Sources: bindings/web/AGENTS.md:45-50, bindings/web/tests/browser/multi-wasm-init.spec.ts:8-19
Initialization Flow
The SDK follows a multi-phase initialization process. Each backend must be explicitly registered to load its respective WASM artifact and register its functionality with the core plugin registry. Sources: bindings/web/AGENTS.md:146-156
- Phase 1: Core Initialization: Loads the base
racommons.wasmand establishes the core SDK state. - Backend Registration: Individual backends (LlamaCPP, ONNX) are registered. This step loads the dedicated backend WASM and registers its vtable with the C++ plugin system.
- Phase 2: Service Completion: Finalizes service setup, including model catalog hydration and background identity verification.
Sources: bindings/web/AGENTS.md:158-164, bindings/web/tests/browser/multi-wasm-init.spec.ts:31-35
Initialization Sequence
This sequence illustrates the mandatory registration steps required before the SDK is fully operational. Sources: bindings/web/AGENTS.md:146-156
WebAssembly and Plugin System
The SDK utilizes a unified C ABI (rac_*) for communication between the TypeScript layer and the WASM modules. The C++ core implements a plugin registry that dispatches requests to the appropriate backend based on the model framework. Sources: bindings/web/AGENTS.md:37-41, AGENTS.md:104-108
Backend Modalities
- LlamaCPP Backend: Handles LLM text generation, GGUF embeddings, and Vision Language Models (VLM). It supports both standard CPU execution and an Asyncify-based WebGPU variant.
- ONNX/Sherpa Backend: Provides speech-related capabilities including STT and TTS. It registers two distinct vtables (
onnxandsherpa) within a single WASM module. - Core Module: Provides foundational services such as authentication, telemetry, and the model registry shared across all backends.
Sources: bindings/web/AGENTS.md:10-12, bindings/web/tests/browser/multi-wasm-init.spec.ts:13-19
Build and Artifact Management
WASM artifacts are compiled using CMake and Emscripten. The build process generates five distinct artifacts to satisfy different execution modes and backend requirements. Sources: bindings/web/AGENTS.md:126-134
Build Commands
# Core commons WASM
npm run build:wasm -- --core
# LlamaCPP variants
npm run build:wasm -- --llamacpp
npm run build:wasm -- --webgpu
# ONNX variants
npm run build:wasm -- --onnx
npm run build:wasm -- --onnx-webgpu
Sources: bindings/web/AGENTS.md:126-134
Deployment Requirements
Browser environments must be configured with specific headers to enable high-performance features like multi-threading (pthreads) and large memory allocations:
Cross-Origin-Opener-Policy: same-originCross-Origin-Embedder-Policy: require-corp
Sources: bindings/web/CLAUDE.md:273-277, bindings/web/AGENTS.md:204-208
Technical Constraints and Rules
- TypeScript Facade: Must remain thin. It should only handle browser I/O, protobuf encoding/decoding, and BackendWorker orchestration. Business logic must not be reimplemented in TypeScript. Sources: bindings/web/AGENTS.md:37-43
- Protobuf Integration: All modality, environment, and error types are derived from generated
@runanywhere/proto-tsmodules. Sources: bindings/web/AGENTS.md:80-84 - Security: Sensitive data such as API keys or secret-bearing URLs must never be persisted in browser storage (localStorage, OPFS, or IndexedDB). Sources: bindings/web/AGENTS.md:95-99
- WASM Isolation: There is no symbol sharing between different WASM artifacts; each module is self-contained. Sources: bindings/web/AGENTS.md:5-7
Summary
The Web SDK & WebAssembly system provides a robust framework for local AI inference. By offloading heavy computation to specialized WASM modules running in background workers, the SDK maintains UI performance while leveraging hardware acceleration. The architecture ensures consistency with other platform SDKs through a shared C++ core and protobuf-based data models. Sources: bindings/web/AGENTS.md:5-12, bindings/web/AGENTS.md:45-50