C & C++ Ports

Relevant source files

The following files were used as context for generating this wiki page:

• oxidize-ffi/src/lib.rs
• oxidize-cpp/src/autotune.cpp
• oxidize-c/CONTRIBUTING.md
• AGENTS.md
• CLAUDE.md
• PRD.md

C and C++ ports provide high-performance, cross-platform deployment options for the Oxidize inference engine. These ports exist as independent implementations of the core Rust logic, allowing for dependency-free deployment in environments where a Rust runtime is not preferred. The C port (oxidize-c) focuses on a dependency-free C11 implementation, while the C++ port (oxidize-cpp) targets C++20 and aims for feature parity with llama.cpp.

Sources: CLAUDE.md:12-25, AGENTS.md:12-25, oxidize-c/CONTRIBUTING.md:16-20

C11 Port (oxidize-c)

The C port is a standalone, dependency-free implementation written in C11. It provides a low-level interface for LLM inference and supports optional hardware acceleration through custom backends.

Technical Conventions

The oxidize-c module follows strict naming and architectural conventions to mirror the Rust core:

• Standards: Uses C11 (-std=c11) without GNU extensions unless gated by macros.
• Naming: Employs OcPascalCase for types and oc_snake_case for functions.
• Memory Management: Uses OcArena for model-lifetime allocations and mmap for GGUF weights to ensure efficient resource handling.
• Error Handling: Returns OcError from all public functions instead of using process exits.

Sources: oxidize-c/CONTRIBUTING.md:27-40

Build Configurations

The C port supports multiple hardware-specific build targets:

Build Flag	Target	Description
Default	CPU	Links only against libc, libm, and libpthread.
make avx512	AVX-512	Enables BW+VNNI instructions for optimized CPU compute.
make cuda	CUDA	Enables OC_CUDA backend for NVIDIA GPU acceleration.
make manual-tests	DFlash2	Runs opt-in DFlash2 parity validators.

Sources: oxidize-c/CONTRIBUTING.md:7-14

C++20 Port (oxidize-cpp)

The C++ port implements Llama-family inference using C++20. It focuses on high-speed execution and feature parity with existing industry standards like llama.cpp.

Hardware Autotuning

The C++ implementation includes a sophisticated autotuning system that detects hardware capabilities to optimize inference parameters. It specifically manages thread counts and NUMA (Non-Uniform Memory Access) modes based on the model size and available system RAM.

The autotuner evaluates system memory via /proc/meminfo and detects 2MiB hugepage availability to determine if mmap_hugepages should be enabled for large models.

Sources: oxidize-cpp/src/autotune.cpp:13-100

Inference Architecture

The C++ port utilizes a specialized model structure defined in oxidize/model_llama.hpp to handle Llama-family architectures. It supports both CPU and optional GPU fast paths.

• LlamaModel: The primary structure for C++ Llama inference.
• Resident Forward: A CUDA fast path that achieves approximately 1 synchronization per token.

Sources: CLAUDE.md:68, AGENTS.md:68, CLAUDE.md:143-148

C-ABI FFI (oxidize-ffi)

While the C and C++ ports are independent implementations, the project also provides oxidize-ffi, which is a C-ABI compatible interface built directly over the Rust oxidize-core.

FFI Components

The FFI layer exposes core primitives as C-compatible structures and handles:

• DType and Quantization: Maps GgufQuantizationType and DType to C-compatible representations.
• Model Loading: Provides C-callable functions to load models and adapters.
• Inference Sessions: Manages stateful inference sessions for external callers.

Sources: oxidize-ffi/src/lib.rs:1-10, CLAUDE.md:46

The sequence above illustrates how the FFI bridge facilitates communication between external C-based applications and the core Rust inference engine.

Sources: oxidize-ffi/src/lib.rs

Cross-Language Parity

A critical requirement for all ports is bit-exact parity with the Rust oxidize-core implementation.

Validation Strategies

• Golden Fixtures: C and C++ ports must match Rust output bit-exactly on tiny fixtures stored in oxidize-core/tests/fixtures/.
• Tolerance Levels: Parity requirements allow 0 tolerance for integer operations and ≤1e-4 relative tolerance for floating-point paths.
• Quantization Consistency: Custom AL-family quantization types (AL5, AL6, AL8) must be implemented consistently across Rust, C++, and C ports.

Sources: oxidize-c/CONTRIBUTING.md:94-106, CLAUDE.md:150-155

The C and C++ ports ensure the Oxidize engine remains versatile, enabling performance-critical deployments without requiring the Rust toolchain on the target hardware.