Workspace & Sub-crate Structure

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• CLAUDE.md
• PRD.md
• README.md
• oxidize-core/src/lib.rs
• oxidize-cli/AGENTS.md
• oxidize-server/src/AGENTS.md

The oxidize project uses a layered Cargo workspace to organize its LLM inference engine and associated tooling. This structure centralizes core inference logic while providing multiple specialized interfaces, such as a CLI, an OpenAI-compatible server, and various language bindings. The architecture ensures consistent behavior across different platforms and deployment methods by strictly separating the compute layer from the interface layer.

Sources: PRD.md:23-40, README.md:3-10, README.md:195-212

Core Architecture Layers

The workspace organizes components into a vertical stack to manage the transition from user input to hardware-accelerated computation.

Crate Summary

The project consists of multiple crates, each serving a distinct role in the inference pipeline or model management lifecycle.

Crate Name	Domain / Purpose	Key Responsibilities
oxidize-core	Core Inference Engine	GGUF parsing, tensor ops, quantization, sampling, and hardware backends.
oxidize-cli	Command Line Interface	Prompting, chat REPL, benchmarking, and profiling.
oxidize-server	HTTP API Server	OpenAI-compatible endpoints (Axum) and continuous batching.
oxidize-quantize	Model Quantization	Offline weight conversion (e.g., F32 to Q4_0).
oxidize-convert	Format Conversion	Converting SafeTensors to GGUF format.
oxidize-ffi	C-ABI Interface	FFI surface for oxidize-core consumption.
oxidize-kernels	CPU Kernels	Hand-tuned CPU GEMV kernels (OXK).
oxidize-py	Python Bindings	PyO3-based bindings for Python integration.

Sources: CLAUDE.md:14-36, AGENTS.md:14-36, README.md:3-10

Sub-crate Details

oxidize-core

This is the central crate that all other components depend on. It flattens most of its internal modules into the crate root via lib.rs using the #[path = "..."] attribute. It handles model loading, memory mapping via memmap2, and provides the ComputeBackend and Model traits.

Key structures include:

• ComputeBackend: A trait implemented by all hardware backends (CUDA, Metal, Vulkan, etc.).
• Model: A trait implemented by specific inference architectures like Llama.
• WorkspaceHealth: A struct used to signal readiness, returned by workspace_health().

Sources: oxidize-core/src/lib.rs:43-118, CLAUDE.md:67-73, PRD.md:143-150

oxidize-cli

The CLI acts as a thin wrapper over oxidize-core. It manages the user session, handles signal interrupts (like Ctrl-C for the chat REPL), and implements pipeline modes for batch processing prompts from files or stdin.

Sources: oxidize-cli/AGENTS.md:1-15

oxidize-server

The server crate utilizes the Axum framework to provide OpenAI-compatible routes. It maintains an AppState that holds the model runtime, rate limiting state, and optional mesh cluster configurations.

Sources: oxidize-server/src/AGENTS.md:1-18

Cross-Language Ports

In addition to the primary Rust workspace, the project maintains parallel ports to other languages to ensure cross-platform deployment flexibility.

• Go (oxidize-golang): A pure-Go port mirroring the Rust structure, including GGUF parsing and tensor operations.
• Python (oxidize-python): A pure-Python implementation distinct from the oxidize-py bindings.
• C (oxidize-c): A dependency-free C11 port designed for minimal environments.
• C++ (oxidize-cpp): A C++20 port focusing on performance parity with llama.cpp.

Sources: CLAUDE.md:104-108, AGENTS.md:104-108, oxidize-golang/AGENTS.md:1-20

Workspace Configurations

The root Cargo.toml defines the workspace members and optimizes the release profile for inference performance.

[workspace]
members = ["oxidize-core", "oxidize-cli", "oxidize-server", "oxidize-quantize"]
resolver = "3"

[profile.release]
lto = true
panic = "abort"

Sources: PRD.md:46-50, CLAUDE.md:139-140, oxidize-core/src/lib.rs:163-172

The project uses a flat module system in oxidize-core, where subdirectories like mesh/, paged_attention/, vision/, and video/ contain their own mod.rs files, while other modules are flattened into the core library.

Sources: CLAUDE.md:112-115, oxidize-core/src/lib.rs:43-118

The oxidize workspace provides a modular architecture that separates high-level interface concerns from low-level compute kernels. By centralizing inference logic in oxidize-core and utilizing a shared GGUF format, the project ensures consistent model behavior across CLI, Server, and various language-specific ports.