Python Bindings & Pure Python Port
Relevant source files
The following files were used as context for generating this wiki page:
The Python integration in the Oxidize workspace consists of two distinct implementations: oxidize-py, which provides high-performance Rust bindings via PyO3, and oxidize-python, a pure-Python port of the core inference engine. These components allow Python developers to access local LLM inference capabilities using either the native Rust engine for speed or a standalone Python implementation for cross-platform portability and ease of modification.
The Rust-based bindings focus on exposing oxidize-core primitives like model loading and token generation to the Python environment. In contrast, the pure-Python port mirrors the architecture of oxidize-core and oxidize-golang, implementing tensor operations, quantization, and model logic directly in Python.
Sources: oxidize-py/AGENTS.md:5-10, oxidize-python/AGENTS.md:5-10, PRD.md:310-315
oxidize-py: Rust Bindings (PyO3)
oxidize-py is a Python extension module built with maturin and pyo3. It wraps the native Rust oxidize-core library to provide high-performance inference.
Core Architecture and Features
The module centers around a Llama class that manages the underlying Rust model state and session. It handles the boundary between Python types and Rust data structures, performing validation before execution.
The diagram shows how oxidize-py acts as a translation layer between Python calls and the native Rust inference engine.
Sources: oxidize-py/AGENTS.md:12-28, CLAUDE.md:65-70
Key Classes and Methods
The primary interface is the Llama class, which exposes methods for both synchronous and asynchronous generation.
| Method | Type | Description |
|---|---|---|
__init__ | Constructor | Initializes the model with configuration (e.g., vocab_size, context_size). |
generate | Synchronous | Generates text based on a provided prompt. |
generate_async | Asynchronous | Bridges to the tokio runtime using a thread pool for non-blocking execution. |
create_chat_completion | Synchronous | Processes a list of messages for chat-based interactions. |
Sources: oxidize-py/AGENTS.md:5-20, oxidize-py/src/lib.rs:5-10
oxidize-python: Pure Python Port
oxidize-python is a standalone reimplementation of the Oxidize inference engine. It targets feature parity with the Rust core and Go ports, using uv for package management and pytest for verification.
System Structure
The port mirrors the Rust crate structure to maintain consistency across the workspace. It organizes code into core tensor operations, model architectures, and internal server components.
The class diagram represents the structural mirroring of Rust traits to Python Protocols and classes within the port.
Sources: oxidize-python/AGENTS.md:12-65, CLAUDE.md:428-435
Implementation Components
| Module | Location | Role |
|---|---|---|
| Tensor Ops | core/tensor/ | Implements GEMV, parallel operations, and basic tensor math. |
| Quantization | core/quantization/ | Handles block-wise quantization, k-quants, and dequantization. |
| Model Logic | core/model/ | Contains Llama architectures, DFlash speculative decoding, and sampling. |
| GGUF Core | core/ggufcore/ | Implements the GGUF v3 parser and metadata handling. |
| API Server | internal/server.py | Provides an OpenAI-compatible HTTP server. |
Sources: oxidize-python/AGENTS.md:15-60, CLAUDE.md:430-445
Cross-Implementation Parity
The workspace enforces strict parity between the different implementations to ensure consistent behavior across languages.
- Shared Fixtures: Both implementations use
oxidize-core/tests/fixtures/valid-v3.gguffor parity testing. - Naming Conventions: Python
@dataclassandpydantic.BaseModelinstances mirror RustXxxConfigstructs. - Trait Mirroring: Rust traits (e.g.,
ComputeBackend) are implemented astyping.Protocolclasses in Python. - Bit-Exact Parity: Dequantization and forward-pass logits are required to match the Rust reference.
Sources: oxidize-python/AGENTS.md:78-85, CLAUDE.md:465-470
Development Workflow
Developers can build and interact with both versions using standard Python and Rust tooling.
# Building Rust Bindings
maturin develop -m oxidize-py/Cargo.toml
# Running Pure Python Port
cd oxidize-python
uv sync
uv run oxidize --help
Sources: oxidize-py/AGENTS.md:32-40, oxidize-python/AGENTS.md:68-75
Conclusion
The Python infrastructure provides a dual-path approach to local LLM inference. oxidize-py offers the highest performance by leveraging the Rust core, while oxidize-python provides a maintainable, pure-Python alternative that mirrors the workspace architecture. Together, they ensure the Oxidize engine is accessible across the entire Python data science and web ecosystem.
Sources: PRD.md:320-330, README.md:10-20