Golang Port

Relevant source files

The following files were used as context for generating this wiki page:

• oxidize-golang/AGENTS.md
• CLAUDE.md
• AGENTS.md
• README.md
• PRD.md
• oxidize-golang/README.md

The Golang Port is a pure-Go reimplementation of the oxidize-core functionality. It serves as a parallel language port within the oxidize workspace to support local-first LLM inference in Go environments. The port aims for feature parity with the Rust core, including GGUF parsing, tensor operations, quantization, and model inference.

Development of this port occurs in the go/initial-oxidize-port branch. It mirrors the Rust workspace structure to maintain consistency across the project. Key priorities for the port include implementing DFlash speculative decoding and ensuring bit-exact parity with Rust CPU kernels.

Sources: oxidize-golang/AGENTS.md:5-9, CLAUDE.md:5-10, oxidize-golang/README.md:1-3

Architecture and Structure

The oxidize-golang module uses a structure that maps Go packages 1:1 to Rust modules from oxidize-core. This organization ensures that developers familiar with the Rust implementation can navigate the Go port efficiently.

Package Organization

The following table summarizes the primary directories and their roles:

Directory	Purpose	Rust Mirror
core/tensor/	Tensor operations, dtypes, and GEMV	tensor.rs
core/ggufcore/	GGUF v3 file format parser	gguf.rs
core/quantization/	Quantize/dequantize logic and k-quant types	quantization.rs
core/model/	Inference engine, sampling, and generation	inference.rs / generation.rs
core/backends/	Compute backend abstractions	ComputeBackend trait
internal/server/	OpenAI-compatible HTTP server	oxidize-server

Sources: oxidize-golang/AGENTS.md:12-40, CLAUDE.md:144-148

Dependency and Interface Mapping

The port implements Rust traits as Go interfaces. For example, the ComputeBackend trait in Rust becomes a ComputeBackend interface in Go. The module is identified as github.com/Zapdev-labs/oxidize/golang in its go.mod file.

The following diagram illustrates the relationship between the core library components and the internal runtime packages.

The diagram shows how the core/ library provides the heavy-lifting logic for tensors and models, while internal/ handles the API and server runtime.
Sources: oxidize-golang/AGENTS.md:12-40, oxidize-golang/AGENTS.md:55-57

Development Guidelines

The Golang port follows specific conventions to ensure quality and parity with the Rust reference implementation.

Implementation Rules

• Error Handling: You must use fmt.Errorf for error wrapping to maintain the cause chain. Avoid using panic() in library code.
• Parity Testing: You must reuse Rust test fixtures, such as valid-v3.gguf, for cross-language parity tests.
• Feature Detection: The internal/buildinfo/ package mirrors the Rust XxxBuildInfo pattern to detect hardware features at compile time.
• Tooling: The project requires CGO_ENABLED=0 for standard builds to ensure portability, though CGO is permitted for native GPU bindings (CUDA, Vulkan).

Sources: oxidize-golang/AGENTS.md:55-63, CLAUDE.md:300-302, PRD.md:13-16

Build and Test Procedures

1. Build the module: Navigate to oxidize-golang and run go build ./....
2. Execute tests: Run go test ./.... For slow GGUF tests, set OXIDIZE_SLOW_TESTS=1.
3. Start the server: Run go run ./cmd/server/main.go. By default, this starts an OpenAI-compatible API.

Sources: oxidize-golang/AGENTS.md:47-53, CLAUDE.md:300-302

Component Deep Dive

GGUF Parsing and Quantization

The ggufcore and gguf packages handle the reading and writing of GGUF v3 metadata and tensors. The quantization package implements block-wise INT4 and INT8 operations, mirroring the AL family of quant types (AL5, AL6, AL8) found in the core engine.

The sequence diagram details the initialization flow when loading a model file within the Go port.
Sources: oxidize-golang/AGENTS.md:19-22, CLAUDE.md:286-290, PRD.md:43-50

Inference and Speculative Decoding

The core/model/ package manages the inference loop. A primary focus is the porting of DFlash speculative decoding from Rust. This involves managing a draft model and a target model to accelerate token generation. If a hidden-size mismatch occurs between the draft and target, the system falls back to target-only inference.

Sources: oxidize-golang/AGENTS.md:69-72, CLAUDE.md:294-298

Summary

The Golang Port provides a native Go implementation of the oxidize inference engine. By mirroring the Rust structure and prioritizing bit-exact parity in tensor operations, it allows Go developers to deploy local LLMs with the same performance and quality characteristics as the Rust core. The port includes a full suite of tools, from GGUF parsing to an OpenAI-compatible HTTP server.

Sources: oxidize-golang/AGENTS.md:5-10, CLAUDE.md:5-10