LoRA & Supervised Fine-Tuning
Relevant source files
The following files were used as context for generating this wiki page:
LoRA (Low-Rank Adaptation) and Supervised Fine-Tuning (SFT) within the Oxidize project provide tools for adapting GGUF models on commodity CPU hardware. This module facilitates model optimization through iterative training rounds, synthetic data generation, and adapter merging. The system is designed to run locally-first, utilizing oxidize-core primitives for model loading and tensor operations.
Sources: oxidize-finetuning/AGENTS.md:1-10, PRD.md:214-214, CLAUDE.md:143-145
System Architecture
The fine-tuning system operates as a Rust CLI and library. It leverages a LayerWiseModel to manage memory-efficient training by processing model layers sequentially. The architecture supports multiple training strategies, including standard SFT and iterative self-training using synthetic dialogue.
Sources: oxidize-finetuning/AGENTS.md:5-15, CLAUDE.md:143-145
Training Components and Data Flow
The following diagram illustrates the data flow from raw datasets through the training loop to exported LoRA adapters.
The SftTrainer coordinates the forward and backward passes, applying gradient updates via the AdamW optimizer to LoRA adapters.
Sources: oxidize-finetuning/AGENTS.md:16-25, oxidize-finetuning/src/trainer.rs:9-9
Training Strategies
The project defines four primary fine-tuning strategies. While SFT and Self-Training are fully implemented, DPO and PPO are currently stubs.
| Strategy | Description | Status |
|---|---|---|
| SFT | Supervised Fine-Tuning on prompt-response pairs. | Fully Implemented |
| Self-Train | Iterative training using self-dialogue and critique. | Fully Implemented |
| DPO | Direct Preference Optimization. | Stub / Not Implemented |
| PPO | Proximal Policy Optimization. | Stub / Not Implemented |
Sources: oxidize-c/include/oxidize/finetune.h:24-29, oxidize-finetuning/AGENTS.md:8-10
Self-Training Loop
The self-training loop executes iterative rounds of data generation and refinement. It produces synthetic.jsonl files through self-dialogue, which are then used for subsequent training rounds to improve model performance without external labels.
Self-training supports resuming from specific checkpoints using the --resume-from flag.
Sources: oxidize-finetuning/AGENTS.md:20-21, CLAUDE.md:143-145
Configuration and Implementation
Fine-tuning parameters are managed through a configuration structure that defines rank, alpha, and learning schedules.
Configuration Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
lora_rank | uint32 | 8 | The rank for the LoRA adapter matrices. |
lora_alpha | uint32 | 16 | Scaling factor for LoRA updates. |
learning_rate | float | 2e-4 | Initial step size for the optimizer. |
epochs | uint32 | 1 | Number of complete passes over the dataset. |
max_seq_len | uint32 | 512 | Maximum number of tokens per sequence. |
Sources: oxidize-c/include/oxidize/finetune.h:31-46, oxidize-finetuning/AGENTS.md:31-35
Key Implementation Modules
The oxidize-finetuning crate is partitioned into specialized modules for math, data handling, and export:
trainer.rs: Implements theSftTrainerwhich handles packing, forward passes, and backpropagation.lora.rs: Defines theLoRAAdapterstructure and associated linear algebra.dataset.rs: Manages loaders for Alpaca and ShareGPT formats, includingpack_chunksfor efficient batching.merge.rs: Provides theAdapterMergerfor combining multiple LoRA GGUFs using different strategies.export.rs: Handles GGUF adapter I/O and manifest serialization.
Sources: oxidize-finetuning/AGENTS.md:16-25, CLAUDE.md:143-145
CLI Usage
The fine-tuning tool is accessed via the oxidize-finetuning binary. It requires a base GGUF model and a dataset in JSONL format.
# Basic Supervised Fine-Tuning
cargo run -p oxidize-finetuning -- sft \
--model base.gguf \
--dataset data.jsonl \
--lora-rank 16 \
--learning-rate 2e-4
# Iterative Self-Training
cargo run -p oxidize-finetuning -- self-train \
--model base.gguf \
--dataset data.jsonl \
--resume-from checkpoint_round_1.gguf
Sources: oxidize-finetuning/AGENTS.md:30-40, CLAUDE.md:118-118
Summary
LoRA & Supervised Fine-Tuning in Oxidize enables local CPU-based model adaptation. By utilizing LayerWiseModel and AdamW optimization, the system provides a memory-efficient path for training GGUF models. Key capabilities include iterative self-training with synthetic data generation and flexible adapter merging for combining different training runs.