Windows Build & Configuration
Relevant source files
The following files were used as context for generating this wiki page:
Windows Build & Configuration encompasses the processes for deploying the Colibrì inference engine on Windows environments, including native execution and Docker-based virtualization. The system supports high-performance inference for the GLM-5.2 744B model by managing memory hierarchies across VRAM, RAM, and NVMe storage. Configuration involves setting environment variables to tune the engine's behavior, such as memory budgeting and GPU acceleration through CUDA.
Sources: README.md:1-20, docker/README.md:10-30
Hardware and Software Prerequisites
Running the Colibrì engine on Windows requires specific hardware capabilities due to the model's massive parameter count. While the engine streams experts from disk to minimize VRAM requirements, a minimum of 32 GB of RAM is recommended for stable performance.
System Requirements
| Component | Minimum Requirement | Recommended / Notes |
|---|---|---|
| RAM | 32 GB | 24 GB may work; 48+ GB preferred for higher hit rates. |
| Storage | 370 GB free space | Fast NVMe SSD essential; 1-15 GB/s bandwidth recommended. |
| GPU | Optional (NVIDIA) | Supports CUDA for resident dense weights and expert tiers. |
| OS | Windows 10/11 | Native execution via MinGW or virtualization via Docker/WSL2. |
| Python | Python 3.x | Required for the coli launcher and API gateway. |
Sources: docker/README.md:35-60, README.md:150-165
Native Execution and CLI Tools
The Windows native engine is distributed as colibri.exe. The coli launcher manages the engine, providing subcommands for health checks, planning, and inference.
Core CLI Commands
coli chat: Starts an interactive chat session.coli plan: Inspects planned expert placement across VRAM, RAM, and disk.coli doctor: Performs a readiness check on the environment and model files.coli serve: Launches an OpenAI-compatible API gateway.
Sources: README.md:175-185, CHANGELOG.md:135-145
Diagnostic Harness
The diag_harness.py tool automates system verification on Windows. It executes multiple phases to ensure the engine operates correctly within hardware constraints.
The diagnostic harness captures telemetry including GPU VRAM, RAM budget, and MTP status.
Sources: c/tools/diag_harness.py:7-25, c/tools/diag_harness.py:200-250
Environment Configuration
Windows users configure the engine primarily through environment variables. These variables control memory allocation, I/O strategies, and GPU utilization.
Key Environment Variables
| Variable | Description |
|---|---|
COLI_MODEL | Path to the model directory (e.g., D:\glm52_i4). |
RAM_GB | Manual override for RAM allocation (0 = auto). |
COLI_CUDA | Set to 1 to enable the CUDA GPU tier. |
COLI_GPU | Specifies the GPU device ordinal. |
DIRECT | Set to 1 to use O_DIRECT for NVMe bypassing the page cache. |
PIPE | Enables async expert I/O pool (default enabled). |
Sources: README.md:85-110, c/tools/diag_harness.py:465-485
Docker and WSL2 Configuration
For Windows users utilizing Docker, performance is heavily dependent on the Windows Subsystem for Linux (WSL2) configuration.
WSL2 Memory Tuning
WSL2 often limits memory to 50% of host RAM by default. Users must create a .wslconfig file in their user profile directory to unlock sufficient resources for the 744B model.
[wsl2]
memory=24GB
processors=12
swap=16GB
Sources: docker/README.md:160-190
Docker Deployment Flow
The following sequence describes the deployment of a Colibrì container on Windows:
Sources: docker/README.md:85-115
Model Acquisition and Conversion
The engine requires a specific int4 container. Users can download pre-converted models or perform local conversion.
- Download: Using
huggingface_hubto fetch the GLM-5.2-colibri-int4-g64 model. - Conversion: The
coli convertcommand transforms FP8 shards into the tiered int4 format compatible with the Windows engine. - Validation: The
coli doctor --deepcommand verifies tensor integrity and shard indexing.
Sources: README.md:165-180, c/download_fp8.py:100-120
Summary
Windows Build & Configuration enables the execution of 700B+ parameter models on consumer-grade hardware by leveraging a sophisticated storage hierarchy. Users can choose between native MinGW-compiled binaries or Docker containers. Success on Windows depends on high-speed NVMe storage and correct WSL2/Environment variable tuning to balance weights between VRAM, RAM, and disk.
Sources: README.md:200-215, docker/README.md:270-280