Docker Deployment
Relevant source files
The following files were used as context for generating this wiki page:
Docker deployment provides an isolated environment to run the Colibrì inference engine. This method allows users to execute large-scale models like GLM-5.2 without local dependency conflicts. The deployment infrastructure supports interactive chat, diagnostic tools, and OpenAI-compatible API services.
Sources: docker/README.md:27-31, README.md:224-230
Architecture and Components
The Docker environment encapsulates the Colibrì C engine, Python launcher scripts, and necessary system libraries. It separates the persistent model weights from the ephemeral execution environment through volume mounting.
The diagram shows the relationship between the host filesystem, the containerized engine, and the external user interfaces.
Sources: docker/README.md:120-135, README.md:162-167
Container Images
Colibrì provides two primary Docker image configurations:
- Standard Dockerfile: Clones the repository inside the image, including the full build toolchain.
- Slim Dockerfile (Dockerfile.slim): A multi-stage build producing a ~115 MB image. It uses a portable instruction set architecture (
x86-64-v3) and runs as a non-root user (UID 1000).
Sources: docker/README.md:267-279
Deployment Modes
The Docker image supports multiple subcommands through its entrypoint, allowing the container to serve different roles.
| Mode | Command | Description |
|---|---|---|
| Chat | coli chat | Interactive terminal-based conversation mode. |
| Serve | coli serve | Starts the OpenAI-compatible API gateway. |
| Info | coli info | Displays engine and model metadata. |
| Doctor | coli doctor | Performs readiness and health checks on the environment. |
| Plan | coli plan | Inspects planned VRAM/RAM/disk placement. |
Sources: docker/README.md:281-285, README.md:225-230
Docker Compose API Service
You can deploy a one-command server using docker compose. This setup serves the OpenAI-compatible API on port 5000.
services:
colibri:
image: colibri
ports:
- "5000:5000"
volumes:
- ${MODEL_DIR}:/model
environment:
- COLI_MODEL=/model
- RAM_GB=${COLI_RAM:-24}
command: serve --host 0.0.0.0 --model-id glm-5.2 --ram ${COLI_RAM:-24}
Sources: docker/docker-compose.yml:1-12, docker/README.md:295-300
Configuration and Environment Variables
Configuration is managed via environment variables passed to the docker run command or defined in the compose file.
| Variable | Description | Default/Example |
|---|---|---|
COLI_MODEL | Internal container path to the model directory. | /app/glm-5.2 |
RAM_GB | Memory budget allocated for resident weights. | 24 |
COLI_CUDA | Enables CUDA GPU tier support. | 1 |
TEMP | Sampling temperature for the model. | 0 |
TOKENS | Enables token-id dumps to stderr. | 1 |
Sources: docker/README.md:120-135, c/tools/diag_harness.py:199-210
Deployment Workflow
The following sequence describes the process of initializing a Docker-based inference session.
The sequence highlights the requirement for pre-downloaded weights before container execution.
Sources: docker/README.md:70-120, README.md:210-220
Troubleshooting and System Limits
Memory Allocation (WSL2/Windows)
On Windows, Docker Desktop uses WSL2, which often defaults to 50% of host RAM. Running Colibrì (minimum 24-32GB RAM) requires manual adjustment of .wslconfig.
[wsl2]
memory=24GB
processors=12
swap=16GB
Sources: docker/README.md:162-180
Permission Management
- Linux Root Access: Docker requires
sudounless the user is added to thedockergroup. - Non-Root Slim Image: The slim image uses UID 1000. The host model directory must have
chmod -R a+rXpermissions to be readable by the container user.
Sources: docker/README.md:204-215, docker/README.md:288-293
Summary
Docker Deployment for Colibrì enables running 744B-parameter models on consumer hardware by abstracting the complex C engine and its hierarchy-aware memory management. By leveraging volume mounts for weight streaming and environment variables for tuning, users can deploy stable, isolated instances for both interactive use and API services.
Sources: docker/README.md:300-305, README.md:235-245