Volume Mounting and Storage

Introduction

In the RunPod worker for Ollama, volume mounting and storage management are critical for maintaining Large Language Model (LLM) weights and ensuring efficient startup times. The system is designed to leverage RunPod's persistent volume features to store models downloaded via the Ollama CLI, preventing the need to re-download heavy model files on every worker initialization.

The storage strategy encompasses two primary methods: dynamic downloading to a mounted volume during runtime and preloading models directly into the Docker image layers for environments where volume mounting might not be used or for faster cold starts.

Sources: README.md:1-15, src/start.sh:35-41

Automated Model Persistence

The worker is configured to automatically utilize a mounted volume if one is present. This is handled by the underlying Ollama service which defaults its storage path to the standard system locations usually mapped in persistent storage configurations.

Runtime Model Initialization

When the worker starts, it checks for the presence of the OLLAMA_MODEL_NAME environment variable. If provided, the system attempts to pull the model using the ollama pull command. This process ensures that the requested model is available in the storage layer before the Python handler begins processing jobs.

The diagram above illustrates the startup logic for model acquisition.
Sources: src/start.sh:35-41, README.md:9-11

Model Preloading and Embedding

For scenarios requiring faster deployments without relying on runtime downloads, the project provides a mechanism to embed models directly into the container storage. This is achieved through a multi-stage process involving a dedicated preloading script and Docker build arguments.

Preloading Architecture

The preloading mechanism starts a temporary Ollama server instance during the image build phase to pull and register model files.

Preloading Sequence

The following sequence diagram describes how models are persisted into the image during the build process:

The sequence ensures that the models are fully downloaded and indexed in the internal storage before the build layer is committed.
Sources: embed_model/preload_model.sh:1-35, embed_model/README.md:3-3

Storage Configuration and Variables

Storage behavior is primarily controlled through environment variables that dictate which models are fetched or initialized.

Sources: README.md:13-17, src/engine.py:23-23

Interaction with Engine

The OllamaEngine class interacts with the storage layer indirectly by referencing the models available to the local Ollama API. It defaults to llama3.2:1b if no specific model is requested or configured, assuming this model exists in the local storage or volume.

# src/engine.py:23
model = os.getenv("OLLAMA_MODEL_NAME", "llama3.2:1b")

Sources: src/engine.py:21-23

Conclusion

The volume mounting and storage system in this worker prioritizes flexibility between persistent volume usage and static image embedding. By automating the ollama pull process in the startup script and providing a preloading utility, the architecture ensures that model weights are efficiently managed and available for the Ollama server to serve requests immediately upon worker activation.