Environment Variables

Environment variables in the Runpod Ollama worker serve as the primary mechanism for configuring the containerized environment, defining model selection, and managing worker concurrency. These variables are consumed by both shell scripts during the initialization phase and Python components during the inference lifecycle.

The system utilizes python-dotenv to load environment configurations, ensuring that variables defined in the environment or locally are accessible to the OllamaEngine and OllamaOpenAiEngine classes.
Sources: src/requirements.txt:2, src/engine.py:19

Core Configuration Variables

The following table summarizes the key environment variables used to configure the worker's behavior and model integration.

Sources: README.md:12, src/handler.py:6, src/engine.py:23, embed_model/preload_model.sh:4

Initialization and Model Management

The OLLAMA_MODEL_NAME variable is critical during the startup sequence. When the container initializes via src/start.sh, it checks for the presence of this variable. If set, the script triggers an ollama pull command to ensure the model is available locally before the Python handler starts.

The flow demonstrates how the initialization script uses environment variables to ensure local model availability.
Sources: src/start.sh:34-40

Build-Time Preloading

During the Docker build process, the MODEL_NAMES variable allows for multiple models to be embedded into the image. The preload_model.sh script parses this comma-separated string to iterate through and download each model.
Sources: embed_model/preload_model.sh:4-15

Runtime Engine Integration

Inside the Python application, environment variables are dynamically retrieved to configure the inference engines and worker scaling.

Model Selection Logic

The OllamaEngine class retrieves OLLAMA_MODEL_NAME at runtime. If the variable is not provided in the environment, it defaults to llama3.2:1b. This model name is then injected into the OpenAI-compatible job input for processing.
Sources: src/engine.py:23-28

Concurrency Control

The src/handler.py script manages worker scaling through the MAX_CONCURRENCY variable. This value is passed to the runpod.serverless.start function via a concurrency_modifier, directly influencing how many parallel requests the worker will accept.
Sources: src/handler.py:6-7, src/handler.py:26-32

This diagram illustrates the flow of environment variables into the main execution components of the worker.
Sources: src/handler.py:6-32, src/engine.py:23

Technical Implementation Details

The implementation relies on os.getenv for safe retrieval of these configurations, providing fallback values where appropriate to ensure system stability.

# From src/handler.py
DEFAULT_MAX_CONCURRENCY = 8
max_concurrency = int(os.getenv("MAX_CONCURRENCY", DEFAULT_MAX_CONCURRENCY))

# From src/engine.py
model = os.getenv("OLLAMA_MODEL_NAME", "llama3.2:1b")

Sources: src/handler.py:6-7, src/engine.py:23

Summary

Environment variables provide the necessary flexibility to deploy the Runpod Ollama worker across different hardware configurations and model requirements. By separating the model definition (OLLAMA_MODEL_NAME) and scaling logic (MAX_CONCURRENCY) from the core logic, the worker can be reconfigured without modifying the underlying source code.