Dependency Management (requirements.txt)

Relevant source files

The following files were used as context for generating this wiki page:

• src/requirements.txt
• src/engine.py
• src/handler.py
• src/utils.py
• src/start.sh
• README.md

Introduction

Dependency management in the runpod-worker-ollama project is primarily handled through the src/requirements.txt file. This file specifies the Python libraries necessary to bridge the RunPod serverless environment with the Ollama inference engine. The dependencies support core functionalities such as serverless worker lifecycle management, environment configuration, OpenAI-compatible API communication, and high-performance JSON serialization.

The project operates as a wrapper around the Ollama server, utilizing these dependencies to process incoming jobs, interact with local LLM endpoints, and stream responses back to the RunPod platform.

Sources: src/requirements.txt, README.md:1-5, src/handler.py:1-10

Core Dependencies and Usage

The project relies on four primary packages to execute its logic. Each package serves a specific role in the architecture, from handling the serverless entry point to managing API client interactions.

Dependency Breakdown

Package	Version	Purpose in Project
runpod	Not specified	Provides the serverless worker SDK to start the handler and manage concurrency.
python-dotenv	Not specified	Used to load environment variables (e.g., OLLAMA_MODEL_NAME) into the application context.
openai	Not specified	Facilitates the OpenAI client used to communicate with the local Ollama API via OpenAI-compatible routes.
orjson	3.10.14	High-performance JSON library for efficient data serialization/deserialization.

Sources: src/requirements.txt:1-4, src/engine.py:4-14, src/handler.py:1-5

Dependency Integration Flow

The dependencies are integrated throughout the source code to manage the lifecycle of a request. The following diagram illustrates how these libraries are utilized from initialization to execution.

Implementation Details

Environment and API Client

The python-dotenv and openai libraries are critical for the OllamaEngine class. The engine uses load_dotenv() during initialization to ensure configurations like model names are available via os.getenv. The openai library provides the OpenAI client, which is configured to point to the local Ollama instance at http://localhost:11434/v1/.

from dotenv import load_dotenv
from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1/',
    api_key='ollama',
)

Sources: src/engine.py:7-14, src/engine.py:18-20

Worker Management

The runpod library is invoked in src/handler.py to bootstrap the serverless worker. It configures the handler function and sets the concurrency limits based on environment variables.

runpod.serverless.start(
    {
        "handler": handler,
        "concurrency_modifier": lambda x: max_concurrency,
        "return_aggregate_stream": True,
    }
)

Sources: src/handler.py:28-34, src/requirements.txt:1

Execution Environment

While Python dependencies are managed via requirements.txt, the project also depends on the Ollama binary being present in the environment. The src/start.sh script coordinates the startup of the Ollama server before the Python handler, which uses the installed dependencies, begins execution.

# Start the ollama server
ollama serve 2>&1 | tee ollama.server.log &
# ...
python -u handler.py $1

Sources: src/start.sh:17-38

Conclusion

Dependency management for runpod-worker-ollama is streamlined to include only essential libraries for API proxying and serverless integration. By pinning orjson and utilizing the standard openai and runpod SDKs, the project ensures a consistent interface between RunPod's infrastructure and the Ollama inference engine.