Testing the Default Configuration

Testing the default configuration of the runpod-worker-ollama ensures that the worker correctly initializes its environment, pulls the necessary models, and processes incoming inference requests. The default configuration is designed to handle simple string prompts via the RunPod serverless interface, mapping them internally to the local Ollama API.

The scope of this testing covers the initial shell script orchestration, the engine selection logic within the Python handler, and the standard input/output formats expected by the system.

Initialization and Environment Setup

The worker's lifecycle begins with the execution of the start.sh script, which manages the Ollama server process and environment-based model preparation.

Server Orchestration

The system first ensures no stale processes are running before starting the Ollama server in the background. It monitors the server logs to confirm readiness before proceeding to model management.

The diagram shows the sequential startup process and model pulling logic.
Sources: src/start.sh:13-44, README.md:12-16

Configuration Variables

The behavior of the default configuration is primarily controlled via environment variables.

Sources: README.md:12-16, src/handler.py:7-8, src/engine.py:22

Default Request Processing Flow

When a job is received, the handler.py script determines which engine to use based on the input structure. For a default configuration test, the input is typically a simple JSON object containing a prompt.

Engine Selection Logic

The worker distinguishes between standard Ollama requests and OpenAI-compatible requests by checking for the presence of an openai_route in the job input.

The sequence diagram illustrates how a standard prompt is routed through the OllamaEngine.
Sources: src/handler.py:11-20, src/engine.py:20-51, src/utils.py:1-6

Data Mapping

In a default test scenario, the OllamaEngine automatically wraps the simple prompt into an OpenAI-compatible completion format before sending it to the local Ollama endpoint.

• Input Handling: The JobInput class extracts the prompt or messages from the job.
• Model Defaulting: If no model is specified in the environment, the engine defaults to llama3.2:1b.
• Route Mapping: Simple string prompts are mapped to /v1/completions, while list-based inputs (messages) are mapped to /v1/chat/completions.

Sources: src/utils.py:1-6, src/engine.py:22-45

Test Input Formats

The project provides specific JSON samples to validate the default configuration.

Standard RunPod Input

This is the most basic test case, ensuring the worker can handle a raw prompt.

{
    "input": {
        "prompt": "How are you?"
    }
}

Sources: test_inputs/default_runpodio.json:1-5, src/test_input.json:1-5

OpenAI-Style Completion (Optional Route)

Testing can also involve explicitly defining the OpenAI route within the input to bypass the default engine logic.

{
    "input": {
        "openai_route": "/v1/completions",
        "openai_input": {
            "model": "llama3.2:1b",
            "prompt": "How are you?",
            "max_tokens": 60
        }
    }
}

Sources: test_inputs/openai_completion.json:1-9

Conclusion

Testing the default configuration validates that the start.sh script correctly bootstraps the local Ollama environment and that handler.py successfully translates simple RunPod job inputs into API calls. By relying on the OllamaEngine's ability to default to llama3.2:1b and route string prompts to completion endpoints, the system ensures a low-friction entry point for basic inference tasks.