Utility Functions (utils.py)
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The utils.py file serves as a foundational component of the RunPod Worker Ollama project, primarily providing data structures to standardize job inputs. Its main purpose is to encapsulate raw input data from RunPod jobs into a structured object that can be easily consumed by the execution engines.
By providing a unified interface for input parameters, utils.py allows the OllamaEngine and OllamaOpenAiEngine to handle both standard Ollama requests and OpenAI-compatible requests through a single data model. This abstraction is critical for the worker's ability to process different request formats seamlessly.
Sources: src/utils.py:1-6, src/handler.py:9-14
The JobInput Class
The core of the utility module is the JobInput class. This class acts as a data wrapper that parses the input dictionary provided by the RunPod serverless environment. It extracts specific fields required for LLM processing and OpenAI routing.
Data Structure and Mapping
The JobInput class maps various input keys to internal attributes, ensuring that the rest of the application has a consistent way to access request parameters.
| Attribute | Data Source (Input Key) | Description |
|---|---|---|
llm_input | messages or prompt | The primary text or message list for the LLM. |
stream | stream | Boolean flag indicating if the response should be streamed. Defaults to False. |
openai_route | openai_route | The specific OpenAI API endpoint being targeted (e.g., /v1/chat/completions). |
openai_input | openai_input | The nested dictionary containing specific OpenAI parameters. |
Sources: src/utils.py:2-6
Implementation Logic
The constructor of JobInput uses the .get() method to safely access dictionary keys, providing a fallback for the llm_input by checking for messages first and prompt second.
class JobInput:
def __init__(self, job):
self.llm_input = job.get("messages", job.get("prompt"))
self.stream = job.get("stream", False)
self.openai_route = job.get("openai_route")
self.openai_input = job.get("openai_input")
Sources: src/utils.py:1-6
Data Flow Integration
The JobInput object is instantiated within the main job handler and then passed into the engine logic. The following diagram illustrates how the utility class facilitates the flow of data from the initial request to the specific engine implementation.
The diagram shows the lifecycle of a job input, starting as a raw dictionary and being transformed into a JobInput object before determining the execution path.
Sources: src/handler.py:9-14, src/engine.py:22-40
Usage in Routing and Engines
The attributes extracted by JobInput are used to determine which engine to instantiate and how that engine should format its internal calls to the Ollama backend.
Engine Selection
In handler.py, the openai_route attribute is checked to decide which engine class to use. If openai_route is present, OllamaOpenAiEngine is selected; otherwise, the base OllamaEngine is used.
Sources: src/handler.py:12-13
Request Formatting
In engine.py, the llm_input and stream attributes are used to build the final payload sent to the OpenAI-compatible Ollama API.
This sequence highlights the interaction between the handler, the utility object, and the engine during the generation process.
Sources: src/engine.py:22-38, src/handler.py:12-15
Summary
The utility functions in src/utils.py provide a streamlined way to manage job data within the RunPod worker. By centralizing the parsing logic in the JobInput class, the project maintains a clean separation between raw input handling and core execution logic, enabling flexible support for various API formats including standard prompts and OpenAI-compatible chat completions.