Image Generation
Relevant source files
The following files were used as context for generating this wiki page:
The Image Generation system in Onyx allows users and AI agents to create visual content directly within the platform. It is implemented as a modular tool within the Gen-AI and Enterprise Search architecture, enabling integration with multiple image generation providers such as OpenAI's DALL-E. This feature is managed through an administrative interface and can be invoked by agents during complex tasks like deep research.
Sources: AGENTS.md:144-148, web/src/app/admin/configuration/image-generation/page.tsx, backend/onyx/tools/tool_implementations/images/image_generation_tool.py
Architecture and Tool Integration
Image generation is primarily exposed as a "Tool" within the Onyx ecosystem. This tool-based approach allows the AI to autonomously decide when a user's request warrants the creation of an image.
Image Generation Tool
The ImageGenerationTool is the core implementation that bridges the agent's request with the backend generation logic. It defines the parameters required for generation, such as the prompt, and handles the execution of the generation request through the configured provider.
The diagram above shows the flow from a user query to the final image output via the specialized tool.
Sources: backend/onyx/tools/tool_implementations/images/image_generation_tool.py, backend/onyx/image_gen/factory.py
Provider Factory
The system uses a factory pattern to instantiate the appropriate image generation client. This abstraction allows Onyx to support different backends (e.g., DALL-E 3) based on environment variables or database configurations.
| Component | Description |
|---|---|
| ImageGenerationTool | The implementation of the tool interface used by agents. |
| get_image_generator | Factory function to retrieve the configured generator instance. |
| DalleImageGenerator | A specific provider implementation for OpenAI's DALL-E models. |
Sources: backend/onyx/image_gen/factory.py, backend/onyx/tools/tool_implementations/images/image_generation_tool.py
Configuration and Management
Administrators configure image generation through the web interface. This setup is crucial for ensuring the backend has the correct credentials and model selections.
Admin Interface
The image generation settings page allows administrators to:
- Enable or disable the image generation feature globally.
- Select the preferred provider and model.
- Configure API keys and other provider-specific settings.
Sources: web/src/app/admin/configuration/image-generation/page.tsx
Environment and Database Settings
The backend relies on configuration objects, often retrieved from the database, to determine which provider to use. These settings include model names, quality levels, and endpoint configurations.
# Conceptual logic from factory.py
def get_image_generator(db_session: Session):
# Retrieve configuration from DB
# Instantiate specific provider (e.g. DalleImageGenerator)
pass
Sources: backend/onyx/image_gen/factory.py:10-25
Usage in Research and Agents
Image generation is not a standalone feature but is integrated into higher-level reasoning flows. Agents like the orchestrator for Deep Research can utilize tools to enhance reports with visual information.
Agentic Execution
When an agent is processing a query, it analyzes the available tools. If the task involves visualization, it invokes the ImageGenerationTool with a descriptive prompt derived from the context.
The sequence diagram illustrates how a research agent communicates with the image generation tool to fulfill a request.
Sources: backend/onyx/prompts/deep_research/orchestration_layer.py, backend/onyx/tools/tool_implementations/images/image_generation_tool.py
Summary
The Image Generation module in Onyx provides a flexible, provider-agnostic framework for creating images via AI. By utilizing a factory pattern and standard tool interfaces, Onyx ensures that visual content can be seamlessly integrated into chat responses and research reports, managed through a centralized administrative dashboard.