LLM Interface (LLM.py)

Relevant source files

The following files were used as context for generating this wiki page:

• LLM.py
• config.yaml
• agents.py
• main.py
• searcher/sementic_search.py

Introduction

The LLM.py module serves as the primary abstraction layer for interacting with Large Language Models (LLMs) within the Chain-of-Ideas (CoI) Agent framework. It provides a unified interface for both synchronous and asynchronous communication with OpenAI and Azure OpenAI services, facilitating tasks such as text generation, embedding extraction, and similarity calculations.

This module is central to the project's multi-agent architecture, supporting the specialized logic in agents.py and the semantic search capabilities in searcher/sementic_search.py. By encapsulating API configurations and retry logic, it ensures robust communication with model endpoints.

Sources: LLM.py:1-40, agents.py:12-25

Architecture and Components

Class Hierarchy

The interface is built upon an abstract base class base_llm, which defines the necessary contract for language model interactions. The concrete implementation, openai_llm, handles the complexities of environment configuration, client initialization for different providers (OpenAI vs. Azure), and error handling.

The diagram shows the inheritance structure where openai_llm implements the abstract interface defined by base_llm.
Sources: LLM.py:46-64

Initialization Logic

The openai_llm class dynamically configures itself based on environment variables defined in config.yaml and loaded via main.py. It supports a dual-provider system:

1. Azure OpenAI: Requires endpoint, API key, and optionally an API version.
2. OpenAI: Requires an API key and optionally a base URL or proxy URL.

Sources: LLM.py:65-103, config.yaml:3-13, main.py:12-18

Core Functionality

Text Generation (Chat Completions)

The module provides both synchronous (response) and asynchronous (response_async) methods to generate text. These methods wrap the OpenAI chat.completions API and include extensive retry logic using the tenacity library.

Method	Type	Description
response	Synchronous	Blocks execution until the LLM returns a response. Used for sequential operations.
response_async	Asynchronous	Non-blocking call used heavily by DeepResearchAgent for parallelizing paper analysis.

Sources: LLM.py:114-131, LLM.py:186-210, agents.py:120-125

Embeddings and Similarity

The interface supports generating vector embeddings for text, which are critical for the semantic searcher to rerank papers and calculate document relevance.

The flow shows how text is converted to vectors via embedding methods and subsequently compared using cosine similarity.
Sources: LLM.py:133-184, searcher/sementic_search.py:116-128

Configuration and Environment

The LLM interface relies on a config.yaml file to determine its behavior and credentials. This configuration is injected into the environment during the application's startup.

Configuration Parameters

Parameter	Type	Default	Description
is_azure	Boolean	True	Determines if the Azure OpenAI backend should be used.
AZURE_OPENAI_ENDPOINT	String	-	The base URL for the Azure OpenAI resource.
MAIN_LLM_MODEL	String	"gpt-4o"	The primary model used for complex reasoning tasks.
CHEAP_LLM_MODEL	String	"gpt-4o-mini"	A secondary, lower-cost model for summarization or simple tasks.
EMBEDDING_MODEL	String	"text-embedding-3-large"	The model used for generating semantic vectors.

Sources: config.yaml:1-26, agents.py:16-25, LLM.py:145

Robustness Features

Retry Mechanism

To handle transient API failures, the module utilizes the tenacity library. Both completion and embedding requests are decorated with retry logic that performs up to 10 attempts with a 10-second fixed wait between retries.

@retry(wait=wait_fixed(10), stop=stop_after_attempt(10), before=before_retry_fn)
def response(self,messages,**kwargs):
    # API call implementation...

Sources: LLM.py:114-115, LLM.py:28-30

Error Handling and Logging

The before_retry_fn function logs retry attempts, allowing developers to monitor the stability of the LLM provider connections. If a request ultimately fails after all retries, the error is caught and logged to prevent system crashes.

Sources: LLM.py:32-34, LLM.py:123-128

Conclusion

The LLM.py module is the backbone of the CoI Agent's cognitive capabilities. By providing a resilient, provider-agnostic interface for chat and embedding APIs, it enables the higher-level agents to focus on research logic, idea generation, and experimental design without managing low-level API concerns.