Troubleshooting

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• main.py
• agents.py
• LLM.py
• searcher/sementic_search.py
• prompts/deep_research_agent_prompts.py

This page provides technical guidance for diagnosing and resolving issues within the Chain-of-Ideas (CoI) Agent system. The system relies on a complex interplay between Large Language Models (LLMs), semantic search APIs, and PDF parsing tools. Troubleshooting primarily focuses on configuration validation, API connectivity, and the recovery logic implemented within the research agents.

The architecture employs a multi-agent approach where DeepResearchAgent and ReviewAgent coordinate to generate and refine research ideas. Failures typically manifest in LLM response timeouts, search query misses, or environment misconfigurations.

LLM and API Connectivity

The system supports both OpenAI and Azure OpenAI providers. Issues often arise from missing environment variables or incorrect API versions. The openai_llm class handles the initialization and includes retry logic for robust communication.

Configuration Validation

The system loads configurations from config.yaml and exports them to environment variables. Missing keys in this file will lead to runtime exceptions during LLM or Searcher initialization.

Configuration Key	Purpose	Required
SEMENTIC_SEARCH_API_KEY	Access to Semantic Scholar API	Yes
MAIN_LLM_MODEL	Primary model for complex reasoning (e.g., GPT-4o)	Yes
CHEAP_LLM_MODEL	Secondary model for summarization tasks	Yes
AZURE_OPENAI_KEY	Authentication for Azure endpoints	If is_azure: True
OPENAI_API_KEY	Authentication for OpenAI endpoints	If is_azure: False

Sources: README.md:46-73, LLM.py:48-83, main.py:12-19

Retry Mechanism

To handle transient network issues or rate limits (HTTP 429), the openai_llm class utilizes the tenacity library. It is configured to wait 10 seconds between attempts and stop after 10 failed attempts.

Search and Research Failures

The DeepResearchAgent and SementicSearcher include logic to handle "empty results" from the Semantic Scholar API by rewriting queries or falling back to different search strategies.

Query Rewriting Logic

If a literature search fails to return papers, the DeepResearchAgent enters a loop (up to 10 iterations) where it asks the LLM to rewrite the query based on failed attempts and the original topic.

Semantic Scholar API Errors

The SementicSearcher class specifically monitors for HTTP 429 (Too Many Requests). When encountered, it waits 5 seconds before retrying the search automatically.

Sources: searcher/sementic_search.py:112-115

Dependency and Environment Issues

Failures in PDF parsing or local tool execution usually stem from missing system-level dependencies.

Common Environment Issues

• Java Missing: Required for Grobid (PDF parsing). Users must ensure JAVA_HOME is set to JDK 11.
• Grobid Service: The scipdf_parser requires a running Grobid instance. If main.py fails to extract text from PDFs, verify Grobid is accessible via bash serve_grobid.sh.
• Model Availability: The MAIN_LLM_MODEL and CHEAP_LLM_MODEL variables must be explicitly set. If they are empty strings, the system raises a ValueError.

Sources: README.md:28-44, agents.py:21-28, searcher/sementic_search.py:228-235

Data Extraction and Parsing

The system relies on specific XML-style tags (e.g., <idea>, <entities>) to parse LLM outputs.

Extraction Failures

If the LLM fails to wrap content in the expected tags, the extract utility function may return an empty string or the raw text, which can cause downstream json.loads errors in the DeepResearchAgent.

Tag Type	Source File	Usage
<queries>	deep_research_agent_prompts.py	Initial search queries
<references>	deep_research_agent_prompts.py	Paper citations for chain building
<similar>	deep_research_agent_prompts.py	Novelty check results (0/1)
<experiment>	deep_research_agent_prompts.py	Generated experimental steps

Sources: utils.py, agents.py:133-145, searcher/sementic_search.py:17-27, prompts/deep_research_agent_prompts.py:34-43

Conclusion

Troubleshooting the CoI Agent requires verifying the API status of both the LLM provider and Semantic Scholar. Most logic-level failures are handled by internal retry loops and query refinement processes. However, persistent failures usually indicate an exhausted token budget, incorrect API keys in config.yaml, or the absence of the Grobid service for PDF parsing.