Agent Orchestration

Relevant source files

The following files were used as context for generating this wiki page:

• agents.py
• main.py
• searcher/sementic_search.py
• prompts/deep_research_agent_prompts.py
• prompts/review_agent_prompts.py
• prompts/juder_prompts.py
• LLM.py

Agent Orchestration in the Chain-of-Ideas (CoI) project refers to the coordinated interaction between specialized autonomous agents designed to automate the scientific research lifecycle. This system primarily manages the flow between literature discovery, research idea generation, evolutionary refinement, and experimental design validation. Sources: agents.py, README.md

The orchestration logic is centralized in the DeepResearchAgent and ReviewAgent classes, which interact to form a recursive loop of proposal and critique. The system leverages Large Language Models (LLMs) to perform complex reasoning tasks, such as judging the novelty of an idea against existing literature or refining experimental steps based on peer-like feedback. Sources: agents.py:75-80, agents.py:125-130

Core Architecture and Agent Roles

The system is built on a multi-agent framework where different agents handle specific stages of the research process.

DeepResearchAgent

The DeepResearchAgent is the primary driver of the research pipeline. It handles:

• Chain Generation: Building a "Chain of Ideas" by traversing literature forwards (citations) and backwards (references). Sources: agents.py:240-270
• Idea Synthesis: Generating novel research motivations, methods, and contributions based on the analyzed chains. Sources: agents.py:365-375
• Experiment Design: Drafting detailed step-by-step procedures to validate generated ideas. Sources: agents.py:465-472

ReviewAgent

The ReviewAgent acts as a critical validator. It:

• Search Query Generation: Identifies related work to challenge the novelty of proposed ideas. Sources: agents.py:98-105
• Critique: Provides suggestions on experimental feasibility and clarity. Sources: agents.py:125-133

Orchestration Flow

The following diagram illustrates the high-level communication between these agents and the external search/LLM interfaces.

The orchestration starts in main.py, initializing the agents and executing the generate_idea_with_chain method to start the research cycle. Sources: main.py:40-55

Idea Generation and Literature Traversal

The orchestration utilizes a specific "chain" logic to contextualize new ideas within historical research trends.

Chain Construction Logic

The deep_research_paper_with_chain function orchestrates a bidirectional search:

1. Forward Search: It searches for future papers that cited the current "anchor" paper to see how the field evolved. Sources: agents.py:255-265
2. Backward Search: It parses the references of a paper to find foundational works. Sources: agents.py:277-290

Iterative Experiment Refinement

A key feature of the orchestration is the improvement loop where the ReviewAgent provides feedback that the DeepResearchAgent must incorporate into the final output.

Component	Function	File Reference
improve_experiment	Coordinates the loop between review and refinement.	agents.py:474
review_experiment	Generates suggestions using ReviewAgent.	agents.py:125
refine_experiment	Updates the plan based on suggestions.	agents.py:440
check_novel	Validates if the final idea is a duplicate of existing work.	agents.py:413

Refinement Workflow

The improve_experiment method runs for a configurable improve_cnt. In each iteration:

1. ReviewAgent analyzes the experiment against extracted entities. Sources: prompts/review_agent_prompts.py:60-70
2. DeepResearchAgent identifies if new information is needed via get_deep_refine_experiment_search_query_prompt. Sources: prompts/deep_research_agent_prompts.py:345
3. The agent extracts specific "info" from relevant papers to address the reviewer's concerns. Sources: agents.py:452-460

LLM and Search Integration

The agents rely on an abstracted LLM layer and a semantic searcher to perform their tasks.

• LLM Interface: The openai_llm class (in LLM.py) supports both OpenAI and Azure OpenAI endpoints, handling retries and embedding generation. Sources: LLM.py:65-80
• Search Interface: The SementicSearcher (in searcher/sementic_search.py) interacts with the Semantic Scholar API and uses scipdf to parse downloaded PDF content into structured dictionaries. Sources: searcher/sementic_search.py:90-110, searcher/sementic_search.py:280-295

Conclusion

Agent Orchestration in this project transforms a simple LLM prompt into a sophisticated research pipeline. By separating the roles of "Generator" (DeepResearchAgent) and "Critic" (ReviewAgent), the system emulates the academic peer-review process, ensuring that the final research ideas and experimental designs are grounded in literature and iteratively improved for technical feasibility. Sources: agents.py, main.py