Customizing Prompts
Relevant source files
The following files were used as context for generating this wiki page:
Customizing prompts is a central mechanism in the Chain-of-Ideas (CoI) Agent framework, enabling specialized LLM-driven agents to perform complex research tasks. The project utilizes a modular prompt engineering approach where distinct prompt templates are defined for different stages of the research lifecycle, including literature search, idea generation, experiment design, and peer review. These prompts are structured to enforce specific output formats (typically XML-like tags or JSON) to facilitate automated parsing and iterative refinement by the system's agents.
Sources: prompts/deep_research_agent_prompts.py, agents.py:1-20
Prompt Architecture and Management
The prompt system is organized into specialized modules within the prompts/ directory. Each module contains functions that dynamically generate prompt strings based on input parameters such as research topics, paper content, or existing ideas. This architecture allows the DeepResearchAgent and ReviewAgent to retrieve context-specific instructions during execution.
Sources: prompts/init.py:1-4, agents.py:90-120
Key Prompt Modules
| Module | Purpose | Key Functions |
|---|---|---|
deep_research_agent_prompts.py | Orchestrates the core research flow, from initial search to final idea generation. | get_deep_search_query_prompt, get_deep_generate_idea_prompt |
review_agent_prompts.py | Provides templates for critiquing ideas and experiments. | get_review_suggestions_from_papers_prompt, get_review_experiment_design_suggestions_prompt |
juder_prompts.py | Defines criteria for evaluating and comparing different research ideas or experiments. | get_judge_idea_all_prompt, get_judge_experiment_all_prompt |
Sources: prompts/deep_research_agent_prompts.py:1-35, prompts/review_agent_prompts.py:1-10, prompts/juder_prompts.py:3-10
Research Flow and Prompt Interaction
The research process is a multi-stage pipeline where the output of one prompt often serves as the input for the next. This creates a "chain of ideas" where prompts guide the LLM through literature synthesis, trend analysis, and novel hypothesis formulation.
Data Flow Diagram
The following diagram illustrates how prompts are utilized by the DeepResearchAgent to move from a topic to a final research idea.
Specialized Prompt Categories
1. Literature Search and Retrieval
These prompts focus on transforming broad topics or failed search attempts into optimized queries for semantic search engines. The system includes logic for rewriting queries if no results are found.
- Functions:
get_deep_search_query_prompt,get_deep_rewrite_query_prompt. - Constraints: Strictly formatted output (e.g.,
<queries>["q1", "q2"]</queries>).
Sources: prompts/deep_research_agent_prompts.py:1-100
2. Synthesis and Trend Analysis
Prompts in this category task the LLM with acting as a "scientific research expert" to extract structured information from paper content and analyze the historical evolution of a field.
- Functions:
get_deep_reference_prompt,get_deep_trend_idea_chains_prompt. - Structure: Extracts Background, Novelty, Contribution, Detail Reason, and Limitation.
Sources: prompts/deep_research_agent_prompts.py:110-210
3. Evaluation and Judging
The juder_prompts.py module contains rigorous evaluation criteria including Novelty, Significance, Feasibility, Clarity, and Effectiveness. These are used by the agents to compare multiple generated ideas.
- Functions:
get_judge_idea_all_prompt. - Output: Thinking process followed by specific tags like
<novelty>{choice}</novelty>.
Sources: prompts/juder_prompts.py:3-45
Formatting and Constraint Enforcement
A critical aspect of customizing prompts in CoI-Agent is the use of strict output formatting. Most prompt functions append an output_format string to ensure the LLM returns data that the extract() utility can parse.
# Example of format enforcement in deep_research_agent_prompts.py
output_format = """
Output strictly in the following format:
<queries>["query1", "query2", ...]</queries>
"""
Sources: prompts/deep_research_agent_prompts.py:38-42, agents.py:10-15
Iterative Refinement Prompts
The system supports experiment refinement through the ReviewAgent. The get_deep_refine_experiment_prompt function incorporates feedback and suggestions into the prompt to generate an improved version of the experimental design.
Conclusion
Customizing prompts in the CoI-Agent repository is more than simple string templating; it is a structured approach to defining agent roles and data protocols. By separating prompts into functional modules and enforcing strict XML-based schemas, the system enables complex, multi-step autonomous research workflows that remain robust and parsable.