Installation & Requirements
Relevant source files
The following files were used as context for generating this wiki page:
The Chain-of-Ideas (CoI) Agent is a multi-agent system designed to automate scientific research workflows, specifically focused on generating and refining novel research ideas. The installation process involves setting up a Python environment, configuring LLM API access (supporting both OpenAI and Azure OpenAI), and installing specialized PDF parsing tools and Java-based dependencies for document processing.
Sources: README.md:1-20, main.py:10-18
System Requirements
The CoI Agent requires several software components to handle large language model (LLM) interactions, semantic searches, and scientific document parsing.
Core Dependencies
The project relies on a variety of Python libraries for asynchronous operations, data processing, and NLP.
| Category | Packages |
|---|---|
| Asynchronous/HTTP | aiohttp, httpx, tenacity |
| Data/Math | numpy, pandas, PyYAML |
| NLP/Parsing | spacy, lxml, beautifulsoup4, textstat |
| API Clients | openai |
Sources: requirements.txt:1-16
External Tools
- Python 3.x: Required for running the agentic framework.
- Java JDK 11: Specifically required to run GROBID, which parses PDF structures.
- GROBID: Used by the
scipdf_parserto extract structured data from research papers.
Sources: README.md:41-55, README.md:92-105
Installation Steps
The installation is structured into four primary stages: environment setup, parser installation, Java environment configuration, and LLM configuration.
Step 1: Clone and Environment Setup
First, clone the repository and install the standard Python requirements.
git clone https://github.com/DAMO-NLP-SG/CoI-Agent.git
cd CoI-Agent
pip install -r requirements.txt
Sources: README.md:25-29
Step 2: Scientific PDF Parser Setup
The agent uses scipdf_parser to convert PDF content into machine-readable formats.
git clone https://github.com/titipata/scipdf_parser.git
pip install git+https://github.com/titipata/scipdf_parser
python -m spacy download en_core_web_sm
Sources: README.md:32-37
Step 3: Java and GROBID Configuration
GROBID is essential for processing scientific literature. If not already available, JDK 11 must be installed and the JAVA_HOME path must be exported.
wget https://download.oracle.com/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz
tar -zxvf openjdk-11.0.2_linux-x64_bin.tar.gz
export JAVA_HOME=Your_path/jdk-11.0.2
After setting up Java, GROBID can be started using the script provided in scipdf_parser:
cd scipdf_parser
bash serve_grobid.sh
Sources: README.md:41-45, README.md:92-95
Configuration
The system uses a config.yaml file to manage environment variables for API keys and model selection. The application loads these values into os.environ at runtime.
API Configuration Table
| Variable | Description | Requirement |
|---|---|---|
SEMENTIC_SEARCH_API_KEY | Key for Semantic Scholar API | Mandatory |
is_azure | Boolean to toggle between OpenAI and Azure | Mandatory |
AZURE_OPENAI_KEY | API Key for Azure services | Required if is_azure: True |
OPENAI_API_KEY | API Key for OpenAI services | Required if is_azure: False |
MAIN_LLM_MODEL | The primary model (e.g., gpt-4o) | Mandatory |
EMBEDDING_MODEL | Model used for vector embeddings | Optional (defaults to text-embedding-3-large) |
Sources: README.md:58-85, LLM.py:126-140, agents.py:16-21
Data Flow for Configuration
The following diagram illustrates how the system initializes its configuration and LLM clients.
The system prioritizes environment variables set in config.yaml. If MAIN_LLM_MODEL or CHEAP_LLM_MODEL are missing, the system will raise a ValueError.
Sources: main.py:10-18, agents.py:15-25, LLM.py:65-103
Runtime Verification
To ensure the installation and requirements are met, the user should be able to run a basic idea generation task. This verifies that the LLM API, GROBID service, and Python environment are correctly integrated.
python main.py --topic "Your research topic"
The system will then initialize the DeepResearchAgent and ReviewAgent, which require the MAIN_LLM_MODEL to be functional.
Sources: main.py:34-45, README.md:108-111
Summary
Successful deployment of the CoI Agent requires a dual-layered setup: a robust Python 3 environment with specific NLP libraries and a functional Java environment for GROBID-based PDF parsing. Configuration is centralized in config.yaml, supporting flexible LLM provider integration. Correct configuration of the Semantic Scholar API is critical for the agent's ability to perform the literature reviews necessary for the Chain-of-Ideas process.