Citation & References
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The "Citation & References" system within the Chain-of-Ideas (CoI) Agent is a critical framework for grounding autonomous research in established scientific literature. It facilitates the discovery, parsing, and extraction of bibliographic data to ensure that generated ideas and experimental designs are novel and technically sound. By leveraging semantic search and LLM-based extraction, the system identifies relevant prior work, manages citations for grounding, and evaluates the relationship between proposed ideas and existing publications.
Sources: README.md:1-24, agents.py:100-110, prompts/deep_research_agent_prompts.py:75-85
Reference Extraction Architecture
The system employs a multi-step process to transform raw PDF content into structured reference data. This architecture relies on the SementicSearcher class and specialized prompts to identify citations and their corresponding metadata within a paper's context.
Extraction Logic
The system uses the get_deep_reference_prompt to guide the LLM in extracting the three most relevant references from a paper's content. This prompt mandates that references are formatted as titles only, excluding author names or publication years, to simplify downstream search queries.
Sources: prompts/deep_research_agent_prompts.py:75-125, agents.py:195-205
Data Structure for References
References are managed within the Result class and processed via semantic search fields.
| Field | Description |
|---|---|
title | The full title of the cited research paper. |
paperId | Unique identifier from Semantic Scholar. |
referenceCount | Total number of references cited by the paper. |
citationCount | Total number of times the paper has been cited. |
openAccessPdf | URL for accessing the full-text PDF if available. |
Sources: searcher/sementic_search.py:78-85, searcher/sementic_search.py:99-105
Semantic Search & Retrieval Flow
The CoI Agent utilizes the Semantic Scholar API to retrieve bibliographic data. This process involves both forward search (citations) and backward search (references) to build an "idea chain" that maps the historical progression of a research topic.
The diagram above shows how queries are transformed into structured research chains through iterative search and extraction.
Sources: agents.py:144-165, searcher/sementic_search.py:108-140
Reference Retrieval Functions
search_papers_async: Queries the Semantic Scholar graph for papers matching specific criteria likeminCitationCountandpublicationDate.search_related_paper_async: Specifically looks for "citations" (papers that cited the target) or "references" (papers cited by the target).read_paper_content_with_ref: Parses thearticledictionary to append a dedicated<References>section containingref_id,title, andyear.
Sources: searcher/sementic_search.py:108-135, searcher/sementic_search.py:228-255, searcher/sementic_search.py:328-338
Citation-Based Novelty Verification
A core function of the reference system is to prevent plagiarism and ensure novelty. The DeepResearchAgent uses extracted references to compare a proposed "final idea" against existing literature.
Novelty Check Process
The agent generates search queries based on the proposed idea, retrieves up to 5 related papers, and uses a specialized prompt (get_deep_check_idea_novel_prompt) to determine if the target idea is identical to any existing research.
The sequence shows the interaction between the agent and search modules to validate research originality.
Sources: agents.py:348-375, prompts/deep_research_agent_prompts.py:265-290
Configuration for Reference APIs
To function correctly, the system requires API keys for semantic search and LLM services, configured in config.yaml and loaded in main.py.
| Configuration Key | Purpose | Required |
|---|---|---|
SEMENTIC_SEARCH_API_KEY | Access to Semantic Scholar's Graph API for retrieval. | Yes |
EMBEDDING_API_ENDPOINT | Endpoint for generating vectors used in reference reranking. | Optional |
MAIN_LLM_MODEL | Model used for judging relevance and novelty (e.g., gpt-4o). | Yes |
Sources: README.md:43-70, main.py:11-20
Summary
The Citation & References system provides the backbone for the CoI Agent's research capabilities. By integrating SementicSearcher for data retrieval and specialized prompts for bibliography parsing, the system ensures that every generated research step is anchored in the existing scientific corpus, maintains an audit trail via "idea chains," and strictly enforces novelty through automated similarity assessments.
Sources: agents.py:300-315, searcher/sementic_search.py:328-340