Installation and Setup
Relevant source files
The following files were used as context for generating this wiki page:
- scripts/generate_knowledge_base.py
- server.js
- knowledge/papers/distributed-software-build-assurance-for-software-supply-chain-integrity.json
- knowledge/papers/a-data-driven-methodology-for-quality-aware-code-fixing.json
- knowledge/papers/devops-integration-with-capability-model-maturity-integration-a-systematic-mappi.json
CodeScholar is a structured research knowledge base and Model Context Protocol (MCP) server designed for AI agent consumption. The installation and setup process involves configuring a Node.js-based server and executing a multi-pass Python pipeline to transform raw academic paper metadata into a tiered architecture of Problems, Techniques, and Papers.
The system relies on a hybrid search architecture, combining standard text matching with vector-based semantic search. Setting up the environment requires installing specific Python dependencies for PDF processing and knowledge extraction, alongside Node.js dependencies for the MCP server interface.
Sources: scripts/generate_knowledge_base.py:1-25, server.js:1-15
Environment Requirements
The project requires both Python 3 and Node.js environments. The Python environment is primarily used for the generate_knowledge_base.py pipeline, while the Node.js environment hosts the MCP server.
Python Dependencies
The knowledge base generator requires several specialized libraries for data processing and AI interaction.
| Dependency | Purpose |
|---|---|
openai | Extracting structured knowledge using GPT-4o models |
pymupdf (fitz) | Extracting methodology and algorithm text from PDF files |
pathlib | Managing cross-platform file system paths |
concurrent.futures | Handling multi-threaded API calls for performance |
Sources: scripts/generate_knowledge_base.py:40-55, scripts/generate_knowledge_base.py:598-610
Node.js Dependencies
The server utilizes the Model Context Protocol SDK and vector search libraries.
| Dependency | Purpose |
|---|---|
@modelcontextprotocol/sdk | Implementing the MCP server and tool definitions |
vectra | Local vector index for semantic search capabilities |
fs/promises | Asynchronous file system operations for library loading |
Sources: server.js:15-25
Knowledge Base Generation Setup
The core of CodeScholar is the generated knowledge base. This is established by running a pipeline that processes a unified library of papers.
The Multi-Pass Pipeline
The setup involves four distinct passes to build the three-layer knowledge architecture.
The diagram shows the sequential transformation of raw data into structured AI-ready knowledge.
Sources: scripts/generate_knowledge_base.py:14-25, scripts/generate_knowledge_base.py:650-800
Execution Commands
To initialize the knowledge base, users run the generator script with various flags depending on the desired scope.
# Full run of the knowledge base generation
python scripts/generate_knowledge_base.py
# Single pass run (e.g., only Pass 1)
python scripts/generate_knowledge_base.py --pass 1
# Resume with existing cards
python scripts/generate_knowledge_base.py --skip-existing
Sources: scripts/generate_knowledge_base.py:27-32, scripts/generate_knowledge_base.py:623-640
Server Configuration
The CodeScholar MCP server must be configured to point to the generated knowledge library and the vector index.
Path Resolution and Indexing
Upon startup, the server loads configuration from config.json and builds internal indices for fast lookup.
The sequence diagram illustrates the initialization flow of the MCP server during setup.
Sources: server.js:27-45, server.js:68-85
Configuration Options
The server behavior is dictated by the following configuration parameters:
| Parameter | Type | Description |
|---|---|---|
config.library.path | String | Relative path to the unified library JSON file |
VECTRA_INDEX_PATH | String | Directory path for the Vectra vector storage |
OPENAI_API_KEY | Env Var | Required for real-time embedding generation during search |
Sources: server.js:27-35, server.js:605-610
Component Integration
Installation is considered complete when the knowledge base files are correctly placed in the knowledge/ directory, allowing the server to expose them via tools.
The ER diagram shows the relationship between the data structure created during setup and the server that provides access.
Sources: scripts/generate_knowledge_base.py:14-20, server.js:210-250
Setup Validation
To verify the installation, the server provides a get_library_stats tool that returns coverage metrics. A successful setup should report a non-zero count for total_papers and identify the search_mode as either text_only or hybrid_vectra (if the vector index was built).
Sources: server.js:180-210, server.js:460-470
The Installation and Setup process ensures that the academic research is not only stored but structured for high-performance retrieval by AI agents, moving from raw PDF/JSON metadata to a tiered, searchable knowledge architecture.