LLM Utilities
Relevant source files
The following files were used as context for generating this wiki page:
LLM Utilities in the DeepCode project encompass a sophisticated suite of prompt templates, memory management systems, and indexing tools designed to facilitate the reproduction of research papers into executable code. These utilities manage the entire lifecycle of an LLM interaction, from initial paper analysis and structural planning to iterative file generation and state persistence.
The system relies on a multi-agent architecture where specialized agents utilize specific prompt templates to perform distinct tasks, such as algorithm extraction, concept mapping, and code implementation. A core component of these utilities is the ConciseMemoryAgent, which optimizes token usage by maintaining only essential context throughout long-running development sessions.
Sources: prompts/code_prompts.py:1-25, workflows/agents/memory_agent_concise_index.py:12-30, workflows/code_implementation_workflow_index.py:15-30
Core Prompt Engineering Architecture
The project utilizes a structured hierarchy of prompt templates to guide LLMs through complex, multi-stage workflows. These templates are categorized by their role in the paper-to-code pipeline.
Paper Analysis and Planning Prompts
These prompts focus on extracting structured data from research papers to create a roadmap for implementation.
| Prompt Name | Purpose | Key Output Fields |
|---|---|---|
PAPER_INPUT_ANALYZER_PROMPT | Classifies input types (URL, PDF, text). | input_type, path, requirements |
PAPER_ALGORITHM_ANALYSIS_PROMPT | Extracts math formulas, pseudocode, and hyperparameters. | pseudocode, mathematical_formulation, training_loop |
PAPER_CONCEPT_ANALYSIS_PROMPT | Maps high-level paper sections to system components. | paper_structure_map, method_decomposition |
CODE_PLANNING_PROMPT | Synthesizes analysis into a file-by-file implementation plan. | file_structure, implementation_components |
Sources: prompts/code_prompts.py:32-55, prompts/code_prompts.py:133-145, prompts/code_prompts.py:270-285
Implementation and Execution Prompts
These prompts guide the agent during the actual coding and environment setup phases.
STRUCTURE_GENERATOR_PROMPT: Directs the LLM to generate shell commands (mkdir,touch) to initialize the project tree based on the plan.PURE_CODE_IMPLEMENTATION_SYSTEM_PROMPT: Defines the "best possible score" objective, emphasizing accuracy over elegance and mandating a dependency-aware implementation cycle.GENERAL_CODE_IMPLEMENTATION_SYSTEM_PROMPT: A variant for non-academic engineering tasks that mandates testing usingexecute_pythonorexecute_bashafter every file implementation.
Sources: prompts/code_prompts.py:465-480, prompts/code_prompts.py:643-660, prompts/code_prompts.py:716-730
Intelligent Memory Management
The ConciseMemoryAgent is a specialized utility that prevents token overflow and maintains role clarity during extended implementation workflows. It implements a "clean slate" approach to conversation history.
The Memory Optimization Cycle
The agent monitors tool calls and triggers a context refresh specifically after a write_file operation is detected.
The diagram shows the logic flow where the ConciseMemoryAgent detects file writes to trigger context pruning.
Sources: workflows/agents/memory_agent_concise_index.py:42-80, workflows/agents/memory_agent_concise_index.py:598-620
Knowledge Base Synthesis
The utility maintains an implement_code_summary.md file. After each file is written, the LLM generates a summary of implementation progress, functional dependencies, and recommended next steps. This summary is injected back into the LLM context as a "Knowledge Base" for the next round.
Sources: workflows/agents/memory_agent_concise_index.py:238-265, workflows/agents/memory_agent_concise_index.py:490-510
Repository Indexing and Retrieval
The CodeIndexer utility provides semantic awareness of existing codebases to provide reference implementations for new components.
Indexing Workflow
The indexer performs recursive traversal and uses LLM-powered similarity analysis to map external code repositories to the current project's target structure.
The sequence diagram illustrates the automated process of scanning a code repository to generate a relationship index.
Sources: tools/code_indexer.py:330-360, tools/code_indexer.py:550-580
Indexing Metadata and Configuration
The indexer tracks various metrics to ensure high-quality retrieval, including confidence scores and relationship types.
| Configuration Field | Default Value | Description |
|---|---|---|
min_confidence_score | 0.3 | Minimum threshold for accepting a file relationship. |
max_file_size | 1MB | Prevents reading extremely large files into LLM context. |
supported_extensions | .py, .js, .ts, etc. | List of code files the indexer is permitted to analyze. |
max_concurrent_files | 5 | Parallelism limit for concurrent LLM analysis. |
Sources: tools/code_indexer.py:100-135, tools/code_indexer.py:165-180
Implementation Workflow Integration
The LLM Utilities are orchestrated through CodeImplementationWorkflow. This system manages the transition between planning and implementation by initializing the ConciseMemoryAgent and CodeImplementationAgent.
- Infrastructure Initialization: Loads API secrets and configures the preferred LLM (Anthropic or OpenAI).
- Structure Generation: Executes the
STRUCTURE_GENERATOR_PROMPTvia a dedicated agent. - Implementation Loop: Iteratively calls the LLM, handles tool execution (MCP-compliant), and applies memory optimization via the
ConciseMemoryAgent. - Reporting: Generates a final completion report detailing total operations, files created, and agent performance statistics.
Sources: workflows/code_implementation_workflow_index.py:45-70, workflows/code_implementation_workflow_index.py:230-260, workflows/code_implementation_workflow_index.py:435-460
Conclusion
LLM Utilities in DeepCode provide a robust framework for managing the high-dimensional complexity of code reproduction tasks. By combining structured prompting with automated indexing and concise memory management, the system ensures that LLMs remain focused, accurate, and within token limits while building complex software systems from research specifications.