Evolution Engine & Self-Improvement

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• ROADMAP.md
• configs/agents/agents_config.json
• modules/evolution_engine/AGENTS.md
• modules/neurons/AGENTS.md
• monGARS/core/cortex/curiosity_engine.py
• scripts/generate_monGARS_llm_dataset.py
• monGARS/core/iris.py
• monGARS_structure.txt

The Evolution Engine is a core component of the monGARS project, designed to enable autonomous self-improvement. It orchestrates a continuous loop of diagnostics, data gathering, model retraining, and deployment. This system allows the AI assistant to adapt its capabilities, learn from new information, and optimize its performance over time, which is central to its function as an Autonomous Research & Support agent.

The self-improvement capabilities were a key milestone of "Phase 6 – Self-Improvement & Research" in the project's development, which was completed in Q1 2026. The system is designed to be modular, with graceful fallbacks for when heavyweight ML dependencies like PyTorch are unavailable.

Sources: README.md, ROADMAP.md, modules/AGENTS.md

Architecture and Core Concepts

The self-improvement process is an automated pipeline that begins with diagnostics and curiosity-driven data collection. This data is used to train new model adapters, which are then evaluated and deployed to enhance the system's conversational and reasoning abilities.

The following diagram illustrates the high-level workflow of the evolution and self-training loop.

Sources: README.md

Key Components

The self-improvement system is composed of several interconnected modules that handle different stages of the pipeline.

Component	Location	Description
EvolutionOrchestrator	modules/evolution_engine/	Sequences the self-improvement pipeline, including stages for validation, training, deployment, rollback, and sustainability checks. It is the central coordinator of the evolution process.
CuriosityEngine	monGARS/core/cortex/	Detects knowledge gaps in conversations by analyzing history and entities. It triggers external research to gather new, relevant information for learning.
MNTPTrainer	modules/neurons/training/	The "Modular Neuron Training Pipeline" trainer. It produces new model adapters (e.g., LoRA) from curated datasets, forming the core of the retraining cycle.
ResearchWorkflow	monGARS/core/cortex/curiosity_engine.py	A class within the CuriosityEngine that encapsulates the process of searching, verifying, and summarizing external information to fill knowledge gaps.
Iris	monGARS/core/	A utility for lightweight, resilient web content retrieval. It is used by the ResearchWorkflow to fetch documents from the web.

Sources: modules/evolution_engine/AGENTS.md, monGARS/core/cortex/curiosity_engine.py, README.md, monGARS/core/iris.py

Knowledge Acquisition and Dataset Generation

Before the system can improve, it must gather new knowledge and structure it into a format suitable for training. This is achieved through curiosity-driven web research and automated dataset generation from the codebase itself.

Curiosity-Driven Research

The CuriosityEngine is responsible for proactively identifying and filling knowledge gaps. It analyzes conversation context to determine if the system has sufficient knowledge to answer a query.

The process works as follows:

1. The engine analyzes the last query and compares it against conversation history using vector similarity to see if a similar topic has been discussed.
2. It extracts entities from the query and checks for their presence in a knowledge graph (Neo4j).
3. If a sufficient number of entities are missing from the knowledge graph, a "knowledge gap" is declared.
4. A new research query is formulated based on the original query and the missing entities.
5. The ResearchWorkflow is invoked to perform the research, which involves searching the web, verifying information across multiple sources, and generating a summary.
6. This new summary ("additional context") is used to enrich the model's understanding.

Sources: monGARS/core/cortex/curiosity_engine.py:278-323

The following diagram shows the sequence of operations within the CuriosityEngine when detecting a knowledge gap.

Sources: monGARS/core/cortex/curiosity_engine.py

Automated Dataset Generation

To fine-tune its own understanding of the repository, monGARS includes a script to automatically generate a high-quality dataset. The script scripts/generate_monGARS_llm_dataset.py parses the source code, OpenAPI specifications, and predefined scenarios to create prompt-response pairs for training.

This process involves:

• AST Parsing: Analyzing Python modules to understand their structure, functions, classes, and dependencies.
• OpenAPI Parsing: Ingesting the openapi.json file to document API endpoints.
• Scenario Processing: Using hardcoded scenarios and playbooks to generate examples for architecture, operations, and safety alignment.

The generated dataset is split into training and validation sets and is categorized to cover different aspects of the system.

Sources: scripts/generate_monGARS_llm_dataset.py

The table below summarizes the categories of data generated.

Category	Description
module_reference	Generates Q&A pairs about specific Python modules, covering their purpose, dependencies, async interfaces, and more.
architecture	Creates examples based on predefined ARCHITECTURE_SCENARIOS to explain system-level workflows and failure modes.
operations	Generates runbook-style Q&A from OPERATIONS_PLAYBOOKS for incident response.
alignment	Creates examples from ALIGNMENT_PLANS to guide the model on safety, persona, and telemetry goals.
api	Documents API endpoints based on openapi.json, explaining usage, payload structure, and testing strategies.

Sources: scripts/generate_monGARS_llm_dataset.py

Training and Deployment Pipeline

Once data is gathered, it feeds into a structured pipeline managed by the EvolutionOrchestrator.

Orchestration and Pipeline Discipline

The EvolutionOrchestrator is designed with a focus on clear sequencing and resilience. Its responsibilities include:

• Instantiating trainers from modules.neurons.training.
• Executing a pipeline with explicit stages: validation, training, deployment, rollback, and sustainability checks.
• Wrapping external calls with targeted exception handling to log actions and propagate errors correctly.
• Returning artifact metadata for catalog updates.

Sources: modules/evolution_engine/AGENTS.md

Configuration

The training process is driven by configuration files located in the configs/training/ directory. This allows operators to tune the behavior of the evolution pipeline without changing the code.

File	Location	Purpose
mntp_dolphin_config.json	configs/training/	Configuration for the MNTP (Modular Neuron Training Pipeline) trainer, likely for a specific model family like Dolphin.
sustainability_policy.json	configs/training/	Defines policies related to sustainability, which are likely used for gates or checks during the evolution pipeline.

These configurations can be overridden in tests to ensure deterministic behavior.

Sources: modules/evolution_engine/AGENTS.md, monGARS_structure.txt

Roadmap and Project Goals

The self-improvement features were the focus of Phase 6 – Self-Improvement & Research on the project roadmap. Key deliverables for this phase included:

• Self-training cycles that produce real adapter artifacts via MNTPTrainer.
• Reinforcement-learning research loops managed by the evolution orchestrator.
• Operator approval workflows and long-haul validation with telemetry.

Future work, planned for Phase 7 – Sustainability & Longevity, aims to fully integrate the outputs of the evolution engine into routine optimization cycles and automate energy usage reporting.

Sources: ROADMAP.md, AGENTS.md, modules/AGENTS.md

Summary

The Evolution Engine and its associated self-improvement systems are a cornerstone of the monGARS project. By combining curiosity-driven research, automated dataset generation, and a robust training and deployment pipeline, monGARS is able to autonomously enhance its own knowledge and capabilities. This makes it a truly adaptive and resilient AI assistant, capable of evolving to meet new challenges and support complex research tasks.

Sources: README.md