RAG and Search Orchestration

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• monGARS/core/cortex/curiosity_engine.py
• openapi.json
• openapi.lock.json
• scripts/generate_monGARS_llm_dataset.py
• AGENTS.md

The Retrieval-Augmented Generation (RAG) and Search Orchestration system in monGARS is designed to enrich conversational context with external information, grounding the AI's responses in verifiable data. This capability is central to the CuriosityEngine, which autonomously performs research to fill knowledge gaps, and is also exposed through a dedicated API for repository-aware code analysis.

The system is highly modular, integrating with external services like a document retrieval API and the SearxNG metasearch engine. Its behavior is controlled by several configuration flags, allowing operators to enable or disable features based on deployment needs. The core logic handles searching, cross-checking results for verification, and summarizing information to augment prompts.

Sources: README.md, monGARS/core/cortex/curiosity_engine.py

Architecture Overview

The RAG and search functionality is primarily driven by the CuriosityEngine and its ResearchWorkflow. There are two main pathways for triggering this system:

1. Curiosity-Driven Research: The CuriosityEngine detects a knowledge gap during a conversation and initiates a research workflow to find additional context.
2. Direct API Call: A user or tool can directly call the /api/v1/review/rag-context endpoint to get contextual information, typically for code-related queries.

In both cases, the SearchOrchestrator queries configured information sources. The results are then processed by a Verifier and summarized before being returned as enriched context.

The following diagram illustrates the high-level data flow for the RAG and search system.

Configuration

The RAG and search features can be configured using the following environment variables.

Setting	Description	Source
RAG_ENABLED	Toggles the repository-aware RAG enrichment feature.	README.md
RAG_SERVICE_URL	The URL for the dedicated RAG enrichment service.	README.md
RAG_REPO_LIST	A list of repositories to scope the RAG search.	README.md
RAG_MAX_RESULTS	Clamps the number of references returned by the RAG service.	README.md
DOCUMENT_RETRIEVAL_URL	The endpoint for the external research service invoked by the CuriosityEngine.	README.md
SEARCH_SEARX_ENABLED	When true, the docker_menu.py script automatically includes the docker-compose.searxng.yml service.	README.md
curiosity_similarity_threshold	The vector similarity score (0-1) required to consider a historical query similar to the current one.	monGARS/core/cortex/curiosity_engine.py:151
curiosity_graph_gap_cutoff	The number of missing entities in the knowledge graph required to trigger a research action.	monGARS/core/cortex/curiosity_engine.py:154

Sources: README.md, monGARS/core/cortex/curiosity_engine.py

Core Components

The search and enrichment logic is composed of several key classes within the monGARS.core module.

CuriosityEngine

The CuriosityEngine is the central component that orchestrates the detection of knowledge gaps and the subsequent research process.

• detect_gaps(conversation_context): This method is the primary entry point for autonomous research. It analyzes the last user query and conversation history to decide if the system has sufficient knowledge. If a knowledge gap is detected (based on historical similarity and entity presence in a knowledge graph), it formulates a research query and calls _perform_research.
• _perform_research(query): This internal method executes the research by running the ResearchWorkflow.
• build_prompt_context(question): This method provides a direct way to get enriched context for a given question by searching, verifying, and summarizing relevant documents.

Sources: monGARS/core/cortex/curiosity_engine.py:304-343, 228-299

ResearchWorkflow

This class, nested within curiosity_engine.py, encapsulates the entire multi-step process of performing external research. Its run method is the core of the workflow.

The sequence diagram below shows the interactions within the ResearchWorkflow.

This workflow ensures that information is retrieved, verified, and summarized from multiple sources before being used.

Sources: monGARS/core/cortex/curiosity_engine.py:848-1017

SearchOrchestrator

Although its source code is not provided in the context, the SearchOrchestrator is a critical dependency of the CuriosityEngine. Based on its usage, its primary responsibility is to abstract away the different search providers.

• search(query, lang, max_results): This method is called by CuriosityEngine and ResearchWorkflow to execute a search query. It is expected to return a list of NormalizedHit objects, which are provider-agnostic representations of search results.

Sources: monGARS/core/cortex/curiosity_engine.py:31, 234, 897

Verifier

The Verifier is used by the ResearchWorkflow to analyze and cross-check the search results provided by the SearchOrchestrator.

• cross_check(query, hits): This method takes the original query and a list of NormalizedHit objects. It returns a VerifiedBundle which contains:
  • agreed_facts: Information that multiple sources agree on.
  • disagreements: Conflicting information between sources.
  • citations: A list of source URLs.
  • confidence: A score representing the confidence in the aggregated information.

Sources: monGARS/core/cortex/curiosity_engine.py:32, 235, 900

API Endpoint: RAG Context Review

The system exposes a dedicated endpoint for performing repository-aware RAG. This is likely used for features like code review assistance.

Endpoint: POST /api/v1/review/rag-context

This endpoint returns contextual code references relevant to the supplied query. It is protected and requires an OAuth2PasswordBearer token.

Sources: openapi.json, README.md

Request Body (RagContextRequest)

Parameter	Type	Required	Description
query	string	Yes	The query to search for. Min length: 1, Max length: 4000.
repositories	array of strings	No	A list of repository names to limit the search.
max_results	integer	No	The maximum number of results to return. Min: 1, Max: 50.

Sources: openapi.json, openapi.lock.json

Response Body (RagContextResponse)

Field	Type	Description
enabled	boolean	Indicates if the RAG feature is enabled. Defaults to true.
focus_areas	array of strings	A list of focus areas identified from the context.
references	array of RagReferenceSchema	The list of contextual references found.

Sources: openapi.json, openapi.lock.json

Reference Schema (RagReferenceSchema)

Each reference in the response follows this structure.

Field	Type	Description
repository	string	The name of the repository where the reference was found.
file_path	string	The path to the file within the repository.
summary	string	A summary of the reference.
score	number	A relevance score for the reference.
url	string	A direct URL to the reference, if available.

Sources: openapi.json, openapi.lock.json

Conclusion

The RAG and Search Orchestration system is a cornerstone of monGARS's ability to provide intelligent and context-aware responses. By integrating a flexible search orchestrator, a robust verification workflow, and a curiosity-driven engine, it can dynamically pull information from external web pages, documents, and internal code repositories. This capability not only enriches the conversational AI but also powers developer-centric tools, making it a vital component for both autonomous research and interactive assistance.

Sources: README.md, monGARS/core/cortex/curiosity_engine.py, openapi.json