Multi-Strategy Retrieval Layer
Relevant source files
The following files were used as context for generating this wiki page:
The Multi-Strategy Retrieval Layer is the core orchestration component for evidence gathering within the agentic medical diagnosis system. It provides a layered architecture that abstracts diverse search methodologies—including vector, hybrid, and graph-based approaches—into a unified SearchEngine. This layer is designed to fan out queries across multiple specialized retrievers based on "recipes" and fuse the results using techniques like Reciprocal Rank Fusion (RRF) or cross-encoders.
The primary goal of this layer is to provide the Agentic RAG Loop with a flexible, data-driven interface for navigating clinical knowledge stored in Neo4j (graph) and Qdrant/Weaviate (vector). By utilizing specialized retrievers for entities, relations, chunks, and communities, the system can answer both atomic clinical questions and broad thematic inquiries.
Sources: README.md:143-146, am_diag/retrieval/search.py
Architecture and Components
The retrieval layer is structured as an orchestration engine that manages several pluggable retrievers. The SearchEngine class acts as the central coordinator, delegating work to specific retriever implementations based on a provided SearchConfig or a pre-defined string recipe.
Core Search Workflow
The following diagram illustrates the flow of a search request through the Multi-Strategy Retrieval Layer:
The search engine concurrently executes selected retrieval methods and merges their outputs into a unified list of ranked SearchResult objects.
Sources: am_diag/retrieval/search.py, README.md:162-171
Specialized Retrievers
| Retriever | Methodology | Target Data |
|---|---|---|
| Entity Retriever | Hybrid (Vector + BM25) + optional BFS expansion | Entity nodes and their names/aliases |
| Relation Retriever | Hybrid search + Graph hydration | RELATES_TO edges and SPO triples |
| Chunk Retriever | Hybrid search | Raw text passages/documents |
| Community Retriever | Semantic search over summaries | CommunityReport thematic summaries |
| Text-to-Cypher | LLM Translation with EXPLAIN gate | Structured graph queries against schema |
| Sources: README.md:147-156, am_diag/retrieval/retrievers/text2cypher.py:27-35 |
Retrieval Recipes
Recipes are data-only constants that define which search methods to run and how to fuse them. This allows the system to change retrieval strategies without code modifications in the agent loop.
| Recipe Name | Methods Included | Fusion/Reranker |
|---|---|---|
entity | Entity search only | RRF |
hybrid_rrf | Entity, Relation, Chunk, Community | Reciprocal Rank Fusion |
hybrid_cross_encoder | All four + BFS expansion | Cross-Encoder |
text2cypher | LLM-generated Cypher query | RRF |
bfs_expand | Entity + Breadth-First Search | RRF |
| Sources: README.md:162-171, tests/am_diag/retrieval/test_search.py:21-36 |
Recipe Resolution Logic
The _resolve_recipe function maps a string identifier to a SearchConfig. If a custom SearchConfig object is passed directly to the search method, it overrides the standard recipes.
Sources: am_diag/retrieval/search.py, tests/am_diag/retrieval/test_search.py:21-36
Text-to-Cypher Implementation
The Text2CypherRetriever utilizes a Large Language Model (LLM) via BAML to translate natural language into Cypher queries. It implements a robust "EXPLAIN gate" to ensure query validity before execution.
The retriever injects a textual representation of the GraphSchema into the prompt, describing the generic :RELATES_TO {type} pattern used by the system.
Sources: am_diag/retrieval/retrievers/text2cypher.py:53-93, am_diag/common/data_models/schema.py:100-117
Schema-Driven Retrieval
Retrieval is tightly coupled with the GraphSchema, which defines valid EntityType and RelationType objects. The schema is used to:
- Prompt Injection: Generate textual descriptions of nodes and relationships for the Text-to-Cypher LLM.
- Validation: Ensure generated queries adhere to allowed head/tail label pairs.
- Entity Resolution: Drive the deduplication and canonicalization of entity names, which improves search precision.
Knowledge Graph Schema Model
class GraphSchema(BaseModel):
entity_types: list[EntityType]
relation_types: list[RelationType]
def as_text(self) -> str:
# Generates: (:Entity:Disease {id, name, ...})
# (h)-[:RELATES_TO {type: 'TREATED_BY'}]->(t)
...
Sources: am_diag/common/data_models/schema.py:38-117
Agent Tool Integration
The Multi-Strategy Retrieval Layer is exposed to the Agentic RAG Loop through a set of specialized tools. These tools wrap the SearchEngine.search method with specific pre-configured recipes.
The make_retrieval_tools function generates the following tools:
entity_search: Targeted entity lookup.relation_search: Fact-based triple retrieval.chunk_search: Contextual passage retrieval.community_search: High-level thematic retrieval.hybrid_search: Comprehensive multi-method retrieval.cypher_search: Direct graph querying.global_map_reduce: Community-level synthesis tool.
Sources: tests/am_diag/agents/test_tools.py:31-45
Summary
The Multi-Strategy Retrieval Layer provides the necessary infrastructure to transform a clinical question into a comprehensive set of evidence. By combining vector-based semantic search with structured graph traversal and LLM-driven query generation, it ensures that diagnostic reasoning is grounded in both low-level facts (entities and relations) and high-level summaries (community reports). The use of data-driven recipes and a unified SearchEngine interface allows for rapid iteration of retrieval strategies without impacting the core agentic reasoning logic.
Sources: README.md:143-171, am_diag/retrieval/search.py