Search & AI Query Synthesis
Relevant source files
The following files were used as context for generating this wiki page:
Archivum features a dual-layer retrieval system that combines semantic search with LLM-powered query synthesis. Unlike traditional RAG (Retrieval-Augmented Generation) tools that treat every interaction as a clean slate, Archivum leverages a persistent knowledge base to compound information over time. The system uses Qdrant for vector-based semantic search and Neo4j (or Kuzu) for multi-hop graph traversal to provide contextually rich, cited answers to natural language questions.
Sources: archivum-prd-v1.0.md:1-20, archivum-prd-v1.0.md:376-381
Architectural Overview
The search and query synthesis architecture is built on a "local-first" philosophy, utilizing specialized databases for different retrieval types. The backend, powered by FastAPI, coordinates between the vector store and the graph database to satisfy user requests.
Data Flow for AI Query Synthesis
When a user submits a natural language question, the system executes a multi-stage pipeline:
- Semantic Retrieval: The query is embedded and searched against the Qdrant vector database to find relevant markdown chunks.
- Context Assembly: Top-k results are retrieved, and their associated wiki pages are identified via SQLite.
- LLM Synthesis: The original question and the retrieved snippets are sent to a configured LLM provider (Anthropic, OpenRouter, or Ollama) for synthesis.
- Citation Mapping: The generated response includes citations linked directly to specific wiki pages.
The query synthesis pipeline combines semantic hits from Qdrant with factual content from the markdown store to generate accurate responses.
Sources: archivum-prd-v1.0.md:376-383, apps/backend/archivum/mcp/server.py:190-230
Semantic Search Implementation
Semantic search allows users to find information using natural language even without exact keyword matches. This is primarily handled by the search_wiki tool and the /api/search endpoint.
Components and Latency
| Component | Technology | Target Latency |
|---|---|---|
| Vector Store | Qdrant | < 1 second |
| Synthesis Engine | LLM (e.g., Claude) | < 3 seconds (First Token) |
Sources: archivum-prd-v1.0.md:417-422, apps/backend/archivum/mcp/server.py:59-65
MCP Tool: search_wiki
The system exposes a Model Context Protocol (MCP) tool that allows AI agents to perform searches. It accepts a query string and a top_k parameter to limit results.
@mcp.tool()
async def search_wiki(query: str, top_k: int = 5, wiki_id: str = "default") -> list[dict[str, Any]]:
"""Semantic search; returns top-k with excerpts."""
return await qdrant.search(query, wiki_id=wiki_id, limit=top_k, settings=settings)
Sources: apps/backend/archivum/mcp/server.py:59-65
AI Query Synthesis Logic
The query tool is the core of the synthesis system. It retrieves content snippets, formats them into a prompt for the LLM, and enforces a strict "answer only from context" rule.
Synthesis Providers
Archivum supports multiple LLM providers for synthesis, configured via environment variables:
- Anthropic: Uses the Anthropic Python SDK with models like
claude-sonnet-4-6. - OpenRouter: Provides access to a variety of models via a unified API.
- OpenAI Compatible: Supports local options like Ollama for fully air-gapped operation.
Sources: archivum-prd-v1.0.md:830-845, apps/backend/archivum/mcp/server.py:231-255
Sequence of Operation
The following diagram illustrates the interaction between the MCP server and the various database components during a query synthesis request.
The query process involves deduplicating hits by slug and limiting citations to the top 8 relevant pages.
Sources: apps/backend/archivum/mcp/server.py:190-230
User Interface and Interaction
The frontend provides specialized components for interacting with the search and query systems.
Query Panel
Users can ask questions through a dedicated UI. A "Save as page" feature allows users to persist valuable AI-generated syntheses back into the wiki as new markdown pages, ensuring that knowledge continues to compound.
Sources: archivum-prd-v1.0.md:307-316
Notes Interaction Panel
The NotesInteractionPanel provides a unified interface for searching and performing quick actions on results, such as opening, editing, or copying links.
| Action | Logic |
|---|---|
| Search | Triggers the runSearch function using the /api/search endpoint. |
| Quick Edit | Opens a dialog to modify page titles and tags without leaving the search view. |
| Open | Navigates the user to the specific wiki page slug. |
Sources: apps/frontend/src/components/NotesInteractionPanel.tsx:32-45, apps/frontend/src/components/NotesInteractionPanel.tsx:112-140
API Reference
The following endpoints facilitate search and query synthesis via the REST API and CLI.
| Endpoint | Method | Description |
|---|---|---|
/api/search | GET | Executes a semantic search; takes a q parameter. |
/api/query | POST | Submits a natural language question for AI synthesis. |
/api/rebuild-indexes | POST | Regenerates vector and graph indexes from canonical markdown files. |
Sources: packages/archivum-cli/src/api.js:31-38, apps/backend/archivum/api/pages.py:461-470
Conclusion
The Search & AI Query Synthesis system is the primary mechanism for knowledge retrieval in Archivum. By combining vector-based semantic search with graph-aware context and LLM synthesis, it transforms a collection of static markdown files into a dynamic, queryable knowledge base. The integration of "Save as page" functionality ensures that the results of AI analysis are not lost but are instead fed back into the wiki to support future queries.