High-Level System Architecture
Relevant source files
The following files were used as context for generating this wiki page:
Archivum is a self-hosted, AI-powered knowledge base designed to run as a multi-service stack within a home network. The system functions as a "wiki that writes itself," utilizing an AI agent to incrementally build and maintain a persistent knowledge base by ingesting various source types (files, URLs), extracting structured entities and relationships, and generating markdown-based wiki pages. Unlike traditional RAG (Retrieval-Augmented Generation) tools that treat queries as transient, Archivum focuses on compounding knowledge over time by syncing data into specialized databases for semantic search and graph navigation.
The architecture is designed to be local-first, utilizing Docker Compose to orchestrate several interoperable services including a React-based frontend, a FastAPI backend, an MCP (Model Context Protocol) server, and multiple data storage engines. While it primarily supports an Anthropic-based LLM pipeline, it is architected for privacy and can be configured to run fully local LLMs via Ollama.
Sources: README.md:1-15, archivum-prd-v1.0.md:14-35, Progress.md:5-18
Core Service Components
The Archivum stack is composed of five primary containers communicating over a bridge network (wiki-net).
Service Table
| Service | Container Name | Technology | Responsibility |
|---|---|---|---|
| Frontend | archivum-frontend | React + Vite | Web-based Wiki UI and CodeMirror 6 editor. |
| Backend | archivum-backend | FastAPI (Python 3.12) | Core REST API, ingestion pipeline, and auth management. |
| MCP Server | archivum-mcp | Python MCP SDK | Exposes wiki tools to external AI clients (Claude, Cursor, etc.). |
| Vector DB | archivum-qdrant | Qdrant | Stores embeddings for semantic search. |
| Proxy | archivum-caddy | Caddy | Handles reverse proxying and automated TLS (HTTPS). |
Sources: README.md:195-207, archivum-prd-v1.0.md:380-388
System Interaction Diagram
This diagram illustrates how external clients and internal services interact through the primary API and MCP gateways.
Data Management & Storage
Archivum follows a "local-first" storage philosophy where Markdown files serve as the canonical source of truth, while other databases serve as derived indexes.
Storage Layers
- Canonical Store (Markdown): All wiki content is persisted as
.mdfiles with YAML frontmatter. These are the ultimate source of truth and allow the wiki to remain portable (e.g., compatible with Obsidian). - Metadata Store (SQLite): Maintains operational data such as users, auth state, page metadata, share links, and the ingest log. It also handles keyword Full-Text Search (FTS).
- Vector Store (Qdrant): Stores vector embeddings derived from the Markdown content to enable semantic natural language search.
- Graph Store (Kuzu/Neo4j): Stores entity relationships (e.g.,
related-to,mentions) extracted by the AI agent to power the visual knowledge graph.
Sources: README.md:209-224, archivum-prd-v1.0.md:392-396
Data Flow: Ingestion Pipeline
The ingestion pipeline transforms raw input into structured knowledge.
AI Ingestion & Synthesis
The core of Archivum's value is the WikiAgent, which orchestrates LLM interactions for two primary purposes: Extraction and Synthesis.
Wiki Agent (Extraction)
Located in apps/backend/archivum/ingest/agent.py, the agent uses a specialized system prompt to process documents into WikiPage objects, entities, and relationships. It supports multiple providers (Anthropic, OpenRouter, OpenAI, Ollama).
# Extraction Result Structure
@dataclass
class ExtractionResult:
pages: list[WikiPage]
entities: list[dict[str, str]] # {name, type}
relationships: list[dict[str, str]] # {from, to, type}
Sources: apps/backend/archivum/ingest/agent.py:73-82
Query Engine (Synthesis)
The query engine handles natural language questions by performing a semantic search in Qdrant to retrieve relevant context snippets, which are then fed into the LLM to generate a synthesized answer with citations linked to specific wiki pages.
Sources: apps/backend/archivum/mcp/server.py:221-275, archivum-prd-v1.0.md:223-233
MCP Server Implementation
The MCP server (archivum-mcp) provides a standardized interface for AI agents to interact with the knowledge base. It supports both stdio (for desktop applications) and SSE (for web-based or remote clients).
Available Tools
The server exposes several tools that wrap backend functionality:
| Tool | Implementation | Description |
|---|---|---|
ingest_source | ingest() | Processes a file/URL into the wiki. |
search_wiki | qdrant.search() | Returns semantic search results with excerpts. |
get_page | sqlite.get_page() | Retrieves full markdown content by slug. |
write_page | page_write_queue | Creates/updates a page and triggers re-indexing. |
query | Synthesis Logic | Asks a question and returns cited answers. |
graph_neighbors | graph.get_neighbors() | Returns connected nodes in the knowledge graph. |
life_daily_note | ensure_daily_note() | Manages "Life OS" daily notes. |
Sources: apps/backend/archivum/mcp/server.py:48-220, README.md:275-290
Security Architecture
Archivum is designed for secure remote access without direct port exposure.
- Tailscale Network: The primary method for authenticated remote access. It provides a VPN layer for the owner and invited guests to reach the stack on the home network.
- Cloudflare Tunnel: Used specifically for Share Links. It exposes a single read-only subdomain (e.g.,
share.archivumapp.com) that allows external users to view specific pages or query results without accessing the main application. - Token-based Auth: API and MCP access are protected via
MCP_API_KEYor JWT tokens. Mutating requests are protected by CSRF and markdown content is sanitized usingDOMPurify(client) andbleach(server) to prevent XSS.
Sources: archivum-prd-v1.0.md:410-445, apps/backend/archivum/api/pages.py:245-255
Conclusion
Archivum's architecture balances local data sovereignty with AI-driven automation. By decoupling the canonical storage (Markdown) from the optimized query indexes (Qdrant, Kuzu), the system remains resilient and extensible. The inclusion of a multi-transport MCP server ensures that the knowledge base is not just a static repository, but an active participant in the modern AI agent ecosystem.