Storage & Database Architecture
Relevant source files
The following files were used as context for generating this wiki page:
Archivum employs a local-first, multi-modal storage strategy designed for a self-hosted environment. The architecture centers on Markdown files as the canonical source of truth, supplemented by several derived indexes—SQLite for operational metadata, Qdrant for semantic vector search, and Kuzu (or Neo4j) for graph-based relationship mapping. This design ensures that knowledge compounds over time while remaining resilient; if derived indexes are corrupted, they can be entirely rebuilt from the raw Markdown content.
Sources: README.md:214-233, archivum-prd-v1.0.md:275-279
Multi-Layer Storage Overview
The system categorizes data into four primary storage layers, each serving a specific retrieval or operational need.
| Store | Technology | Purpose | Resilience |
|---|---|---|---|
| Canonical Content | Markdown (.md) | The primary source of truth for wiki pages. | Primary Source |
| Operational Metadata | SQLite | Stores users, auth state, page metadata, share links, and keyword FTS. | Rebuildable from MD |
| Vector Index | Qdrant | Facilitates semantic search via embeddings. | Rebuildable from MD |
| Graph Database | Kuzu / Neo4j | Maps entity relationships (references, mentions, etc.). | Rebuildable from MD |
Sources: README.md:214-225, archivum-prd-v1.0.md:275-277
Data Flow and Synchronization
When a new document is ingested or a page is edited, the system orchestrates a synchronized write across all storage layers.
The diagram shows the sequential and parallel update process triggered by content changes to ensure consistency across the derived indexes.
Sources: apps/backend/archivum/api/pages.py:228-251, archivum-prd-v1.0.md:79-85
Core Storage Components
Canonical Markdown Storage
Markdown files are stored in a dedicated volume (defaulted to wiki_data). These files include YAML frontmatter containing metadata such as titles, tags, and entity associations.
- Path:
/data/wiki - Identifier: Slugs generated from titles (e.g.,
getting-started.md).
Sources: README.md:216-217, apps/backend/archivum/api/pages.py:238-243
Relational & Meta-Data (SQLite)
SQLite handles the relational aspects of the application, including user management, session handling, and Life OS entities. It also provides Full-Text Search (FTS) capabilities.
- Key Features: Page summaries, authored_by tracking, share link tokens, and ingest logs.
- Multi-tenancy: A
wiki_idfield is present on all models from day one to support future multi-user partitioning.
Sources: README.md:218-219, archivum-prd-v1.0.md:280-282, Progress.md:24-25
Vector Search (Qdrant)
Qdrant stores vector embeddings of wiki pages. The system supports local embedding via fastembed or external providers like OpenAI and Anthropic.
- Endpoint:
http://localhost:6333 - Functionality: Used for the
search_wikiandqueryMCP tools to provide RAG-based synthesised answers with citations.
Sources: README.md:222-223, apps/backend/archivum/mcp/server.py:214-230
Graph Relations (Kuzu / Neo4j)
The knowledge graph maps how concepts relate across the wiki. While v1 PRD mentions Neo4j, the implementation and progress notes highlight Kuzu as a lighter embedded alternative for home servers.
- Node Types:
page,person,concept,organization. - Edge Types:
references,related-to,contradicts,mentions.
Sources: archivum-prd-v1.0.md:149-155, README.md:220-221, Progress.md:24-25
Database Schemas and Interaction
Page Detail Model
The backend API defines the structure for page persistence and retrieval.
class PageDetail(PageSummary):
id: int
slug: str
title: str
content: str
tags: list[str]
created_at: str
updated_at: str
authored_by: str
Sources: apps/backend/archivum/api/pages.py:38-46
Update Logic Sequence
Updating a page involves a strict sequence to maintain the integrity of the canonical file and its derived counterparts.
Index Maintenance
Because the indexes are derived, the system provides a specific administrative endpoint to force a synchronization between the Markdown files and the databases.
- Endpoint:
POST /api/rebuild-indexes - Action: Scans the
/data/wikidirectory, parses all Markdown files, and repopulates SQLite, Qdrant, and Kuzu/Neo4j.
Sources: README.md:386-391, archivum-prd-v1.0.md:275-279
Conclusion
The Storage & Database Architecture of Archivum balances the portability of flat Markdown files with the advanced query capabilities of modern vector and graph databases. By isolating the canonical source from its indexes, the system provides a robust platform for long-term knowledge accumulation that remains manageable on consumer-grade home server hardware.
Sources: archivum-prd-v1.0.md:38-43, Progress.md:15-18