Data Persistence

Data persistence is a core pillar of the Sovereign Communications platform, designed to ensure user data is stored locally on their device, enabling offline functionality and data sovereignty. The system employs a cross-platform strategy, using native database technologies for each target environment (Web, Android, and iOS) while sharing a common conceptual schema. This local-first approach means all user data, including identity keys, contacts, and messages, remains under the user's control and is not stored on any central servers.

The persistence layer is responsible for storing all application state required for the mesh network to function across sessions. This includes cryptographic identities, peer information, routing tables, session keys for forward secrecy, conversations, and messages. The integration of this layer is critical for features like the "Sneakernet" (offline data relay) and ensuring a seamless user experience where data is never lost on application restart.

Sources: V1_READINESS.md, V1_READINESS_REPORT.md, PERSISTENCE_INTEGRATION_GUIDE.md

High-Level Architecture

The persistence architecture is designed around a central @sc/core library that defines the data structures and interfaces, with each platform providing a concrete implementation using its native storage technology.

This diagram illustrates the architectural separation of concerns for data persistence.

• @sc/core: Defines the TypeScript interfaces for all data models (e.g., Identity, PersistedPeer, Message) that need to be stored.
• Platform Implementations: Each platform translates these interfaces into its native database schema.
  • Web: Uses IndexedDB, a browser-based transactional database.
  • Android: Uses Room, an abstraction layer over SQLite.
  • iOS: Uses Core Data, Apple's object graph and persistence framework.
• Application Logic: Both the UI layer and the background mesh networking services interact with the platform-specific database implementation to save and retrieve data.

Sources: V1_COMPLETE_TASK_LIST.md, PERSISTENCE_INTEGRATION_GUIDE.md, PLATFORM_PARITY_100_COMPLETE.md

Common Data Schema

While each platform has its own database technology, they all persist a common set of data entities essential for the network's operation.

The following Entity-Relationship diagram shows the main data models and their relationships.

Platform Implementations

Web (IndexedDB)

The web platform uses IndexedDB for its persistence layer, managed primarily by DatabaseManager in web/src/storage/database.ts.

Key Components:

• database.ts: Contains the DatabaseManager class, which handles database initialization, schema definition, and all CRUD operations.
• schema-validator.ts: Implements a migration system to handle database version upgrades gracefully.
• Object Stores: The database is structured into several object stores:
  • identities
  • contacts
  • conversations
  • messages
  • persistedPeers (named peers in some docs)
  • routes
  • sessionKeys
  • offlineQueue (for unsent messages)

Schema Versioning and Migration
The database includes a versioning system to manage schema changes over time. When the application starts, it checks the database version and runs any necessary migrations.

This flowchart shows the database initialization and migration process on the web platform.

Android (Room)

The Android application uses the Room Persistence Library, which provides an abstraction layer over SQLite.

Key Components:

• SCDatabase.kt: The main database class, annotated with @Database, which defines the entities and database version.
• Entities: Kotlin data classes annotated with @Entity that define the tables (e.g., IdentityEntity, PeerEntity, MessageEntity).
• DAOs (Data Access Objects): Interfaces annotated with @Dao that provide methods for interacting with the database (e.g., insert, query, delete).
• Repositories: Classes that abstract the data sources and provide a clean API for the ViewModels.
• KeystoreManager.kt: A crucial security component that stores cryptographic private keys in the Android Keystore, separate from the main database, to protect them.

This class diagram shows the relationship between the main components of the Android persistence layer.

iOS (Core Data)

The iOS application uses Apple's Core Data framework to manage its object graph and persistence.

Key Components:

• Model.xcdatamodeld: The data model file that visually defines entities, attributes, and relationships.
• CoreDataStack.swift: A singleton class that manages the Core Data stack, including the NSPersistentContainer and the NSManagedObjectContext.
• Managed Object Subclasses: Swift classes generated from the data model (e.g., MessageEntity, ContactEntity) that are used to interact with records in the database.
• Keychain Integration: Similar to Android's Keystore, private keys are stored securely in the system Keychain rather than directly in the Core Data database.

Sources: ios/SovereignCommunications/Data/CoreDataStack.swift, V1_COMPLETE_TASK_LIST.md:86-87

Data Sovereignty Features

A core tenet of the project is user data sovereignty. The persistence layer supports this through features that give users full control over their data.

This sequence diagram illustrates the flow for exporting and importing data between two devices.

Offline & Relay Persistence

To support asynchronous communication and the "Sneakernet" use case, the persistence layer includes an offline queue.

• OfflineQueue: Implemented in core/src/offline-queue.ts and backed by IndexedDB on the web, this queue stores messages that fail to send. It features an exponential backoff retry mechanism (1s, 2s, 4s, etc.) up to a maximum of 5 retries.
• MessageRelay Persistence: The core MessageRelay component, responsible for store-and-forward functionality, requires a PersistenceAdapter. This allows it to persist messages intended for offline peers, ensuring they are not lost if the application restarts. This is critical for enabling a user to physically carry messages from one location to another.

Sources: V1_PRODUCTION_IMPLEMENTATION_PLAN.md, V1_READINESS_REPORT.md, core/src/offline-queue.ts

Conclusion

The data persistence system in Sovereign Communications is a robust, multi-platform architecture designed with user control and offline capability at its core. By leveraging native database technologies, it achieves high performance and reliability on each platform. The clear separation between the core schema definition and platform-specific implementations allows for maintainability and consistency. Features like data export/import and a persistent offline queue directly support the project's goals of data sovereignty and resilient, decentralized communication. The successful integration of these persistence layers into the application logic is a critical step for the V1.0 release.

Sources: PERSISTENCE_INTEGRATION_GUIDE.md, V1_READINESS_REPORT.md