Client-Side Persistence Layer

Relevant source files

The following files were used as context for generating this wiki page:

• client/persistence/persistence.ts
• client/persistence/persistence.type.ts
• client/persistence/dexie/dexie.local.ts
• client/persistence/surreal/surreal.local.ts
• client/persistence/rxdb/rxdb.local.ts

The client-side persistence layer in Nucleus is responsible for managing all data interactions on the client. This includes both communication with the remote backend API for account management, data synchronization, and other services, as well as local data storage within the user's browser. The layer is designed to be flexible, providing a unified interface for key-value storage while supporting multiple underlying database technologies like Dexie (IndexedDB), RxDB, and SurrealDB for local persistence.

This dual responsibility ensures that the application can function in an offline-first manner, manage user session data, and interact with remote services through a consistent and centralized set of modules. The Persistence class handles all remote API calls, while a set of classes implementing the IClientStorage interface manages local browser storage.

Sources: client/persistence/persistence.ts, client/persistence/persistence.type.ts

Architecture Overview

The persistence layer is composed of two main parts: a remote persistence handler and a local storage abstraction.

1. Remote Persistence (Persistence class): A singleton class that encapsulates all fetch calls to the backend API. It handles tasks like authentication, user plans, file uploads, and retrieving web data.
2. Local Storage (IClientStorage interface): An interface that defines a standard contract for key-value storage. This allows the application to use different local database implementations interchangeably.

The following diagram illustrates the high-level architecture of the client-side persistence layer.

Remote Persistence: The Persistence Class

The Persistence class, exposed as a singleton persistenceInstance, is the sole entry point for all client-to-server communication. It uses the performApiCall utility to make authenticated requests to various backend endpoints.

Sources: client/persistence/persistence.ts:13-255

Key Methods

The class provides methods for various domains, including account management, subscriptions, file handling, and utility actions.

Method	Description	API Endpoint
refreshToken	Refreshes the user's authentication token.	account/refreshToken
getUserInfo	Fetches user information using a token.	account/n/refresh
ping	Pings the server to check for a valid session.	v2/account/ping
getUserPlan	Retrieves the current user's subscription plan.	v2/plan/get
initiateSubscription	Starts a new subscription process.	v2/plan/subscribe
getSignedUrl	Gets a pre-signed URL from the server for file uploads.	utils/n/getsignedurl
uploadFile	Uploads a file blob to a given pre-signed URL.	(External S3/R2 URL)
retrieveUrlData	Asks the server to scrape and parse data from a given URL.	utils/n/run (get-webpage)
browseUnsplash	Searches for images on Unsplash via the backend.	utils/n/run (unsplash-browse)

Sources: client/persistence/persistence.ts:14-253

File Upload Flow

The process of uploading a file involves two steps: first obtaining a signed URL from the backend, and then using that URL to upload the file directly to the file storage service.

This sequence diagram shows the file upload process.

Local Storage

Local storage is managed through the IClientStorage interface, which provides a simple abstraction for a key-value store. This allows for multiple underlying implementations that can be swapped out as needed.

Sources: client/persistence/persistence.type.ts:12-17

IClientStorage Interface

This interface defines the fundamental operations required for any local storage provider.

// Source: client/persistence/persistence.type.ts:12-17
export interface IClientStorage {
  get: (key: ClientStorageKey) => Promise<any>;
  set: (key: ClientStorageKey, value: any) => Promise<any>;
  remove: (key: ClientStorageKey) => Promise<any>;
  clear: () => Promise<any>;
}

ClientStorageKey Enum

All keys used for local storage are defined in the ClientStorageKey enum to prevent typos and ensure consistency.

Key	Description
PRODUCT	The current product being used (e.g., nucleus).
USER_INFO	Cached user information object.
TOKEN	The user's authentication JWT.
REFRESH_TOKEN	The token used to refresh the authentication JWT.
SETTINGS	User-specific application settings.
APP_DATA	Static application data fetched from the server.
LAST_SYNCED_AT	Timestamp of the last successful data sync.
LAST_SYNC_STATUS	The status of the last sync operation.
LAST_SYNC_ERROR	Any error message from the last failed sync.
LAST_SYNC_DURATION	The duration of the last sync operation in ms.

Sources: client/persistence/persistence.type.ts:1-11

Implementations

The repository includes three different implementations of the IClientStorage interface.

1. Dexie (DexieClientStorage)

This implementation uses Dexie.js, a wrapper around IndexedDB. It creates a database named 21n with a single table store for key-value pairs. This is a robust choice for persistent client-side storage.

• Database Name: 21n
• Table: store
• Schema: { key: string, value: any } ( key is the primary key)

Sources: client/persistence/dexie/dexie.local.ts:4-31

2. SurrealDB (SurrealClientStorage)

This implementation uses an in-memory instance of SurrealDB. It connects to a 'memory' endpoint and uses a namespace and database both named 21n. Data is stored in a store table. Since it's an in-memory database, data will not persist across page reloads.

• Endpoint: memory
• Namespace/Database: 21n
• Table: store

Sources: client/persistence/surreal/surreal.local.ts:4-44

3. RxDB (RxDBClientStorage)

This implementation uses RxDB, a reactive database for JavaScript applications. It uses the Dexie-based RxStorage adapter, so it also relies on IndexedDB. It creates a database named 21ndb with a collection named store.

• Database Name: 21ndb
• Collection: store
• Schema: { key: { primary: true, type: 'string' }, value: { type: 'string' } }

Sources: client/persistence/rxdb/rxdb.local.ts:5-47

Summary

The client-side persistence layer is a critical component that isolates data management logic from the rest of the application. Its Persistence class provides a clean, promise-based API for all backend interactions. Simultaneously, its abstracted local storage mechanism, with multiple concrete implementations (Dexie, SurrealDB, RxDB), offers flexibility in how and where client-side data is stored, supporting both persistent and in-memory scenarios. This architecture promotes separation of concerns and makes the application more maintainable and adaptable to future changes in storage technology.