Local Graph Persistence
Relevant source files
The following files were used as context for generating this wiki page:
Local Graph Persistence in Arbor refers to the system responsible for storing the Code Property Graph (CPG) on disk to enable incremental synchronization and session persistence. This system ensures that the analyzed state of a codebase—including nodes, edges, and metadata—can be recovered across tool restarts without requiring a full re-index of the entire project.
The persistence layer is primarily implemented in the arbor-graph crate using the sled embedded database, providing ACID-compliant storage. It utilizes binary serialization via bincode for high-performance state saving and manages a versioned cache to prevent data corruption during schema updates.
Sources: crates/arbor-graph/src/store.rs:1-10, PR_DESCRIPTION.md:20-24, crates/arbor-graph/README.md:16-21
Persistence Architecture
The persistence architecture revolves around the GraphStore struct, which wraps a sled::Db instance. The system uses a key-value approach to index different facets of the graph, such as file modification times (mtimes), file-to-node mappings, and the raw binary data of the code nodes themselves.
Storage Key Schema
The GraphStore uses specific prefixes to organize data within the key-value store:
| Prefix | Description | Data Type |
|---|---|---|
meta: | Metadata such as the cache version | String |
m: | Mapping of file paths to their last modification time (mtime) | u64 |
f: | Mapping of file paths to a list of contained Node IDs | Vec<String> |
n: | Mapping of a specific Node ID to its serialized CodeNode data | CodeNode |
Sources: crates/arbor-graph/src/store.rs:70-130, PR_DESCRIPTION.md:22-23
Incremental Sync Logic
Arbor implements an incremental synchronization mechanism that only writes "dirty" nodes to disk. When a file is modified, the system removes the stale node entries associated with that file and inserts the updated ones within an atomic sled::Batch operation.
The diagram shows the atomic sequence of events during a file update in the persistence layer.
Sources: crates/arbor-graph/src/store.rs:98-132, PR_DESCRIPTION.md:22
GraphStore Lifecycle and Initialization
The GraphStore is initialized through the .arbor/ directory within a project. It validates the cache version upon opening to ensure compatibility.
Key Methods
open(path): Opens the database and checks themeta:version. If the version (e.g.,arbor-1.3) does not match, it returns aVersionMismatcherror.open_or_reset(path): Opens the database but automatically clears existing data if a version mismatch is detected, ensuring the tool starts with a fresh, compatible cache.load_graph(): Scans all keys with then:prefix to reconstruct theArborGraphin memory using aGraphBuilder. This process re-links edges and recreates the global symbol table.
Sources: crates/arbor-graph/src/store.rs:30-65, crates/arbor-graph/src/store.rs:164-188, crates/arbor-cli/src/commands.rs:14-25
Integration with CLI and Indexing
The CLI utilizes the persistence layer during the index command. By default, Arbor looks for a cache in .arbor/cache.
The sequence diagram illustrates how the CLI and Watcher use the GraphStore to skip re-parsing files that have not changed on disk.
Sources: crates/arbor-cli/src/commands.rs:56-75, crates/arbor-graph/src/store.rs:72-85, CHANGELOG.md:58-60
Data Integrity and Serialization
Arbor employs binary serialization for efficiency.
- Bincode: Used as the serialization format for all data stored in
sled, chosen for its high performance and compact binary representation. - Atomic Batches: The use of
sled::Batchensures that file updates are ACID compliant. If an update fails, the index (thef:andn:keys) will not be left in a partial or corrupted state. - Cache Versioning: A hardcoded
CACHE_VERSION(currentlyarbor-1.3) is checked during every store opening to prevent the application from reading data that might have a different binary layout than the current software version.
Sources: crates/arbor-graph/src/store.rs:7-15, crates/arbor-graph/src/store.rs:128-132, PR_DESCRIPTION.md:23-24
Summary of Store Operations
The following table summarizes the primary operations available for managing the local graph state:
| Operation | Function | Description |
|---|---|---|
| Get Mtime | get_mtime | Checks if a file needs to be re-parsed. |
| Get File Nodes | get_file_nodes | Retrieves cached nodes for a specific file. |
| Update File | update_file | Atomically replaces node data and updates mtime. |
| Remove File | remove_file | Cleans up nodes and indices for deleted files. |
| Load Graph | load_graph | Performs a full scan of n: entries to build the in-memory graph. |
| List Files | list_cached_files | Scans the f: prefix to return all currently tracked file paths. |
Sources: crates/arbor-graph/src/store.rs:72-188
Local Graph Persistence is a critical component that enables Arbor to handle large codebases efficiently. By leveraging sled and bincode, the system achieves sub-second synchronization for incremental changes, providing the foundation for real-time impact analysis and visual feedback in the "Logic Forest" visualizer.