Philosophy and Use Cases
Relevant source files
The following files were used as context for generating this wiki page:Cyberpath Sentinel is a document-based Database Management System (DBMS) written in Rust. Its core philosophy diverges from modern databases that prioritize speed. Instead, Sentinel prioritizes trust, transparency, and compliance. It is designed as a "transparency layer" for the modern data stack, where all data is stored as files on disk, tables are represented by folders, and each document's primary key is its filename. This approach makes every piece of data inspectable, auditable, and compliant by design.
The project's vision is to become the "gold standard for transparent, auditable data storage" and to be synonymous with "Compliance as Code" and "Security by Design." It is not intended to replace high-throughput databases like PostgreSQL or MongoDB but to serve specific use cases where accountability and verifiability are paramount.
Sources: README.md:13-15, README.md:210-216, IMPLEMENTATION_PLAN.md:16-19
Core Philosophy
Sentinel is built on a set of fundamental design principles that guide its architecture and development.
Sources: README.md:120-125
This diagram shows the core design principles of Sentinel and their implications.
Sources: README.md:120-125
1. Filesystem is the Database
Sentinel leverages the reliability and mature tooling of the underlying operating system's filesystem. Tables are folders, and documents are files (e.g., JSON/BSON). This allows standard UNIX tools like grep, rsync, and tar to be used for queries, replication, and backups, respectively. This simplifies operations significantly, as there is no database daemon to manage.
Sources: README.md:13-14, README.md:52-55, README.md:121
2. Immutability by Default
To ensure a clear and verifiable history, operations are designed to be immutable. Deletions are soft, moving files to a .deleted/ directory, and updates can be handled through versioning. This creates append-only patterns, perfect for audit logs.
Sources: README.md:122, README.md:92, README.md:151
3. Security First
Security is a foundational concept. Sentinel relies on OS-level Access Control Lists (ACLs) for permissions, eliminating complex database user management. It also plans for file-level encryption, cryptographic hashing for integrity, and digital signatures to verify who changed what.
Sources: README.md:46, README.md:123, IMPLEMENTATION_PLAN.md:275-288
4. UNIX Philosophy
The system is designed to "do one thing well" and compose with other standard tools. This principle promotes simplicity and modularity, allowing users to integrate Sentinel into existing workflows with familiar utilities.
Sources: README.md:124, README.md:215
5. Zero External Dependencies
Sentinel is engineered to work on any device with a filesystem, from edge devices to servers, without requiring a network connection or external services. This makes it ideal for disconnected systems and offline-first applications.
Sources: README.md:34, README.md:125
Use Cases
The unique philosophy of Sentinel makes it a perfect fit for certain applications while being unsuitable for others.
Ideal Use Cases
Sentinel excels in scenarios where auditability, security, and compliance are the primary concerns.
| Use Case | Description |
|---|---|
| Audit Logs | Every entry is a file, which can be versioned with Git to track all changes. |
| Certificate Management | Certificates are stored as secure, inspectable files with OS-level ACLs. |
| Compliance Rules & Policies | A GDPR "right-to-delete" request is as simple as rm file, with a full audit trail. |
| Encryption Key Management | Keys are stored as files, protected by filesystem security. |
| Regulatory Reporting | All data is stored in a plain, forensic-friendly format. |
| Edge & Disconnected Systems | Works without a server and can be synchronized using Git. |
| Zero-Trust Infrastructure | Every piece of data can be inspected and verified before being trusted. |
Sources: README.md:23-37
Unsuitable Use Cases
The design trade-offs mean Sentinel is not a good fit for systems that require high throughput or low-latency transactional processing.
- Real-time bidding systems
- High-frequency trading platforms
- Streaming analytics pipelines
- Multi-million row transactional systems (at least, not yet)
Sources: README.md:40-44
Key Advantages
Auditability & Security
Every document is a file that can be inspected with standard tools like cat. When integrated with Git, it provides a transparent version history, showing who changed what, when, and why. Security is managed through the operating system's file permissions.
Sources: README.md:48-51, IMPLEMENTATION_PLAN.md:22-23
Operational Simplicity
Sentinel eliminates the need for a database daemon. Standard filesystem tools handle common database tasks.
- Replication:
rsync - Backups:
tar - Queries:
grep - Disaster Recovery:
git clone
Sources: README.md:52-57, IMPLEMENTATION_PLAN.md:24
Compliance-Ready
The architecture is inherently suited for regulatory compliance.
This diagram illustrates how a GDPR deletion request is handled, creating a verifiable audit trail.
Sources: README.md:60-61, README.md:151, IMPLEMENTATION_PLAN.md:403-406
The system is designed to satisfy requirements for SOC 2, ISO 27001, and HIPAA.
Sources: README.md:70-75, IMPLEMENTATION_PLAN.md:321-326
Zero Lock-In
Data is stored in standard JSON or BSON formats, not proprietary binary formats. This ensures that users can easily migrate their data to other systems like PostgreSQL, MongoDB, or DuckDB using standard tools.
Sources: README.md:78-82, IMPLEMENTATION_PLAN.md:25
Acknowledged Trade-offs and Solutions
Sentinel is transparent about its limitations and has a clear plan to address them.
| Pain Point | Description | Proposed Solution |
|---|---|---|
| Concurrency Complexity | Multi-writer scenarios can be complex and are not optimized for thousands of concurrent writes. | Implement file locking, Write-Ahead Logging (WAL), and eventually Multi-Version Concurrency Control (MVCC). |
| Query Performance | Initial queries require scanning files, as there are no native B-tree indices. | Introduce in-memory caching, lazy indexing (hash-based), and sharding. |
| Scaling Limits | A single folder's performance can degrade with millions of files (e.g., ~4M). | Use hash-based sharding, hierarchical folders, and distributed stores. |
| Partial Write Safety | A power failure during a file write could lead to data corruption. | Implement Write-Ahead Logging, atomic rename patterns, and checksums for verification. |
Sources: README.md:88-105, IMPLEMENTATION_PLAN.md:434-498
Data Storage Model
The "filesystem is the database" principle is reflected in a simple and intuitive folder structure.
This diagram shows how a Sentinel store is organized on the filesystem, with collections as folders and documents as files.
Sources: README.md:108-118, IMPLEMENTATION_PLAN.md:45-66