Data Retention & Deletion

Relevant source files

The following files were used as context for generating this wiki page:

• services/api/src/retention.ts
• services/api/prisma/migrations/0002_abuse_controls/migration.sql
• services/api/src/routes/admin.ts
• services/api/src/index.ts
• services/web/src/pages/Legal/PrivacyPolicy.tsx
• blueprint.md
• services/api/src/utils/audit.ts

The email-platform employs a multi-faceted strategy for data retention and deletion, combining automated cleanup of transient data with explicit, audited administrative actions for permanent records. The system is designed to manage the lifecycle of temporary inboxes and messages through a configurable Time-to-Live (TTL) mechanism, while also providing administrators with the tools to permanently remove user data in a cascaded fashion.

This approach balances the need for system hygiene and resource management with data governance and compliance requirements. Deletion mechanisms include both soft deletes (marking records as deleted for later cleanup) and hard deletes (permanent removal from the database). All significant deletion events are logged in an audit trail for accountability.

Automated Retention Sweep

The core of the automated data cleanup is the runRetentionSweep function, which periodically purges expired messages, attachments, and inboxes from the system. This process is crucial for managing storage and enforcing the "temporary" nature of the email inboxes.

Sources: services/api/src/retention.ts, services/api/src/index.ts

Execution Trigger

The retention sweep is executed at a regular interval, defined by the retentionSweepMinutes configuration. The main application server sets up a setInterval timer upon startup to invoke the runRetentionSweep function.

Sources: services/api/src/index.ts:70-72

Configuration

The behavior of the retention sweep is controlled by several environment variables, which are parsed into the appConfig object.

Variable	Description	Default
MESSAGE_TTL_HOURS	The age in hours after which a message is considered expired and eligible for deletion.	168 (7 days)
INBOX_TTL_HOURS	The age in hours after which a soft-deleted inbox is eligible for permanent deletion.	720 (30 days)
RETENTION_SWEEP_MINUTES	The interval in minutes at which the retention sweep job runs.	60

Sources: services/web/src/pages/Legal/PrivacyPolicy.tsx:39-42, README.md:92, blueprint.md:73

Sweep Process Flow

The runRetentionSweep function executes a two-phase cleanup: first for messages and their attachments, and then for inboxes and any remaining associated data.

This diagram illustrates the logic within the runRetentionSweep function.

The process is as follows:

1. Expired Messages: It queries for messages where receivedAt is older than messageTtlHours OR deletedAt is older than messageTtlHours.
2. Attachment File Deletion: For each expired message, it identifies associated attachments and deletes the corresponding physical files from the storage directory.
3. Message & Attachment DB Deletion: It performs a deleteMany operation to remove the database records for the attachments and the messages themselves.
4. Expired Inboxes: It queries for inboxes where expiresAt is in the past OR deletedAt is older than inboxTtlHours.
5. Inbox Data Deletion: For any expired inboxes found, it first finds and deletes all associated attachment files before deleting the inbox records from the database.
6. Logging: The function concludes by logging the number of deleted messages and inboxes.

Sources: services/api/src/retention.ts:27-59

Data Deletion Mechanisms

The platform supports both soft and hard deletion of data. Soft deletion is used for user-facing delete actions, while hard deletion is typically reserved for administrative tasks and the automated retention sweep.

Soft Deletion

Soft deletion is implemented by setting a deletedAt timestamp on a record instead of removing it from the database immediately. This allows for potential recovery and ensures that the automated retention sweep can properly clean up associated data later.

The database schema was updated to include this field on key tables.

Sources: services/api/prisma/migrations/0002_abuse_controls/migration.sql:6-9

This ER diagram shows the relevant tables and their deletedAt field.

Records with a non-null deletedAt value are treated as deleted in most application queries and are eventually hard-deleted by the runRetentionSweep process.

Hard Deletion (Admin Actions)

Administrators have the capability to permanently delete a user and all their associated data. This is a destructive, cascading operation.

Sources: services/api/src/routes/admin.ts:200-241

Admin User Deletion Endpoint

• Endpoint: DELETE /admin/users/:id
• Description: Permanently deletes a user and all their associated domains, inboxes, and messages.
• Protection: This action is protected and can only be performed by an administrator. An admin cannot delete their own account.

Sources: services/api/src/routes/admin.ts:200-204, services/api/src/routes/admin.ts:215-217

Deletion Flow

The deletion is performed within a single database transaction to ensure atomicity.

This sequence diagram shows the process of an admin deleting a user.

Auditing Deletion Events

All critical deletion actions are recorded in the AuditLog table to provide a trail of administrative activities. The recordAudit utility is used for this purpose.

Action	Description	Trigger
USER_DELETED	An administrator permanently deleted a user account and all their data.	DELETE /admin/users/:id
MESSAGE_DELETED	A user or admin deleted a message (likely a soft delete).	DELETE /messages/:id

Sources: services/api/src/routes/admin.ts:237-240, services/web/src/components/admin/AdminLogs.tsx:81

The audit log entry captures the performing user, the action taken, relevant metadata (like the ID and email of the deleted user), and a timestamp. This information is viewable in the admin panel.

Sources: services/api/src/routes/admin.ts:316-340, services/web/src/components/admin/AdminLogs.tsx