Backup Retention Policies
Relevant source files
The following files were used as context for generating this wiki page:
Backup Retention Policies bound the growth of backup storage by automatically removing old archives based on age or count rules. The system applies these policies to local filesystem storage and S3-compatible object storage independently. This ensures that unattended scheduled backups do not consume unbounded disk space while maintaining a minimum safety floor of the most recent backup.
Sources: specs/004-backup-retention-policy/spec.md:1-15, specs/004-backup-retention-policy/plan.md:12-18
Core Architecture
The retention system uses a pure selection function that evaluates parsed backup references against a RetentionPolicy. A storageLocation interface abstracts the physical storage, allowing the same logic to apply to local directories and S3 prefixes.
Selection Logic
The system employs union keep-semantics. A backup is kept if it meets either the age rule or the count rule. Additionally, an unconditional keep-newest floor prevents the deletion of the most recent backup at any location, even if it violates all other rules.
The diagram shows the decision flow for a single backup candidate. The newest backup always survives regardless of policy settings.
Sources: specs/004-backup-retention-policy/spec.md:200-210, specs/004-backup-retention-policy/research.md:R2
Data Structures
| Structure | Description |
|---|---|
RetentionPolicy | Contains Days and Count integers. A value of 0 indicates the rule is inactive. |
backupRef | Represents a discovered backup. Includes the Name and a derived CreatedAt timestamp. |
pruneOutcome | Records the results of a retention run, including kept, candidates for deletion, and actual deleted items. |
Sources: specs/004-backup-retention-policy/data-model.md
Retention Rules and Configuration
Operators configure retention through CLI flags or environment variables. At least one rule must be active (value ≥ 1) for the system to perform pruning.
- Max Age (
Days): Retains backups whose age is strictly less than the specified number of 24-hour periods. - Max Count (
Count): Retains the specified number of newest backups at a location.
Sources: specs/004-backup-retention-policy/spec.md:197-205, specs/004-backup-retention-policy/data-model.md:RetentionPolicy
Backup Identification
The system only considers files matching the specific naming pattern: infrahub_backup_YYYYMMDD_HHMMSS.tar.gz(.enc)?. The age of a backup is derived exclusively from the timestamp embedded in this filename. Files that do not match this pattern are ignored and never deleted.
Sources: specs/004-backup-retention-policy/research.md:R1, specs/004-backup-retention-policy/spec.md:220-225
Operational Workflows
Retention logic is integrated into two primary commands: create and prune.
Automatic Retention (create)
When configured, infrahub-backup create applies the retention policy automatically after a successful backup and S3 upload. If the backup fails, the system skips pruning. If pruning fails, the system returns a non-zero exit code but does not roll back the successful backup.
Manual Pruning (prune)
The prune command allows operators to trigger retention logic manually.
- Dry Run: Using
--dry-runlists candidates for deletion without removing them. - Interactive Prompt: By default, the command requires a
y/Nconfirmation. - Force Mode: The
--forceflag bypasses the interactive prompt for use in scripts.
Sources: specs/004-backup-retention-policy/spec.md:207-219, specs/004-backup-retention-policy/research.md:R5
Multi-Location Execution
The sequence shows how the orchestrator evaluates and prunes each storage location independently.
Sources: specs/004-backup-retention-policy/research.md:R3, specs/004-backup-retention-policy/plan.md:20-25
Failure Handling
Pruning is designed to be "best-effort." Within a storage leg, the system attempts to delete every eligible candidate even if individual deletions fail. Errors are aggregated using errors.Join and reported at the end of the operation. This prevents one transient failure from blocking the reclamation of other storage space.
Sources: specs/004-backup-retention-policy/spec.md:230-235, specs/004-backup-retention-policy/tasks.md:T009
Implementation Constraints
- Plakar Backend: In the initial version (v1), retention is not supported for the Plakar backend. Using retention flags with Plakar results in a warning during
createand a validation error duringprune. - Encrypted Archives: Retention treats encrypted (
.enc) and plain archives identically because it relies solely on filenames. - Permissions: S3 retention requires
ListBucketandDeleteObjectpermissions in addition to standard upload permissions.
Sources: specs/004-backup-retention-policy/spec.md:245-255, specs/004-backup-retention-policy/research.md:R4
Backup Retention Policies provide a critical safety mechanism for automated environments. By centralizing selection logic and enforcing a keep-newest floor, the system ensures storage stability without risking the complete loss of backup history.