Upgrading to v9
Relevant source files
The following files were used as context for generating this wiki page:
Upgrading to version 9 of sidekiq-unique-jobs involves a transition to a more streamlined Redis data structure and updated dependency requirements. The primary goal of this version is to reduce Redis overhead by consolidating lock metadata and requiring modern versions of Ruby, Sidekiq, and Redis. The upgrade process is designed to be automatic, migrating existing version 8 locks to the new format upon the first startup.
Sources: README.md:144-152, lib/sidekiq_unique_jobs/upgrade_locks.rb:48-62
System Requirements
Version 9 introduces stricter dependency requirements to leverage modern features like atomic LMOVE in Redis.
| Requirement | Minimum Version |
|---|---|
| Ruby | >= 3.2 |
| Sidekiq | >= 8.0 |
| Redis | >= 6.2 |
Sources: README.md:20-23
Automatic Migration Logic
The SidekiqUniqueJobs::UpgradeLocks class handles the transition from v8 to v9 data structures. It uses a specific key, #{LIVE_VERSION}:UPGRADED, to track if a migration has already been performed to prevent redundant processing. The migration process identifies obsolete v8 keys and unlinks them while preserving the core LOCKED hash and the digests sorted set.
The migration flow ensures data integrity by checking the current status before attempting to modify Redis keys.
Sources: lib/sidekiq_unique_jobs/upgrade_locks.rb:48-62, spec/sidekiq_unique_jobs/upgrade_locks_spec.rb:42-45
Key Consolidation
In version 9, the number of Redis keys per lock has been significantly reduced from 13 down to 2. The system now primarily relies on:
digest:LOCKED: A hash containing lock metadata.uniquejobs:digests: A sorted set containing all active digests.
Sources: README.md:148-150
Obsolete Key Removal
The upgrade process specifically targets and deletes several suffixes that were used in version 8 but are no longer required in version 9.
| Obsolete Suffix | Description |
|---|---|
:QUEUED | Job queue status (v8) |
:PRIMED | Lock preparation status (v8) |
:INFO | Metadata storage (v8) |
:RUN | Variants for concurrent execution (v8) |
Sources: lib/sidekiq_unique_jobs/upgrade_locks.rb:108-115, spec/sidekiq_unique_jobs/upgrade_locks_spec.rb:47-60
Expiring Digests Merging
Previously, expiring locks were stored in a separate uniquejobs:expiring_digests sorted set. In v9, this is unified into the main uniquejobs:digests ZSET. The migration iterates through the expiring digests and moves them to the primary set while preserving their scores (expiry times).
Reliable Fetch Strategy
Version 9 includes a new SidekiqUniqueJobs::Fetch::Reliable strategy. This replaces standard fetching with a mechanism that supports atomic LMOVE operations and lock-aware job acknowledgment.
- Atomic LMOVE: Moves jobs from the queue to a per-process working list to prevent job loss during crashes.
- Crash Recovery: Automatically recovers jobs from dead worker processes on startup.
- Lock-Preserving Requeue: Ensures locks persist for jobs that are requeued during server shutdown.
To enable this, the fetch_class must be configured in the Sidekiq server initializer:
Sidekiq.configure_server do |config|
config[:fetch_class] = SidekiqUniqueJobs::Fetch::Reliable
end
Sources: README.md:118-131
Observability and Reflection
The legacy logging and changelog systems have been removed in favor of the Reflection API. This system allows for observing lock lifecycle events (e.g., locked, unlocked, lock_failed) without altering job behavior.
SidekiqUniqueJobs.reflect do |on|
on.locked { |job| logger.info("Locked: #{job['class']}") }
on.unlocked { |job| logger.info("Unlocked: #{job['class']}") }
end
Sources: README.md:158-165, UPGRADING.md:5-12, lib/sidekiq_unique_jobs/web/helpers.rb:50
Conclusion
Upgrading to v9 represents a major shift toward efficiency and modern Redis usage. By automating the migration of v8 keys and consolidating expiring digests into a single ZSET, the gem reduces memory usage and simplifies lock management. Developers should ensure their environment meets the new Ruby and Sidekiq version requirements before deploying the update.
Sources: README.md:144-156