Uniqueness Scopes & Digests

Relevant source files

The following files were used as context for generating this wiki page:

• lib/sidekiq_unique_jobs/lock_digest.rb
• lib/sidekiq_unique_jobs/digests.rb
• lib/sidekiq_unique_jobs/job.rb
• lib/sidekiq_unique_jobs/key.rb
• lib/sidekiq_unique_jobs/upgrade_locks.rb
• spec/sidekiq_unique_jobs/lock_digest_spec.rb
• CLAUDE.md

Introduction

Uniqueness in sidekiq-unique-jobs is governed by the generation of a unique Lock Digest. This digest serves as the primary identifier in Redis to track whether a specific job is already enqueued or executing. By hashing job metadata—such as the worker class, arguments, and queue—the system can identify duplicates across the Sidekiq ecosystem.

The scope of uniqueness can be highly customized through worker options, allowing developers to ignore specific job parameters (like the queue name) or include external context (like multi-tenancy keys). This mechanism ensures that the gem can handle various concurrency requirements, from simple duplicate prevention to complex time-based uniqueness.
Sources: CLAUDE.md:41-45, lib/sidekiq_unique_jobs/lock_digest.rb

Digest Generation Logic

The LockDigest class is responsible for calculating the unique string that represents a job's identity. It constructs a "digestable hash" based on the job's configuration and then applies a hashing algorithm to produce the final key.

Digest Components

By default, the following attributes are included in the digest calculation:

• Class: The Sidekiq worker class name.
• Queue: The name of the queue the job is pushed to.
• Lock Args: The arguments passed to the perform method (often filtered).
• Apartment: Tenant information if using the apartment gem.

Sources: lib/sidekiq_unique_jobs/lock_digest.rb:39-50, spec/sidekiq_unique_jobs/lock_digest_spec.rb:88-100

Hashing Algorithms

The gem supports two primary hashing algorithms:

• Legacy: Uses the MD5 algorithm. This is the default but may not be compatible with FIPS-enabled Redis environments.
• Modern: Uses SHA3-256 for FIPS compatibility and improved collision resistance.

Sources: CLAUDE.md:143-146, spec/sidekiq_unique_jobs/lock_digest_spec.rb:58-86

Data Flow for Digest Creation

The following diagram illustrates how job data is transformed into a unique Redis key.

Uniqueness Scopes

Scopes allow developers to expand or narrow what constitutes a "duplicate" job.

Option	Description	Effect on Digest
unique_across_queues	If true, the queue name is ignored.	Jobs with same args on queue_a and queue_b conflict.
unique_across_workers	If true, the worker class is ignored.	WorkerA and WorkerB with same args conflict.
lock_args_method	A Proc or Symbol defining how to filter arguments.	Only specific arguments contribute to the digest.

Sources: lib/sidekiq_unique_jobs/lock_digest.rb:67-85, CLAUDE.md:118-125

Customizing Arguments

Developers often use lock_args_method to exclude transient data (like timestamps or UUIDs) from the uniqueness check. If a worker defines self.lock_args(args), that method will be called to return only the values that should be part of the digest.
Sources: CLAUDE.md:140-142, spec/sidekiq_unique_jobs/lock_digest_spec.rb:43-55

Storage & Redis Key Structure

In Version 9+, the storage of digests has been streamlined. Instead of multiple keys per lock, the system uses a centralized sorted set and a single hash for lock metadata.

Redis Keys

• uniquejobs:digests: A Sorted Set (ZSET) containing all active digests. The score represents the time the lock was created or will expire.
• [digest]:LOCKED: A Hash containing metadata for a specific lock, including the Job ID (JID) and configuration.
• uniquejobs:working:[host]: Keys used by the Reliable Fetch mechanism to track active worker processes.

Sources: lib/sidekiq_unique_jobs/digests.rb:39-55, lib/sidekiq_unique_jobs/key.rb:14-36, README.md:120-125

Relationships between Entities

Lifecycle Management

Digests are not permanent; they are managed through various cleanup and transition phases.

Addition and Deletion

When a job is prepared, the SidekiqUniqueJobs::Job.prepare method adds the lock_digest to the job payload. Once the lock is acquired, the digest is added to the uniquejobs:digests ZSET.
Deletion occurs via delete_by_digest (for specific jobs) or delete_by_pattern (for bulk cleanup via CLI or Web UI).
Sources: lib/sidekiq_unique_jobs/job.rb:21-40, lib/sidekiq_unique_jobs/digests.rb:28-65

Version Upgrades

During the transition from v8 to v9, the gem performs an automated upgrade of digests. This includes:

1. Merging ZSETs: Moving entries from uniquejobs:expiring_digests into the main uniquejobs:digests ZSET.
2. Cleanup: Unlinking obsolete v8 keys like :QUEUED, :PRIMED, and :INFO.
   Sources: lib/sidekiq_unique_jobs/upgrade_locks.rb:116-160

Conclusion

Uniqueness Scopes & Digests provide the foundation for Sidekiq concurrency control. By transforming job metadata into a standardized LockDigest and storing it in a optimized Redis structure, the gem ensures high-performance duplicate detection while offering the flexibility to define uniqueness across queues, workers, and custom argument filters.
Sources: lib/sidekiq_unique_jobs/lock_digest.rb, lib/sidekiq_unique_jobs/digests.rb