Background Workers

Relevant source files

The following files were used as context for generating this wiki page:

• src/bgworker/mod.rs
• src/bgworker/pg.rs
• src/bgworker/redis.rs
• src/cli.rs
• loco-new/tests/templates/background.rs
• docs-site/config.toml

Background Workers in the Loco framework provide a robust system for offloading compute-intensive or I/O-heavy tasks from the main request-response lifecycle. By moving these operations to the background, applications can maintain high performance and responsiveness while processing complex logic asynchronously.

The system supports multiple execution modes—ranging from in-process async tasks to persistent, queue-backed processing using Redis, Postgres, or SQLite. Implementing a background task involves defining a struct that implements the BackgroundWorker trait, allowing developers to focus on the perform logic while the framework handles serialization, enqueuing, and execution management.

Sources: docs-site/config.toml:161-163, src/bgworker/mod.rs:596-616

Execution Modes

Loco provides three primary modes for handling background jobs, configured via the WorkerMode setting. The behavior of perform_later changes significantly depending on the selected mode.

Mode	Description	Persistence
BackgroundQueue	Jobs are serialized and pushed to a persistent queue (Redis, Postgres, or SQLite).	Yes
BackgroundAsync	Jobs are spawned as non-persistent Tokio tasks in the same process.	No
ForegroundBlocking	Jobs are executed immediately and synchronously, blocking the current thread.	No

Sources: src/bgworker/mod.rs:619-646, loco-new/tests/templates/background.rs:71-92

Mode Logic Flow

The following diagram illustrates how the framework determines execution behavior when a worker is triggered via perform_later.

Queue Providers

When running in BackgroundQueue mode, Loco utilizes a Queue abstraction that supports different backends. Each backend requires specific feature flags (e.g., bg_redis, bg_pg, bg_sqlt) to be enabled during compilation.

Supported Backends

• Redis: High-performance, memory-backed queue management.
• Postgres: Database-backed persistence using a dedicated jobs table.
• SQLite: Local file-based persistence, ideal for smaller deployments or development.

Sources: src/bgworker/mod.rs:68-100, src/bgworker/mod.rs:709-725

Job Status Lifecycle

Jobs within the queue move through various states throughout their processing lifecycle.

The BackgroundWorker Trait

Developers define tasks by implementing the BackgroundWorker trait. This trait requires a serializable argument type and provides hooks for customizing queue behavior.

Key Trait Methods

• perform(&self, args: A): The core logic of the job.
• build(ctx: &AppContext): Constructor for the worker instance.
• queue(): Optional method to specify a custom/priority queue name.
• tags(): Optional method to associate tags with the worker for selective processing.
• class_name(): Automatically derives a CamelCase name for the worker used in enqueuing.

Sources: src/bgworker/mod.rs:596-618

Enqueuing Sequence

This sequence shows the interaction between the Application Context, the Worker, and the Queue Provider.

Management CLI

Loco provides a set of CLI commands for inspecting and managing the job queue, available via the cargo loco jobs subcommand.

Command	Description
cancel	Sets specific jobs to cancelled status by name.
tidy	Deletes completed or cancelled jobs from the database.
purge	Deletes jobs based on age (default 90 days) and status.
dump	Exports job details to a YAML file for auditing or debugging.
import	Re-enqueues jobs from a previously exported YAML file.
requeue	Moves stale processing jobs back to queued status based on age.

Sources: src/cli.rs:608-654, src/bgworker/mod.rs:411-536

Job Cleanup and Maintenance

The clear_jobs_older_than and requeue functions are critical for maintaining database health in DB-backed queues. These operations filter by the updated_at timestamp and current JobStatus.

Sources: src/bgworker/mod.rs:460-482, src/bgworker/mod.rs:509-523

Summary

The Background Worker system in Loco is a flexible, trait-based architecture that allows for seamless transitions between simple async tasks and robust, persistent job queues. By abstracting the underlying provider (Redis, Postgres, or SQLite), the framework ensures that developers can scale their background processing requirements without changing their core business logic. Management tools integrated into the CLI further provide the operational control necessary for maintaining production job queues.

Sources: src/bgworker/mod.rs:102-132, src/bgworker/mod.rs:596-648