Job Scheduling & Reminders
Relevant source files
The following files were used as context for generating this wiki page:
The Job Scheduling & Reminders system is a core infrastructure component of CodeBot designed to manage asynchronous background tasks, user-defined alerts, and system-wide maintenance routines. It leverages the python-telegram-bot JobQueue and APScheduler to ensure reliable execution of tasks such as database cleanups, GitHub synchronization, and time-sensitive user notifications.
This system is divided into two primary domains: Reminders, which are user-facing scheduled notifications for code snippets or tasks, and Background Jobs, which are system-level processes tracked and managed via a centralized registry and monitor.
1. System Architecture
The architecture relies on a central JobRegistry for definition and a JobTracker for monitoring runtime execution. User-facing reminders utilize a dedicated ReminderScheduler that interacts with a MongoDB-backed RemindersDB.
The diagram shows the relationship between the definition registry, the runtime tracker, and the underlying persistence layers.
Sources: services/job_registry.py, services/job_tracker.py, reminders/scheduler.py
2. Background Job Management
Background jobs are categorized, registered, and tracked to provide observability into system health.
2.1 Job Registry and Categories
Jobs are classified into logical categories to facilitate management via ChatOps or the Webapp monitor.
| Category | Description | Example Jobs |
|---|---|---|
BACKUP | Data preservation tasks | backups_cleanup, drive_reschedule |
SYNC | External service integration | drive_scheduled_backup |
CLEANUP | Maintenance and TTL enforcement | cache_maintenance |
MONITORING | Health checks and samplers | jobs_stuck_monitor, predictive_sampler |
BATCH | Large scale code processing | batch_analyze, batch_validate |
Sources: services/job_registry.py:10-25, chatops/jobs_commands.py:115-130
2.2 Job Tracking and Lifecycle
The JobTracker provides a context manager to wrap job execution, automatically logging start times, end times, and success/failure statuses.
This flow illustrates how the system ensures that even if a job crashes, its state is captured in the database.
Sources: services/job_tracker.py:75-150
3. User Reminders
Reminders allow users to schedule notifications through an interactive Telegram flow or quick slash commands.
3.1 Reminder Lifecycle
The lifecycle involves validation of time inputs, persistence in MongoDB, and scheduling in the Telegram JobQueue.
3.2 Scheduling Logic
The ReminderScheduler is responsible for reloading reminders upon bot restart to ensure no notifications are lost.
- Initial Load: Scans
RemindersDBfor pending jobs on startup. - Due Jobs: Jobs that should have run while the bot was offline are sent immediately.
- Future Jobs: Future jobs are re-added to the Telegram
JobQueue.
Sources: reminders/scheduler.py:72-110, main.py:1700-1720
4. Monitoring and ChatOps
The system provides several tools for real-time monitoring of scheduled tasks.
4.1 ChatOps Commands
Administrators can use the /jobs command to inspect the state of the background worker.
| Command | Usage | Result |
|---|---|---|
/jobs active | List currently running jobs | Shows progress % and direct log links |
/jobs failed | List recent job failures | Shows error messages and timestamps |
/jobs <id> | Inspect specific job | Shows metadata and last 5 execution runs |
Sources: chatops/jobs_commands.py:15-60
4.2 Stuck Job Detection
A specialized monitoring job, jobs_stuck_monitor, runs periodically to identify jobs that have remained in the running state longer than a configured threshold (default: 20 minutes). It appends an error log and emits an observability event when a stuck job is detected.
Sources: main.py:1560-1610
5. Implementation Details
5.1 Job Registry Definition
Jobs are defined using a structured metadata approach in job_registry.py:
# services/job_registry.py
class JobDefinition(TypedDict):
job_id: str
name: str
description: str
category: JobCategory
job_type: JobType
interval_seconds: Optional[int]
enabled: bool
Sources: services/job_registry.py:35-45
5.2 Persistence Layer
Reminders are stored with the following schema characteristics:
reminder_id: Unique identifier (UUID).user_id: Target recipient.remind_at: UTC timestamp for execution.is_sent: Boolean flag to prevent duplicate notifications.
Sources: reminders/handlers.py:50-80
The Job Scheduling & Reminders system provides a robust framework for managing temporal tasks within CodeBot. By separating user-level reminders from system-level background jobs and providing centralized tracking, the project achieves high reliability and observability for all its asynchronous operations.