Asynchronous Job Management
Relevant source files
The following files were used as context for generating this wiki page:
Asynchronous Job Management in the DXKB platform facilitates the execution and monitoring of long-running bioinformatics tasks. Since many computational analyses exceed the duration of a standard HTTP request, the system employs a background processing architecture where jobs are submitted to an underlying AppService, tracked via a centralized manager, and updated in the user interface through real-time polling and event-driven notifications.
Users interact with this system through service-specific application forms, a dedicated Job Manager dashboard, and a status ticker that provides immediate feedback on queued, running, and completed tasks. This architecture ensures a responsive user experience while the backend handles heavy computational workloads.
Sources: public/js/p3/JobManager.js, public/js/p3/widget/app/AppBase.js
System Architecture and Job Lifecycle
The system is built on a decoupled architecture where the frontend manages state and presentation while delegating execution to the AppService. The lifecycle of a job transitions through several states: initialization, queuing, in-progress, and finally completion or failure.
Job Lifecycle Flow
The following diagram illustrates the transition of a job from submission through final state resolution.
The workflow begins in AppBase.js, which handles the RPC call to start the application. The JobManager.js then takes over, polling the service at regular intervals (every 5 seconds) to detect status changes and publish updates to the UI via Dojo topics.
Sources: public/js/p3/JobManager.js:145-165, public/js/p3/widget/app/AppBase.js:305-325
Key Components
JobManager (Core Logic)
The JobManager.js file serves as the singleton controller for job state across the entire application. It maintains an internal MemoryStore of jobs and performs regular polling of the AppService.
| Function | Description |
|---|---|
PollJobs() | Initiates recurring checks (5000ms interval) for job status changes. |
getStatus() | Queries AppService.query_task_summary to get global counts of queued, running, and completed jobs. |
updateJobsList() | Retrieves the detailed list of jobs using AppService.enumerate_tasks and updates the internal data store. |
killJob(id) | Sends a termination signal to the backend for a specific task ID. |
setJobHook() | Allows application forms to "hook" into a specific job to receive a callback when that specific task finishes. |
Sources: public/js/p3/JobManager.js:35-140
JobStatus Widget
The JobStatus.js widget provides a persistent visual indicator (ticker) of job activity, typically located in the application header. It subscribes to the /JobStatus topic to display real-time counts.
Job Submission and "Live" Monitoring
Jobs are submitted via the AppBase class, which serves as the base for all service forms. It includes a "lookahead" feature that allows the UI to stay synced with a specific job the user just submitted.
AppBase Submission Logic
When a user clicks "Submit," AppBase.doSubmit calls AppService.start_app2. If lookaheadJob is enabled, the system provides a "Live Job" visual indicator and sets a hook in the JobManager to trigger a specific callback once the job completes.
Sources: public/js/p3/widget/app/AppBase.js:290-330
Job Manager Dashboard
The public/js/p3/widget/JobManager.js widget provides a full-screen administrative view of all user jobs. It combines a JobsGrid for data display with a JobContainerActionBar for filtering and actions.
Filtering and Actions
The dashboard allows users to filter jobs by service type (e.g., "Genome Assembly") or by status. Key actions available for selected jobs include:
- View: Navigates to the results in the workspace.
- Kill Job: Terminates a running task.
- Rerun: Re-populates the original service form with the parameters used for that job.
- Report Issue: Generates a pre-filled support ticket containing job metadata, Stdout, and Stderr URLs.
Sources: public/js/p3/widget/JobManager.js:65-150, public/js/p3/widget/JobContainerActionBar.js:100-150
Job Results and Visualization
Once a job reaches the "completed" state, results are viewed through the JobResult viewer.
Data Resolution Logic
The JobResult.js viewer performs the following steps to render job outputs:
- Metadata Retrieval: Accesses
autoMetato find job parameters and output file lists. - Path Rewriting: If the job result was moved in the workspace, it updates the internal paths to match the current location.
- Hidden Folder Access: Jobs store actual data in a hidden folder (prefixed with a dot, e.g.,
.JobName). The viewer retrieves folder contents to ensure it has the most recent file objects. - Dynamic Headers: Displays job runtime, ID, and parameters in a formatted table.
Sources: public/js/p3/widget/viewer/JobResult.js:40-80
Conclusion
The Asynchronous Job Management system provides a robust framework for handling intensive bioinformatics computations. By centralizing job state in the JobManager and utilizing Dojo's topic-based communication, the platform maintains real-time synchronization between backend task execution and the user's frontend workspace experience.