File Uploads and Downloads
Relevant source files
The following files were used as context for generating this wiki page:
The DXKB web application provides a robust system for managing data transfer between local environments and the user's workspace. This system supports multi-file uploads with progress tracking and various download methods, including single-file retrieval, batch archiving, and table exports in multiple formats.
The architecture relies on specialized widgets for user interaction, a centralized manager for handling asynchronous upload tasks, and integration with the Workspace service to maintain data integrity and metadata synchronization.
File Upload Architecture
The upload system is split into a user interface layer and a background management layer. The Uploader.js widget provides the interface for selecting files via file pickers or drag-and-drop actions, while UploadManager.js handles the actual XMLHttpRequest operations and progress monitoring.
Components and Data Flow
- Uploader Widget: A templated widget that allows users to select files, choose upload types, and specify destination paths.
- Upload Manager: A singleton service that executes asynchronous
PUTrequests to the workspace API. It maintains the state of active, completed, and failed uploads. - Upload Status Widget: A global controller that subscribes to upload events and displays real-time progress to the user.
The diagram shows the interaction between the UI widgets and the background manager during a file transfer.
Sources: public/js/p3/widget/Uploader.js, public/js/p3/UploadManager.js
Key Functions and Structures
| Function/Property | Location | Description |
|---|---|---|
upload(files, token) | UploadManager.js | Main entry point for starting one or more file uploads. |
_uploadFile(file, url, path) | UploadManager.js | Internal method that configures XMLHttpRequest and attaches progress listeners. |
onFileSelectionChange(evt) | Uploader.js | Handles file selection events and builds the internal file table for staging. |
initDragAndDrop() | Uploader.js | Configures event listeners for the dnd-zone element to support file dropping. |
updateAutoMetadata(paths) | WorkspaceManager.js | Triggered after successful upload to ensure the workspace index is current. |
Sources: public/js/p3/UploadManager.js:30-45, public/js/p3/widget/Uploader.js:235-250
Download Systems
The platform provides three distinct ways to retrieve data from the workspace: direct file download, advanced genome/feature bundling, and table exports.
1. Direct and Archive Downloads
Managed primarily through the WorkspaceBrowser.js, users can download single files or select multiple items to be packaged into a compressed archive (e.g., ZIP).
- Single File: Executed via
WorkspaceManager.downloadFile(path). - Archive Download: Executed via
WorkspaceManager.downloadArchiveFile(paths, name, type, recursive). This opens a dialog for the user to specify the archive name.
Sources: public/js/p3/widget/WorkspaceBrowser.js:485-585
2. Advanced Download Widget
The AdvancedDownload.js widget specialized in bundling genomic data. It allows users to filter specific file types (e.g., protein sequences, GFF features) before generating a bundle.
| Config Option | Data Type | Available Formats |
|---|---|---|
genome_data | genome | .fna, .faa, .gff, .features.tab, .ffn, .frn, .pathway.tab |
sequence_data | sequence | Table Data |
feature_data | feature | Table Data |
Flowchart illustrating the decision logic for standard downloads in the Workspace Browser.
Sources: public/js/p3/widget/AdvancedDownload.js:15-45, public/js/p3/widget/WorkspaceBrowser.js:500-550
3. Table Export
Users can export metadata and search results directly as formatted tables. This is handled by generating requests to the data service with specific http_accept headers.
- Supported Formats:
text/tsv,text/csv,application/vnd.openxmlformats(Excel). - Implementation: Constructs a URL including the user's authorization token and the requested format, then uses
window.open()to trigger the download.
Sources: public/js/p3/widget/WorkspaceBrowser.js:600-630
Progress Monitoring and State Management
Upload status is managed centrally by the UploadManager and broadcasted via the /upload topic.
Upload State Object
The UploadManager maintains an internal state for tracking metrics:
this.activeCount = 0;
this.completeCount = 0;
this.completedUploads = [];
this.errorCount = 0;
this.inProgress = {}; // Keyed by filename
Sources: public/js/p3/UploadManager.js:10-15
User Notification
The UploadStatus.js widget listens for these updates to provide visual feedback:
- UploadStart: Increments the active count and adds the file to the progress dropdown.
- UploadProgress: Calculates the percentage based on
evt.loadedandevt.total. - UploadComplete: Moves the file to the "Completed" list and decrements active counts.
- Unload Prevention: The
UploadManagerattaches abeforeunloadlistener to the window ifactiveCount > 0to prevent users from accidentally cancelling transfers.
Sources: public/js/p3/widget/UploadStatus.js:45-100, public/js/p3/UploadManager.js:17-25
Conclusion
DXKB's data transfer system provides a seamless experience for moving large biological datasets. By decoupling the UI widgets from the transfer logic, the system ensures that uploads can continue in the background while users navigate other parts of the application. The integration of progress tracking and multi-format downloads allows for flexible data retrieval tailored to bioinformatics workflows.