Snapshot Evaluation Web Worker
Relevant source files
The following files were used as context for generating this wiki page:
The Snapshot Evaluation Web Worker handles the intensive tasks of fetching, decompressing, and evaluating market data snapshots within a background thread. By offloading these operations, the system ensures the user interface remains responsive while processing large datasets containing technical and fundamental stock metrics.
The worker executes two primary tasks: running complex screens against stock universes and performing symbol comparisons. If the browser environment does not support Web Workers or if worker construction is blocked by Content Security Policy (CSP), the system utilizes a main-thread fallback to maintain functionality. Sources: frontend/src/api/snapshotClient.ts:25-30, frontend/README.md:275-285
Architecture and Data Flow
The architecture employs a client-worker pattern where the snapshotClient manages the lifecycle and communication with the background snapshot.worker.ts. The client assigns a unique sequence ID to every task to track concurrent requests and ensure responses return to the correct callers. Sources: frontend/src/api/snapshotClient.ts:3-10
Request Sequence
The following diagram illustrates the lifecycle of a snapshot task from the UI request to the worker response:
The client maintains a pending Map to store resolution callbacks, keyed by the sequence ID. If a worker fails or returns an invalid response, the client terminates the worker and rejects all pending tasks. Sources: frontend/src/api/snapshotClient.ts:10-23, frontend/src/api/snapshotClient.ts:46-55
The Snapshot Engine
The snapshotEngine resides within the worker and manages the underlying data logic, including snapshot loading and result caching. It utilizes a loader function to retrieve data, which defaults to readSnapshot. Sources: frontend/src/api/snapshotEngine.ts:28-30
Decompression and Validation
The engine supports both raw JSON and Gzip-compressed snapshots. If the URL ends in .gz, the engine checks the magic bytes (0x1f 0x8b) and uses the browser's DecompressionStream to decode the data. The engine validates the snapshot by checking:
- Revision identity matching the source.
- Total stock count consistency.
- Date format (YYYY-MM-DD).
- Session date alignment.
Sources: frontend/src/api/snapshotEngine.ts:13-26
Caching Strategy
The engine implements a multi-level caching system to optimize performance:
- Revision Cache: Stores at most two loaded snapshot revisions.
- Match Cache: A
WeakMapassociated with snapshots that stores up to four result sets per snapshot to accelerate pagination and sorting changes.
Sources: frontend/src/api/snapshotEngine.ts:31-40, frontend/src/api/snapshotScreen.ts:101-104
Evaluation Logic
The snapshotScreen module provides the predicates required to filter stocks based on the expressionTree provided in the ScreenerRunRequest. Sources: frontend/src/api/snapshotScreen.ts:107-110
Condition Compilation
The engine compiles boolean expressions into predicates. It handles nested groups with Match All (AND) and Match Any (OR) logic. The combine function resolves these groups, correctly propagating null (unavailable) states. Sources: frontend/src/api/snapshotScreen.ts:20-22, frontend/src/api/snapshotScreen.ts:93-98
Supported Snapshot Metrics
The worker evaluates several categories of metrics directly from the snapshot:
| Category | Supported Fields / Conditions |
|---|---|
| Momentum | RVOL, Period Returns (1D, 5D, 21D, etc.), Gap Up/Down |
| Trend | Price vs SMA (10, 20, 50, 200), Price within % of MA |
| Fundamentals | P/E Ratio, ROE, Free Float %, EPS TTM, Dividend Yield |
| Liquidity | Average Turnover (20, 50, 100 days), Market Cap |
| Surveillance | ASM/GSM status exclusion |
| Presets | 45 built-in scans (precomputed in snapshot) |
Sources: frontend/src/api/snapshotScreen.ts:36-91, frontend/README.md:158-165
Fallback Mechanisms
When the Web Worker is unavailable, the runFallback function executes the engine on the main thread. Sources: frontend/src/api/snapshotClient.ts:12-23
Timeout Handling
The worker implementation includes strict timing controls to prevent UI freezes:
- Task Timeout: Individual tasks in the client are rejected after 60 seconds.
- Worker Termination: If a worker produces a fatal error, it is terminated immediately.
- Engine Fallback: If
Workeris undefined or blocked, the system dynamically importssnapshotEngineand runs the task in the current context.
Sources: frontend/src/api/snapshotClient.ts:7-11, frontend/src/api/snapshotClient.ts:46-55
Conclusion
The Snapshot Evaluation Web Worker serves as a critical performance layer, enabling the Nexus Scanner to process thousands of stock records with complex boolean logic in the browser. By combining Gzip decompression, intelligent caching of result sets, and robust fallback mechanisms, it provides a seamless screening experience regardless of hardware constraints.