Async & Concurrency Utilities
Relevant source files
The following files were used as context for generating this wiki page:
Async and concurrency utilities in Vantage manage high-latency operations, including external API data collection, browser automation, and multi-tenant background scans. The system coordinates these tasks through cancellation signals, resource leasing, and stateful resumption to ensure data consistency and efficient resource utilization.
Vantage relies on structured concurrency patterns to prevent race conditions during UI updates and to handle upstream rate limits without blocking the execution pipeline. These utilities interact with the Collector Framework to maintain healthy data ingestion even when facing provider failures.
Request Synchronization and Cancellation
Vantage uses AbortSignal to propagate cancellation throughout the network and process stack. Collectors and social provider clients monitor these signals to stop processing before incurring costs or performing unnecessary operations.
UI State Synchronization
The DraftPanel component prevents race conditions in asynchronous UI updates using a requestRevision reference. When a user triggers a new draft generation or target change, the component increments the revision ID. The asynchronous callback only updates the React state if the revision ID matches the current version at the time of completion.
The revision pattern ensures that stale API responses do not overwrite newer user intent.
Sources: app/opportunities/[id]/draft-panel.tsx:38-50, app/opportunities/[id]/draft-panel.tsx:82-88
Network Cancellation
Provider clients pass AbortSignal to external fetch requests. If the signal aborts, the client throws the signal's reason immediately, bypassing follow-up paid escalations or fixture fallbacks.
Sources: lib/collectors/github.ts:32, test/social-provider-routing.test.ts:140-146
Workspace Concurrency Control
The system serializes concurrent operations for a specific workspace using a database-backed leasing mechanism. The workspace table stores a scanLeaseToken and scanLeaseUntil timestamp to prevent overlapping scans.
Scan Leases
The withWorkspaceScanLease utility wraps critical sections, such as feed installation and manual scans. It ensures that only one process holds the lease at a time, protecting shared resources like the source count limits and document deduplication logic.
| Field | Type | Description |
|---|---|---|
scanLeaseToken | text | Unique identifier for the current lease holder. |
scanLeaseUntil | timestamp | Expiration time after which the lease is considered stale. |
Sources: lib/db/schema.ts:105-106, test/source-actions.test.ts:7-13
The lease lifecycle prevents redundant costs and database contention during automated and manual triggers.
Sources: test/source-actions.test.ts:52-57
Rate Limit and Timeout Strategies
Collectors handle asynchronous delays and provider-enforced pauses through deterministic state management and timeout enforcement.
Rate Limit Deferral
The GitHub collector implements a rate-limit deferral flow. When it receives a 429 Too Many Requests or 403 Forbidden (with rate limit headers), it calculates a deferUntil timestamp based on Retry-After or x-ratelimit-reset headers. The collector then returns a partial result and saves the resumption state in the cursor.
Timeout Encodement
Browser automation tasks use an explicit timeoutMs. If the execution exceeds this limit, the utility throws a BrowserRunError with the code timeout. The system records the elapsed wall time as billable units even upon failure to account for reserved compute resources.
Sources: test/browser-run.test.ts:167-175
Resumption and State Persistence
To maintain progress across asynchronous executions, Vantage uses a JSON-based cursor to store pagination and filtering state.
| Cursor Field | Description |
|---|---|
since | The timestamp watermark for data lookback. |
until | The upper boundary of the current scan window. |
queryIndex | The index of the current keyword in a multi-query scan. |
nextPage | The next numeric page to fetch from the provider. |
deferUntil | The timestamp before which the collector must not restart. |
Sources: lib/collectors/github.ts:40-42, lib/collectors/github.ts:148-150
Vantage ensures that long-running async tasks remain resilient to transient errors by persisting these watermarks into the source table after every partial batch.
Sources: lib/db/schema.ts:139, test/collector-runner.test.ts:103-112