Performance & Concurrency Optimizations

Relevant source files

The following files were used as context for generating this wiki page:

• backend/lib/inflight-dedup.ts
• backend/lib/request-abort.ts
• extension/src/content/long-wait-manager.ts
• extension/src/content/index.ts
• extension/src/background/index.ts
• tasks/todo.md

Performance and Concurrency Optimizations in Oddity 1 ensure the system remains responsive during intensive Large Language Model (LLM) operations. The architecture minimizes redundant network traffic, manages browser resource limits, and provides visual feedback for long-running processes. These optimizations span the background service worker, content scripts, and the backend inference engine.

The system prioritizes low-latency revisits through multi-tier caching and prevents UI flicker by using specialized DOM guards and stability watchers.

Concurrency and Request Management

The background service worker acts as a central orchestrator for all API traffic, implementing strict limits to prevent browser resource exhaustion and API rate limiting.

Tab-Level Concurrency Limiting

The background script enforces a maximum of three concurrent API requests per browser tab. This prevents a "burst" of requests when a user rapidly switches personalities or enables "All" mode for a large document.

• Slot Acquisition: The acquireSlot function checks the tabConcurrency map. If a tab already has three active requests, the system waits and retries every 200ms until a slot opens.
• Release Logic: The releaseSlot function decrements the count when a request completes, fails, or is aborted.

Sources: extension/src/background/index.ts:33-48

Request Aborting

Oddity 1 uses AbortController to cancel stale in-flight requests. This is critical when a user changes the intensity or personality mid-generation.

• Keying: Abort controllers are keyed by tabId:mode:regionId.
• Replacement: If a new request arrives for the same region and mode, the background script immediately aborts the existing controller to free up the concurrency slot and backend resources.

Sources: extension/src/background/index.ts:24-31, extension/src/background/index.ts:168-175

Concurrency Flow

The following sequence diagram illustrates how a request acquires a concurrency slot and handles potential aborts.

This diagram shows the acquisition and release of concurrency slots to manage browser traffic.

Caching Strategies

Oddity 1 implements a four-tier caching strategy to eliminate redundant LLM inference calls and provide near-instant rendering on revisited pages.

Caching Tiers

Tier	Location	Keying Logic	TTL / Purpose
Session Cache	chrome.storage.session	contentHash + personality	In-memory; survives tab life.
URL Cache	chrome.storage.local	URL	Instant revisits via URL prediction.
DB Cache	Supabase	content_hash + intensity	30-day global cache shared by users.
In-Flight Dedup	Memory	tabId:regionId	Prevents duplicate calls for same region.

Sources: extension/src/background/index.ts:152-165, tasks/todo.md:144-150

URL Prediction and speculative Render

On page load, the content script attempts a "Speculative Render" by querying the getUrlCache. If a previous visit for that exact URL is found, the extension renders existing annotations immediately while the background pipeline verifies the current content stability.

Sources: extension/src/content/index.ts:1145-1175

Rendering and DOM Optimizations

The rendering engine uses several techniques to ensure that injecting annotations does not degrade host page performance or crash modern web frameworks.

React-Safe Anchor Guard

On sites using React or similar virtual DOM frameworks (e.g., ChatGPT, Claude), Oddity 1 implements an anchorGuard. If a user clicks outside the annotated region, the extension pre-emptively strips all anchor spans. This prevents the host framework from crashing when it encounters unexpected DOM nodes during a re-render.

Sources: extension/src/content/index.ts:434-474

Stability Detection and Debouncing

The ChatObserver and StabilityWatcher prevent premature annotation requests on streaming content.

• Stability Signal: The system waits for a specific signal (like the appearance of a "copy" button) or a mutation debounce before considering a region "stable."
• Rescan Debounce: A 1.5-second timer prevents the MutationObserver from firing repeated scans during rapid DOM updates.

Sources: extension/src/content/index.ts:945-965, extension/src/content/index.ts:994-1005

Rendering Process Flow

The flowchart below depicts the checks performed before requesting annotations.

This flowchart illustrates the deduplication logic used to prevent redundant API calls.

Inference and Data Flow Optimizations

Specific optimizations reduce the payload size and perceived latency of AI generation.

Streaming JSON and Progressive Rendering

Instead of waiting for a full JSON object, the backend streams individual annotation objects.

1. Partial Arrivals: The background worker forwards annotationReady messages to the content script as they are parsed from the SSE stream.
2. Incremental Display: The content script resolves selectors and injects highlights one-by-one, allowing the user to start reading before the entire document is processed.

Sources: extension/src/background/index.ts:187-200, extension/src/content/index.ts:1255-1310

Long Request Management

The LongWaitManager tracks regions that exceed a 500ms response threshold.

• Visual Indicator: If a request is identified as "long," the system displays a "generating annotations" toast.
• Resolution: The toast dismisses automatically once the first annotation arrives or the request fails.

Sources: extension/src/content/long-wait-manager.ts, extension/src/content/index.ts:1060-1070

Content Truncation

To maintain backend performance and adhere to LLM context limits, the extension truncates extracted text to MAX_TEXT_LENGTH (defined in shared constants) before hashing and transmission.

Sources: extension/src/content/index.ts:1025-1030

Conclusion

Performance and concurrency in Oddity 1 are managed through a multi-layered approach of throttling, caching, and smart DOM handling. By limiting concurrent requests to three per tab and utilizing SHA256 content hashing for deduplication, the system ensures efficiency. The integration of speculative rendering and progressive streaming minimizes perceived latency, providing a seamless user experience even on long-form or dynamic content.