Multi-layered Caching Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• extension/src/background/sw-cache.ts
• extension/src/background/url-cache.ts
• backend/routes/annotate.ts
• extension/src/background/index.ts
• tasks/todo.md
• packages/shared/src/constants.ts

Oddity 1 employs a multi-layered caching architecture to minimize latency for AI-generated annotations and reduce unnecessary LLM API costs. The strategy combines local browser storage for instant revisits with a server-side database for cross-user persistence.

The system prioritizes speed by attempting to resolve annotations through three distinct tiers: a Service Worker session cache, a URL-based local storage cache for speculative rendering, and a centralized Supabase database cache with a 30-day Time-To-Live (TTL).

Sources: tasks/todo.md:104-123, extension/src/background/index.ts:241-285

Caching Architecture Overview

The caching flow follows a "stale-while-revalidate" pattern. When a user requests annotations for a specific text region, the extension checks local caches first. If a match is found, the system renders the cached data immediately while simultaneously fetching fresh data from the backend to update the cache.

Data Flow for Annotation Requests

The following diagram illustrates how an annotation request navigates through the different cache layers before reaching the LLM generation stage.

The diagram shows the sequential lookup from local browser memory to server-side persistence.
Sources: extension/src/background/index.ts:245-310, backend/routes/annotate.ts:135-180

Client-Side Caching Layers

The browser extension maintains two distinct local caches to provide a near-zero latency experience for repeated visits and common navigation patterns.

1. Service Worker Session Cache

The Service Worker (SW) cache uses chrome.storage.session to store annotations in memory for the duration of the browser session. It keys data using a combination of the content_hash and the specific personality or mode requested. This ensures that different AI personalities (e.g., Terry, Jerry, Sally) do not share incorrect cached results for the same text.

• Key Structure: contentHash:mode:personality
• Storage: chrome.storage.session
• Behavior: Provides instant revisits within the same session.

Sources: extension/src/background/sw-cache.ts, extension/src/background/index.ts:250-260

2. URL Prediction Cache

The URL cache uses chrome.storage.local to map URLs to their last known content_hash and associated annotations. This allows the extension to perform "speculative rendering"—predicting what annotations should look like before the content script has even finished hashing the page text.

• Key Structure: url
• Storage: chrome.storage.local
• Benefit: Instant rendering on page load for previously visited URLs.

Sources: extension/src/background/url-cache.ts, extension/src/background/index.ts:326-330

Server-Side Caching (Database Layer)

The backend implements a centralized cache in Supabase using the annotation_cache table. This layer allows different users to benefit from annotations previously generated for the same content.

Cache Validation Logic

The backend validates cached entries against the current application state. A cache hit is only valid if:

1. The content_hash matches exactly.
2. The model_version in the cache matches the current activeModelName.
3. The prompt_version in the cache matches the version defined in backend/config/prompts.json.
4. The expires_at timestamp is in the future.

Sources: backend/routes/annotate.ts:140-165

Database Schema: annotation_cache

Field	Type	Description
content_hash	Text (Primary Key)	SHA-256 hash of the normalized text region.
intensity	Text (Primary Key)	String identifying mode and personality (e.g., overview:terry).
annotations	JSONB	The array of generated Annotation objects.
model_version	Text	The LLM model name used for generation.
prompt_version	Text	The version of the system prompt used.
expires_at	Timestamp	Expiry date, set to 30 days after creation.

Sources: backend/routes/annotate.ts:140-160, README.md:280-290

Cache Management and Invalidation

The system includes mechanisms to prevent stale data from degrading the user experience, particularly concerning user feedback and deletions.

• Tombstoning: When an annotation or feedback item is deleted, the extension adds the ID to deletedAnnotationIds or deletedFeedbackIds. This prevents the "stale-while-revalidate" fetch from re-adding the deleted item to the local UI if the server response is still being processed.
• Write-Through: Every successful LLM generation results in an automatic upsert to the database cache to ensure subsequent requests are served from the cache.
• Time-To-Live (TTL): The CACHE_TTL_DAYS constant is set to 30, after which database entries are considered expired.

Sources: extension/src/background/index.ts:51-55, backend/routes/annotate.ts:175-190, packages/shared/src/constants.ts

In-Flight Deduplication

To prevent redundant LLM calls for identical content requested simultaneously (e.g., by multiple users or rapid page refreshes), the backend uses an in-flight deduplication utility. If a request for a specific cacheKey is already processing, subsequent requests wait for the existing promise to resolve rather than starting a new LLM generation.

Sources: backend/routes/annotate.ts:170-174

Summary

The Multi-layered Caching Strategy in Oddity 1 transforms high-latency AI generation into a responsive user interface. By layering session-based memory, persistent URL predictions, and a shared database cache, the system achieves 0ms revisits for known content while maintaining data integrity through version tracking and deduplication.