Streaming Chatbot Integration

Relevant source files

The following files were used as context for generating this wiki page:

• extension/src/content/chat-observer.ts
• extension/src/background/api-client.ts
• backend/lib/openrouter.ts
• CHANGELOG.md
• extension/src/background/index.ts
• tasks/todo.md

Streaming Chatbot Integration enables the Oddity 1 extension to detect, track, and annotate live responses from Large Language Model (LLM) interfaces. The system monitors the Document Object Model (DOM) for incremental text updates and processes message segments in real-time.

This feature supports specific adapters for platforms like ChatGPT and Claude. It ensures that annotations appear progressively as the chatbot completes message segments, rather than waiting for the entire response to finish. Sources: CHANGELOG.md:23-26, tasks/todo.md:79-84

System Architecture

The integration spans the content script, the background service worker, and the backend API. Each layer handles a specific stage of the streaming data lifecycle.

Data Flow Overview

The following diagram illustrates the sequence of events from detecting a chatbot response to rendering an annotation.

The sequence shows how the system maintains an active Server-Sent Events (SSE) connection to provide low-latency updates. Sources: extension/src/background/api-client.ts:153-228, extension/src/background/index.ts:251-297

Content Detection and Observation

The ChatObserver class monitors the chatbot interface for new message regions and completion signals.

Site Adapters

Oddity 1 uses site-specific logic to determine when a streaming response is complete.

Site	Detection Method	Completion Signal
ChatGPT	Streaming Adapter	Appearance of copy or edit buttons.
Claude	Streaming Adapter	Specific class markers and completion signals.
General	Fallback	MutationObserver debounce for non-streaming sites.

Sources: CHANGELOG.md:23-28, tasks/todo.md:44-48

Observation Logic

The content script uses a MutationObserver to watch for DOM changes within the chat container. It snapshots pre-existing elements and treats new elements as active streams. The system uses a WeakSet of tracked elements to prevent redundant processing. Sources: extension/src/content/chat-observer.ts, tasks/lessons.md:7-14

Background Communication and API Processing

The background service worker orchestrates the communication between the browser tab and the backend.

API Client and SSE

The api-client.ts module manages the HTTP connection to the backend. It uses fetch with the text/event-stream accept header to initiate a Server-Sent Events (SSE) stream.

1. The client acquires a concurrency slot for the tab to prevent rate limit bursts.
2. It reads the response body using a ReadableStreamDefaultReader.
3. It decodes incoming bytes and parses data: prefixed JSON lines.
4. Each valid annotation object is passed to an onAnnotation callback.

Sources: extension/src/background/api-client.ts:153-228, extension/src/background/index.ts:27-46

Rate Limiting and Error Handling

The system implements per-tab concurrency limits and global rate limiting gates. If the API returns a 429 status, the background script sets a rateLimitUntil timestamp and blocks subsequent requests for that period. Sources: extension/src/background/api-client.ts:184-192, extension/src/background/index.ts:33-41

Backend Stream Generation

The backend uses OpenRouter to generate annotations. It processes the text and emits chunks as they become available from the LLM.

Chunk Extraction

The backend implements an extractCompleteObjects function to identify valid JSON objects within a continuous text buffer. This allows the system to emit individual annotations as soon as the LLM completes the JSON structure for a single item.

This logic prevents the system from attempting to parse partial JSON fragments. Sources: backend/lib/openrouter.ts:743-802

LLM Reliability and Fallbacks

OpenRouter integration includes a retry plan that rotates through backup API keys if rate limits or 5xx errors occur. If a stream fails after emitting partial data, the system can fall back to a buffered generation method to ensure the complete set of annotations is provided. Sources: backend/lib/openrouter.ts:804-878, backend/lib/tests/openrouter.test.ts:114-142

Progressive Rendering

In the content script, the UI renders annotations immediately upon receipt of the annotationReady message. This progressive approach ensures the user sees insights while the chatbot is still responding.

• Real-Time Annotation: Annotations appear automatically as messages complete streaming.
• Stale Request Cancellation: The system uses AbortController to cancel in-flight requests if the user navigates away or triggers a new request for the same region.

Sources: CHANGELOG.md:23-28, extension/src/background/index.ts:19-25, tasks/lessons.md:16-20

Conclusion

Streaming Chatbot Integration reduces the perceived latency of AI annotations by parallelizing detection, generation, and rendering. By leveraging site-specific adapters and SSE-based communication, Oddity 1 provides a seamless experience on highly dynamic chat interfaces.