OpenRouter Model Integration

Relevant source files

The following files were used as context for generating this wiki page:

• backend/lib/openrouter.ts
• backend/config/prompts.json
• backend/api/index.ts
• backend/routes/gdocs-chat.ts
• backend/lib/tests/openrouter.test.ts
• backend/routes/gdocs-mcq.ts

The OpenRouter Model Integration provides a unified interface for interacting with Large Language Models (LLMs) through the OpenRouter API. It handles core application features including document annotation, "Sketch Pad" summary generation, PDF page summarization, and Google Docs specialized writing assistants.

The integration implements robust error handling, API key rotation, and streaming response parsing. It translates raw LLM outputs into structured application data types such as Annotation objects, while managing complex system prompts and persona overlays defined in the project configuration.
Sources: backend/lib/openrouter.ts:1-50, README.md:Key Features

Core Architecture and Data Flow

The integration acts as a middleware between the backend route handlers and the OpenRouter API. It utilizes a retry and rotation strategy to ensure high availability.

Request Execution Flow

When a feature requests content, the integration executes a retry plan across multiple API keys. If a streaming request fails after emitting partial data, the system falls back to a buffered completion request to ensure the user receives a complete result.

Reliability Mechanisms

The system tracks reliability through specific counters and implements a tiered retry logic:

• Key Rotation: Switches to BACKUP1 or BACKUP2 keys after failed attempts with the primary key.
• Backoff & Jitter: Implements exponential backoff with a base delay of 250ms and a max delay of 2000ms, plus random jitter in non-test environments.
• Validation Retries: If an LLM response fails schema validation, the system automatically sends a correction prompt to the LLM to fix the JSON structure.

Sources: backend/lib/openrouter.ts:31-45, backend/lib/openrouter.ts:480-501, backend/lib/openrouter.ts:592-612

Component Roles and Configuration

The integration supports multiple operation modes, each mapped to specific system prompts and models.

Model Configuration

Variable	Default Value	Description
OPENROUTER_MODEL	meta/muse-spark-1.3-contributor	Default model for annotations and sketches.
OPENROUTER_REASONING_MODEL	Same as default	High-capacity model for complex Google Docs edits.
OPENROUTER_ROUTER_MODEL	Same as default	Lightweight model for classifying edit intents.
OPENROUTER_TIMEOUT_MS	45000	Global timeout for API requests.

Sources: backend/lib/openrouter.ts:24-40, backend/lib/openrouter.ts:799-805

Prompt Management

System prompts are stored in backend/config/prompts.json and are dynamically injected based on the requested personality (e.g., Jerry, Terry, Sally).

• Overview Prompts: Focus on structural summaries and "skeleton" extraction.
• Depth Prompts: Use "Provocateur" personalities to surface friction and logical flaws.
• Persona Overlays: For public figure personalities, the system appends a "PERSONA THINKING OVERLAY" to the base prompt.

Sources: backend/config/prompts.json:1-150, backend/lib/openrouter.ts:133-159

Structured Output Mapping

The integration transforms raw LLM strings into typed Annotation objects used by the extension frontend.

Annotation Mapping Logic

The mapLlmOutput function standardizes raw objects into the following structure:

• Anchor: Uses TextQuoteSelector with exact, prefix, and suffix for precise document anchoring.
• Content: Contains the note or provocation generated by the model.
• Metadata: Assigns unique IDs and maps internal skill names (e.g., "caveat", "insight") to annotation types.

Google Docs Integration (GDocs)

The integration provides specialized modes for Google Docs, handled via generateGDocsChatStream.

GDocs Operating Modes

Mode	Prompt Source	Target Model	Max Tokens
chat	gdocs_chat_prompt	Default	2048
essay	gdocs_essay_prompt	Reasoning	8192
edit	gdocs_edit_prompt	Reasoning	8192
fast	gdocs_fast_prompt	Default	2048
outline	gdocs_plan_outline_prompt	Default	2048

Sources: backend/lib/openrouter.ts:792-814, backend/routes/gdocs-chat.ts:16-20

The MCQ Generator

For Google Docs writing tasks, the system generates 5 Multiple Choice Questions (MCQs) to clarify user intent. It parses model output into McqQuestion objects, ensuring each has a header (e.g., "Tone", "Audience") and 2–4 options with trade-off descriptions.
Sources: backend/lib/openrouter.ts:887-920, backend/routes/gdocs-mcq.ts:1-30

Error Classification

The classifyOpenRouterError function determines if a request should be retried based on HTTP status and error messages.

Category	Indicators	Retryable
rate_limit	HTTP 429	Yes
upstream_5xx	HTTP 500-599	Yes
timeout	AbortError / "timed out"	Yes
stream_parse	"failed to parse stream"	Yes
permanent	HTTP 401, 400, 403, 404	No
abort	signal.aborted	No

Sources: backend/lib/openrouter.ts:333-388, backend/lib/tests/openrouter.test.ts:145-155

Conclusion

OpenRouter Model Integration serves as the high-reliability bridge between the Oddity 1 backend and LLM providers. By combining aggressive retry logic with sophisticated prompt engineering and structured data mapping, it enables complex features like multi-personality annotations and intelligent document editing while maintaining system stability despite upstream API instability.