LLM & Embedding Integrations

Relevant source files

The following files were used as context for generating this wiki page:

• src/lib/llm.ts
• src/mcp/server.ts
• src/dream/dream-cycle.ts
• src/ingestion/entities.ts
• src/ingestion/ingest-markdown.ts
• paper/drafts/cortex-v2-4-technical-note.md
• README.md

LLM & Embedding Integrations provide the core natural language processing capabilities for CORTEX. This module handles interactions with external Large Language Model (LLM) providers and embedding services to power memory ingestion, semantic search, and cognitive synthesis.

The system supports multiple providers including Anthropic, OpenAI, and OpenRouter for text generation, while defaulting to VoyageAI for high-performance vector embeddings. These integrations allow the agent to convert raw text into structured episodic memory, extract semantic meaning, and generate abstractive insights during the dream cycle.

Architecture and Provider Interface

CORTEX uses a provider-agnostic interface for LLM operations. The src/lib/llm.ts file acts as the primary gateway, abstracting the specific API requirements of different models into a unified llmComplete function.

LLM Completion Flow

When the system requires text generation (e.g., for memory summarization or entity extraction), it calls llmComplete. This function identifies the configured provider via environment variables and routes the request to the corresponding client.

This diagram shows how different CORTEX subsystems interact with the unified LLM interface to receive text responses.

Key Components:

• LLMProvider: A type defining supported services: anthropic, openai, or openrouter.
• LLMConfig: A structure containing the model name, API key, and provider.
• llmComplete: The main export that handles message formatting and API calls.

Sources: src/lib/llm.ts:16-25, src/lib/llm.ts:78-93

Embedding Integrations

Embeddings are fundamental to the CORTEX memory architecture. The system converts every ingested text chunk into a dense vector representation to enable semantic similarity matching in the hybrid search engine.

Default Embedding Configuration

CORTEX prioritizes VoyageAI (voyage-3) as its default embedding provider due to its performance on long-context retrieval benchmarks. It also supports OpenAI as a secondary option.

Feature	Configuration / Default	Source File
Primary Provider	VoyageAI (voyage-3)	paper/drafts/cortex-v2-4-technical-note.md:195
Vector Dimension	1024-dim	paper/drafts/cortex-v2-4-technical-note.md:46
Fallback Provider	OpenAI	README.md:213
Search Utility	embedQuery	src/mcp/server.ts:48
Ingestion Utility	embedTexts	src/ingestion/ingest-markdown.ts:104

Embedding Usage in Search

The search subsystem uses embeddings to calculate cosine similarity, which contributes 45% (or 50% in specific modes) to the total hybrid search score.

The sequence shows the transformation of a user query into a vector for database retrieval.

Sources: src/mcp/server.ts:48-50, paper/drafts/cortex-v2-4-technical-note.md:65-75

LLM-Powered Subsystems

Several cognitive processes within CORTEX rely on LLMs for complex reasoning and synthesis rather than simple retrieval.

Abstractive Consolidation

During the Dream Cycle, the system clusters related memories. If an LLM is available, phaseConsolidation generates an abstractive summary of these clusters to synthesize a single high-level insight from multiple episodic entries.

const llmResult = await llmComplete(
  [
    {
      role: "user",
      content: `These ${clusterNodes.length} memories form a connected cluster... Synthesize them into 1-2 sentences of actual insight.`,
    },
  ],
  { maxTokens: 150, temperature: 0.3 }
);

Sources: src/dream/dream-cycle.ts:316-330

Semantic Entity Extraction

While CORTEX includes a regex-based "fast" entity extractor, it provides an LLM-based mode for advanced Named Entity Recognition (NER). This mode resolves co-references (e.g., mapping "the CEO" to a specific name) and disambiguates aliases based on a known entity list.

Configuration: Set CORTEX_LLM_ENTITIES=true in the environment to enable this feature.

Sources: src/ingestion/entities.ts:15-39, src/ingestion/entities.ts:88-111

Configuration Options

Integration behavior is controlled through environment variables.

Variable	Type	Description
CORTEX_LLM_PROVIDER	string	Sets the LLM service (anthropic, openai, openrouter). Defaults to anthropic.
CORTEX_LLM_MODEL	string	Specific model ID (e.g., claude-sonnet-4-6-20250514).
VOYAGE_API_KEY	string	Required for default embedding operations.
CORTEX_LLM_ENTITIES	boolean	Enables LLM-based entity extraction during ingestion.

Sources: src/lib/llm.ts:27-35, src/ingestion/entities.ts:41, src/mcp/server.ts:333-334

Summary

The LLM & Embedding Integrations module serves as the linguistic and semantic foundation of CORTEX. By wrapping industry-standard APIs into a unified interface, CORTEX performs high-dimensional vector search for memory retrieval and uses advanced generative models for cognitive maintenance tasks like consolidation and entity resolution. This architecture ensures the system can adapt to new model releases with minimal changes to the core cognitive logic.