Core Engine API Reference
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Core Engine (@zilliz/claude-context-core) serves as the central processing unit for the Claude Context ecosystem. It provides the essential logic for codebase indexing, semantic search, and incremental synchronization, enabling AI agents to access deep codebase context through vector embeddings and hybrid search techniques.
This engine is designed with a pluggable architecture, supporting multiple embedding providers (OpenAI, VoyageAI, Gemini, Ollama) and utilizing Milvus/Zilliz Cloud as the primary vector database backend. It handles complex tasks such as Abstract Syntax Tree (AST) based code splitting and Merkle tree-based file synchronization to ensure efficient and accurate code retrieval.
Sources: packages/core/README.md, README.md:244-255
Core Architecture and Data Flow
The engine operates by transforming raw source code into searchable vector representations. The flow involves file scanning, intelligent chunking, embedding generation, and storage in a vector database.
High-Level Data Flow
The following diagram illustrates how the core engine processes a codebase directory from initial scan to searchable index.
The system utilizes a Context class as the primary entry point to orchestrate these components.
Sources: packages/core/README.md:129-140, packages/mcp/src/index.ts:50-70
Key Components and APIs
The Context Class
The Context class is the main orchestrator. It manages the lifecycle of indexing and searching.
| Method | Description | Parameters |
|---|---|---|
indexCodebase | Indexes a directory for hybrid search. | path, progressCallback?, forceReindex? |
semanticSearch | Searches indexed code using natural language. | path, query, topK?, threshold?, filterExpr? |
reindexByChange | Incrementally re-indexes modified files only. | path, progressCallback? |
clearIndex | Removes the search index for a specific path. | path, progressCallback? |
Sources: packages/core/README.md:112-124, packages/mcp/src/handlers.ts:133-145
Code Splitting (Splitters)
The engine supports two primary splitting strategies to break down code into manageable chunks while preserving context.
- AST Code Splitter (Default): Uses Tree-sitter to analyze the logical structure (functions, classes) of the code.
- LangChain Code Splitter: A character-based fallback or alternative for generic text chunking.
Embedding Providers
The engine abstracts embedding generation through a common interface, allowing various providers to be used.
| Provider | Environment Variable | Default Model |
|---|---|---|
| OpenAI | OPENAI_API_KEY | text-embedding-3-small |
| VoyageAI | VOYAGEAI_API_KEY | voyage-code-3 |
| Gemini | GEMINI_API_KEY | gemini-embedding-001 |
| Ollama | OLLAMA_HOST | nomic-embed-text |
Sources: packages/core/README.md:88-93, packages/mcp/src/index.ts:55-60
File Synchronization System
The core engine implements an intelligent synchronization system using Merkle trees to avoid redundant processing of unchanged files.
Merkle Tree Sync Process
The system computes SHA-256 hashes of file contents and organizes them into a tree.
Configuration Options
The ContextConfig interface defines the behavior of the engine during instantiation.
| Property | Type | Default / Description |
|---|---|---|
embedding | Embedding | Pluggable embedding provider instance. |
vectorDatabase | VectorDatabase | Milvus/Zilliz database instance (Required). |
codeSplitter | Splitter | Strategy for chunking (AST or LangChain). |
supportedExtensions | string[] | List of extensions to index (e.g., .ts, .py). |
ignorePatterns | string[] | Glob patterns for files to exclude. |
Sources: packages/core/README.md:104-110, examples/basic-usage/index.ts:52-56
Code Implementation Example
Initialization of the core engine typically involves configuring the embedding and vector database layers.
// Sources: [packages/core/README.md:41-61]
import { Context, OpenAIEmbedding, MilvusVectorDatabase } from '@zilliz/claude-context-core';
const embedding = new OpenAIEmbedding({
apiKey: process.env.OPENAI_API_KEY,
model: 'text-embedding-3-small'
});
const vectorDatabase = new MilvusVectorDatabase({
address: process.env.MILVUS_ADDRESS || 'localhost:19530',
token: process.env.MILVUS_TOKEN
});
const context = new Context({ embedding, vectorDatabase });
Summary
The Core Engine API provides a robust framework for managing codebase intelligence. By combining AST-based structural analysis, diverse embedding support, and high-performance vector storage via Milvus, it enables precise semantic retrieval. Its synchronization logic ensures that even large-scale projects can be indexed efficiently with minimal overhead.