Running Tests & Benchmarks
Relevant source files
The following files were used as context for generating this wiki page:
Testing and benchmarking within the Claude Context project ensure the reliability of the core indexing engine and the performance of build processes across different packages. The project utilizes a combination of automated build benchmarks, end-to-end (E2E) workflow tests, and package-specific quality checks to maintain high standards of code quality and system efficiency.
Sources: scripts/build-benchmark.js, python/test_context.ts, packages/core/package.json
Build Performance Benchmarking
The project includes a dedicated benchmarking script to measure and report build times for all core packages and extensions. This system helps developers identify performance regressions in the compilation and bundling pipelines.
Benchmark Workflow
The scripts/build-benchmark.js script executes a series of build commands and records the duration of each operation. Results are persisted to a JSON file to track historical performance.
The diagram shows the sequential execution of build tasks for performance measurement.
Sources: scripts/build-benchmark.js:56-78
Key Functions and Data Structures
| Component | Type | Description |
|---|---|---|
measureBuildTime | Function | Executes a command and returns an object containing success status, duration, and command description. |
saveBenchmark | Function | Appends current results to build-benchmark.json, maintaining a history of the last 10 runs. |
BENCHMARK_FILE | Constant | The filename (build-benchmark.json) where performance data is stored. |
Sources: scripts/build-benchmark.js:15-54
End-to-End (E2E) Testing
The project implements comprehensive end-to-end tests that simulate a complete user workflow, from configuring providers to executing semantic searches.
Workflow Logic
The E2E test validates the integration between the Context engine, embedding providers (e.g., OpenAI), and vector databases (e.g., Milvus).
The sequence diagram illustrates the flow of data during an end-to-end test.
Sources: python/test_context.ts:13-90
Test Stages
- Initialization: Configures
OpenAIEmbeddingandMilvusVectorDatabaseinstances. - Indexing: Scans the codebase, splits files using
AstCodeSplitter, and generates embeddings. - Validation: Checks if the index exists using
hasIndex. - Search: Executes a query and verifies similarity scores and content previews.
Sources: python/test_context.ts:22-68
Package-Specific Verification
Different packages within the monorepo have specialized testing and verification requirements.
Core Engine Testing
The core package uses jest and ts-jest for unit and integration testing. It also relies on mock-fs to simulate file system interactions without actual disk I/O.
Sources: packages/core/package.json:28-33
Chrome Extension Testing
The Chrome extension requires manual and automated verification across different Chromium-based browsers.
| Category | Requirements |
|---|---|
| Manual Testing | UI appearance on GitHub, indexing public repos, search relevance. |
| Cross-Browser | Chrome, Edge, Brave. |
| Debug Commands | pnpm typecheck, pnpm lint. |
Sources: packages/chrome-extension/CONTRIBUTING.md:124-150
Build Commands
Common commands used to verify package integrity:
# General type checking across all packages
pnpm typecheck
# Linting and automatic fixing
pnpm lint
pnpm lint:fix
# Run build benchmark
pnpm benchmark
Sources: package.json:11-20
Conclusion
The testing and benchmarking suite provides a multi-layered approach to project stability. By combining granular unit tests in the core package with high-level E2E tests and build performance tracking, the architecture ensures that both the functionality and the developer experience remain optimized as the codebase scales.