Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• jest.config.ts
• src/__tests__/dentate-gyrus.test.ts
• src/__tests__/drizzle-array-cast.test.ts
• src/__tests__/entities.test.ts
• benchmarks/cogbench/run.ts
• benchmarks/cogbench/scoring.ts
• README.md

CORTEX employs a multi-layered testing strategy to verify its synthetic cognition infrastructure. This strategy combines traditional unit testing for core algorithms with a specialized cognitive benchmark suite named CogBench to evaluate high-level architectural behaviors.

The project uses Jest for unit testing and TypeScript for type safety across all test suites. Testing focus areas include hippocampal encoding, entity extraction, database integrity, and cognitive performance metrics.

Sources: README.md:374-378, jest.config.ts:1-20

Unit Testing Framework

CORTEX utilizes Jest as its primary test runner to validate individual components and algorithms. The configuration supports TypeScript via ts-jest and handles ESM modules through specific transformations.

Test Environment Configuration

The testing environment is configured with the following parameters:

• Root Directory: Projects are defined within the root directory.
• Module Mapping: Paths are mapped to ensure correct resolution of internal modules.
• Transformations: ts-jest processes .ts and .tsx files to enable modern JavaScript features in a Node.js environment.

Sources: jest.config.ts:1-20

Component-Level Validation

Unit tests target critical cognitive subsystems to ensure mathematical and logical correctness.

Component	Test Focus	Source File
Dentate Gyrus	Sparsity, determinism, normalization, and pattern separation.	src/__tests__/dentate-gyrus.test.ts
Entity Extraction	Accuracy of entity identification from text chunks.	src/__tests__/entities.test.ts
Database Casts	Correct serialization of number arrays to PostgreSQL integer arrays.	src/__tests__/drizzle-array-cast.test.ts

Sources: README.md:374-378, src/__tests__/dentate-gyrus.test.ts:1-50

CogBench: Cognitive Benchmarking

CogBench is a specialized evaluation harness that measures the performance of cognitive capabilities rather than simple retrieval accuracy. It runs scenarios against a live CORTEX instance to compute task-specific metrics and a composite score.

CogBench Architecture

The benchmark harness operates by loading pre-generated datasets and executing them through a task registry.

The diagram shows the flow of data from the CogBench dataset through specific task evaluators to the final scoring engine.
Sources: benchmarks/cogbench/run.ts:1-50

Evaluation Tasks

CogBench evaluates seven distinct cognitive capabilities:

1. Temporal Validity: Measures precision and recall for time-scoped memories.
2. Reconsolidation: Evaluates belief updates and labile window compliance.
3. Novelty Detection: Uses AUC to classify novel vs. redundant information.
4. Emotional Recall: Measures the recall advantage of salient memories.
5. Cross-Agent Transfer: Validates knowledge transfer between agent instances.
6. Compounding Intelligence: Measures improvement gained through dream cycles.
7. Procedural Learning: Tracks skill proficiency accuracy.

Sources: benchmarks/cogbench/scoring.ts:65-104

Scoring and Analytics

The scoring engine aggregates individual scenario results into task-level metrics and a final composite score.

Metrics Calculation

• Mean and Median Scores: Calculated for every task based on scenario pass/fail rates.
• Composite Score: Uses the geometric mean of all task scores. This prevents a single high-performing area from masking failures in other cognitive capabilities.
• Confidence Intervals: Employs a bootstrap method (default 1000-2000 resamples) to determine the 95% confidence interval for results.

Sources: benchmarks/cogbench/scoring.ts:112-145

Result Formatting

Results are output in both human-readable Markdown tables and a machine-readable results.json file. The output includes:

• Pass rates per task.
• Timing data (total and per task).
• Specific metrics like noveltyAUC or temporalPrecision.

Sources: benchmarks/cogbench/run.ts:180-210, benchmarks/cogbench/scoring.ts:160-200

Execution Procedures

Running Unit Tests

To execute the unit test suite, use the following command:

1. Open a terminal in the project root.
2. Run npm test.

Sources: README.md:374-375

Running CogBench

To run the cognitive benchmark suite:

1. Ensure the PostgreSQL database and embedding services are active.
2. Generate the dataset using npx tsx benchmarks/cogbench/generate.ts.
3. Execute the harness: npx tsx benchmarks/cogbench/run.ts.

Sources: benchmarks/cogbench/run.ts:1-20

CORTEX's testing strategy ensures that the system maintains cognitive integrity through both discrete algorithm validation and holistic behavioral benchmarking. This dual approach is critical for a system grounded in computational neuroscience.