Testing Framework

Relevant source files

The following files were used as context for generating this wiki page:

• packages/core/src/methodology/packs/testing/red-green-refactor/SKILL.md
• packages/core/src/methodology/packs/testing/test-patterns/SKILL.md
• packages/core/src/methodology/packs/testing/anti-patterns/SKILL.md
• e2e/methodology-plan.e2e.test.ts
• tests/validate.test.ts
• packages/core/src/methodology/tests/validator.test.ts

The SkillKit testing framework is a multi-layered system designed to ensure the reliability of AI agent skills, core methodology logic, and CLI functionality. It encompasses unit testing for internal components, validation logic for skill specifications, and end-to-end (E2E) testing for command-line workflows. The framework is built on top of the Vitest testing runner and follows strict Test-Driven Development (TDD) principles to maintain high code quality and prevent regressions.

The framework is structured into distinct tiers: internal unit tests for core logic, validation suites for SKILL.md files to ensure they adhere to the Agent Skills specification, and E2E tests that simulate user interactions with the methodology and planning systems.

Sources: CONTRIBUTING.md:158-164, e2e/methodology-plan.e2e.test.ts:10-15, packages/core/src/methodology/packs/testing/red-green-refactor/SKILL.md:20-25

Test-Driven Development (TDD) Methodology

The framework enforces the Red-Green-Refactor cycle as a core principle for development. This methodology ensures that every feature or bug fix begins with a failing test, driving the design through iterative cycles of implementation and refinement.

The Red-Green-Refactor Cycle

The development workflow follows a strict three-phase process:

1. RED Phase: Developers must write a failing test before writing production code. This test defines the expected behavior and must fail for the correct reasons (logic failure, not syntax errors).
2. GREEN Phase: The developer writes the minimum amount of code required to pass the test. Simplicity is prioritized over elegance at this stage.
3. REFACTOR Phase: Once the test passes, the code is cleaned up to remove duplication and improve readability while ensuring tests remains green.

Testing Patterns and Structure

The framework utilizes standardized patterns to maintain readability and reliability across the test suite.

Arrange-Act-Assert (AAA)

The primary structure for unit tests is the Arrange-Act-Assert pattern, which separates test setup from execution and verification.

Phase	Purpose	Description
Arrange	Setup	Initialize test data, mocks, and dependencies.
Act	Execution	Trigger the specific function or method being tested.
Assert	Verification	Confirm the outcome matches the expected result.

Sources: packages/core/src/methodology/packs/testing/test-patterns/SKILL.md:20-35

Test Data Management

To reduce boilerplate, the framework employs Test Data Builders and Object Mothers. These allow for the creation of complex test objects with sensible defaults while allowing specific overrides for individual test cases.

// Example Test Data Builder usage found in methodology tests
function createTestOrder(overrides = {}) {
  return {
    id: 'order-123',
    status: 'pending',
    items: [],
    total: 0,
    ...overrides
  };
}

Sources: packages/core/src/methodology/packs/testing/test-patterns/SKILL.md:55-70

Skill Validation System

A specialized component of the testing framework is dedicated to the validation of SKILL.md files. This ensures that any authored skills are technically sound and compatible with supported AI agents.

Validation Criteria

The validation logic checks for the existence of required YAML frontmatter fields and content constraints.

• Required Fields: name (lowercase, no spaces) and description.
• Constraints: Descriptions must not exceed 1024 characters, and names must be 64 characters or fewer.
• Warnings: Triggered for short descriptions or content that is excessively long (over 600 lines).

End-to-End (E2E) Test Suite

The E2E suite validates the integration between the CLI, the core logic, and the methodology packs. It uses a custom cli-runner helper to execute commands in isolated temporary directories.

Methodology and Planning E2E

The E2E tests verify that methodologies can be listed, installed, and synced, and that plans can be generated based on these methodologies.

Command	E2E Focus	Verification Method
methodology list	Registry Discovery	Checks for output containing standard packs (agile, tdd).
methodology install	Package Acquisition	Verifies files are placed in .skillkit directory.
plan create	Plan Orchestration	Confirms JSON plan files are created with correct steps.
plan validate	Structural Integrity	Ensures plans meet methodology constraints.

Sources: e2e/methodology-plan.e2e.test.ts:40-150, e2e/helpers/cli-runner.ts:30-50

Anti-Patterns and Quality Control

The framework includes guidelines to identify "test smells" that lead to flaky or misleading results.

Critical Anti-Patterns Avoided

• The Liar: Tests that pass even when code is broken due to weak assertions (e.g., only checking toBeDefined()).
• The Inspector: Tests that rely on private implementation details rather than public behavior.
• The Mockery: Excessive mocking that results in testing the mock configuration rather than the actual logic.
• The Chain Gang: Tests that must run in a specific order, creating dependencies between suites.

Sources: packages/core/src/methodology/packs/testing/anti-patterns/SKILL.md:20-80

Test Performance Targets

The framework sets specific performance benchmarks for different test types:

• Unit Tests: Must execute in less than 100ms.
• Integration Tests: Target execution time of less than 1s.

Sources: packages/core/src/methodology/packs/testing/anti-patterns/SKILL.md:120-125

Summary

The SkillKit Testing Framework provides a comprehensive safety net for the project. By combining strict TDD methodologies with automated validation of skill specifications and robust E2E command-line testing, it ensures that the system remains stable as new AI agent adapters and methodology packs are added. The framework's reliance on standardized patterns like AAA and the rejection of common anti-patterns ensures that the test suite remains maintainable for long-term development.