Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• tests/integration.test.js
• packages/cli/vitest.config.ts
• packages/webapp/playwright.config.ts
• scripts/run-integration-tests.sh
• packages/webapp/TESTING_GUIDE.md
• packages/webapp/E2E_TEST_REPORT.md
• packages/converters/src/tests/from-agents-md.test.ts
• AGENTS.md
• packages/cli/src/tests/package-scanner.test.ts
• packages/webapp/E2E_SETUP_COMPLETE.md

The prpm project employs a multi-layered testing strategy encompassing unit, integration, and end-to-end (E2E) tests to ensure code quality, correctness, and reliability across its different components, including the CLI, format converters, and the web application. The strategy leverages modern testing frameworks like Vitest for unit and integration tests and Playwright for comprehensive E2E testing of the webapp.

A core philosophy is the use of "round-trip tests" for format converters to ensure conversion quality and a goal of 100% test coverage for these critical components. The webapp's E2E testing is particularly robust, with multiple execution modes (mock API, real API, Docker) to provide flexibility for developers and CI/CD environments.

Sources: AGENTS.md, packages/webapp/TESTING_GUIDE.md

Testing Levels and Tools

The project utilizes different testing levels, each with a dedicated toolset.

This diagram shows the separation of testing concerns: Playwright for user-flow-level E2E tests and Vitest for component and function-level unit/integration tests.

Sources: packages/cli/vitest.config.ts, packages/webapp/playwright.config.ts, AGENTS.md

Level	Tool	Target Packages	Purpose
End-to-End (E2E)	Playwright	webapp	Verifies complete user flows, UI rendering, and API integration.
Integration	Vitest	cli, converters	Tests interactions between modules, like CLI commands or file scanning.
Unit	Vitest	converters, cli	Tests individual functions and components in isolation, especially format converters.

Sources: packages/webapp/playwright.config.ts, packages/cli/vitest.config.ts, AGENTS.md, packages/webapp/E2E_TEST_REPORT.md

Unit and Integration Testing (Vitest)

Unit and integration tests are built using Vitest. The configuration for the CLI package enables global setup files and specifies test environments.

Sources: packages/cli/vitest.config.ts, AGENTS.md

Format Converter Testing

A critical part of the testing strategy is ensuring the reliability of format converters. This is achieved through "round-trip" tests, where a canonical format is converted to a target format and then converted back, with the expectation that the result matches the original. The project aims for 100% test coverage for these converters.

Sources: AGENTS.md

The following diagram illustrates the round-trip testing logic for format converters.

An example from the agents.md converter tests demonstrates this pattern:

// packages/converters/src/__tests__/from-agents-md.test.ts
describe('fromAgentsMd', () => {
  // ...
  it('should parse instruction without frontmatter', () => {
    const content = `# Project Coding Guidelines
This project follows TypeScript best practices.
`;
    const result = fromAgentsMd(content, basicMetadata);
    expect(result.sourceFormat).toBe('agents.md');
    // ...
  });

  it('should infer examples section from title', () => {
    const content = `# Guide
## Examples
Here are some examples.
`;
    const result = fromAgentsMd(content, basicMetadata);
    const examplesSections = result.content.sections.filter(s => s.type === 'examples');
    expect(examplesSections.length).toBeGreaterThan(0);
  });
});

Sources: packages/converters/src/tests/from-agents-md.test.ts:10-41, packages/converters/src/tests/from-agents-md.test.ts:223-233

CLI and Package Scanner Testing

Tests for the CLI functionality, such as the package scanner, are designed to run against a temporary file system to avoid side effects. The tests create temporary directories and files, run the scanner, and assert that packages of various formats (cursor, claude, windsurf, etc.) are correctly detected.

// packages/cli/src/__tests__/package-scanner.test.ts
describe('scanForPackages', () => {
  it('should detect cursor rules in .cursor/rules/', async () => {
    // Create cursor rules directory and file
    await mkdir(join(testDir, '.cursor/rules'), { recursive: true });
    await writeFile(
      join(testDir, '.cursor/rules/typescript-rules.mdc'),
      `---
name: typescript-rules
description: TypeScript coding standards
---
# TypeScript Rules`
    );

    const packages = await scanForPackages(testDir);
    expect(packages).toHaveLength(1);
    expect(packages[0]).toMatchObject({
      name: 'typescript-rules',
      format: 'cursor',
    });
  });
});

Sources: packages/cli/src/tests/package-scanner.test.ts:29-53

End-to-End Testing (Playwright)

The webapp package has a comprehensive E2E test suite built with Playwright, covering 34 distinct test cases across major user flows.

Sources: packages/webapp/E2E_SETUP_COMPLETE.md, packages/webapp/E2E_TEST_REPORT.md

Test Suites and Coverage

The E2E tests are organized into suites that correspond to key areas of the web application.

Test Suite	# of Tests	Covered Flows
e2e/home.spec.ts	8	Hero section, feature cards, CTAs, navigation, mobile responsiveness.
e2e/authors.spec.ts	10	Leaderboard rendering, loading/error states, API mocking, stats summary.
e2e/claim.spec.ts	16	Invite claim form, token validation, OAuth simulation, success/error pages.
Total	34

Sources: packages/webapp/E2E_TEST_REPORT.md:10-48, packages/webapp/E2E_SETUP_COMPLETE.md:11-14

E2E Test Execution Modes

The E2E tests can be executed in three different modes, providing flexibility for local development and CI environments.

This diagram illustrates the three available modes for running E2E tests.

Sources: packages/webapp/TESTING_GUIDE.md:9-12, packages/webapp/E2E_SETUP_COMPLETE.md:26-44

1. Mock API Mode (Default): Uses Playwright's page.route() to intercept and mock API calls. This is the default mode, designed to be fast and reliable without external dependencies.
2. Real API Mode: Runs tests against a live registry backend. This mode is activated by setting the USE_REAL_API=true environment variable.
3. Docker Mode: Provides a fully isolated, production-like environment by running the entire stack (Postgres, Redis, Registry, Webapp) in Docker containers. This is the recommended approach for CI.

Sources: packages/webapp/TESTING_GUIDE.md:46-77, packages/webapp/E2E_SETUP_COMPLETE.md:26-44

E2E Testing Techniques

The Playwright suite employs several modern testing techniques to ensure robustness.

• API Mocking: Tests intercept network requests to provide consistent, predictable API responses for both success and error states.
• Loading State Verification: Tests explicitly wait for and verify the appearance of loading indicators to ensure the UI correctly handles asynchronous operations.
• Mobile Responsiveness: Tests are run against mobile viewports (e.g., 375x667) to confirm the layout is correct on smaller devices.

The sequence diagram below shows how API mocking is implemented.

Configuration and Execution

The Playwright configuration in packages/webapp/playwright.config.ts is set up to automatically start the web server before tests run. It also defines projects for multiple browsers and mobile devices.

// packages/webapp/playwright.config.ts
export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  webServer: {
    command: 'npm run start',
    url: 'http://127.0.0.1:3000',
    reuseExistingServer: !process.env.CI,
  },
  // ...
});

Sources: packages/webapp/playwright.config.ts

A shell script, run-docker-e2e-tests.sh, automates the process for Docker-based testing, which includes starting services, seeding the database, and running the tests.

Sources: packages/webapp/E2E_FINAL_REPORT.md:83-93, scripts/run-integration-tests.sh

Test Scripts

Several npm scripts are available to streamline the testing process.

Script	Description
npm run test:e2e	Runs all E2E tests in the default mock API mode.
npm run test:e2e:ui	Opens the Playwright UI for interactive debugging.
npm run test:e2e:real	Runs E2E tests against the real API backend.
npm run test:docker	Runs the full E2E suite in an isolated Docker environment.
npm run test:docker:down	Stops and cleans up the Docker test environment.
npm run seed:test	Seeds the test database with required data for real API mode.

Sources: packages/webapp/E2E_SETUP_COMPLETE.md:129-143, packages/webapp/TESTING_GUIDE.md:38-59

Conclusion

The testing strategy for prpm is comprehensive, covering multiple layers of the application stack with appropriate tools for each. The emphasis on round-trip testing for converters ensures data integrity, while the multi-modal E2E suite for the webapp provides high confidence in user-facing features. The use of Docker for isolated E2E testing makes the process reproducible and ideal for CI/CD pipelines, demonstrating a mature approach to quality assurance.

Sources: AGENTS.md, packages/webapp/E2E_FINAL_REPORT.md, packages/webapp/TESTING_GUIDE.md