Running End-to-End Tests

Relevant source files

The following files were used as context for generating this wiki page:

• e2e/playwright.config.ts
• e2e/helpers.ts
• frontend/package.json
• frontend/playwright-ct.config.ts
• frontend/src/tests/component-testing.ts
• apps/scratch/src/pages/tests/Edit.spec.tsx
• e2e/tests/sessions/session-analysis.spec.ts

The project utilizes the Playwright framework for its end-to-end (E2E) and component testing strategy. This ensures comprehensive testing across different browsers and user scenarios, from individual component behavior to full application workflows. The E2E test suite is designed for robustness and isolation, running a full stack environment including a dedicated backend, frontend, and a mock OpenID Connect (OIDC) provider for each test run.

This document provides a technical overview of the testing architecture, configuration, and execution flow. It covers both the full-stack E2E tests and the separate testing setups for frontend components and the embedded Scratch application.

Test Framework and Configuration

The core of the testing setup is defined in e2e/playwright.config.ts. It leverages @playwright/test and playwright-test-coverage to run tests and collect coverage information.

Key Configuration

The main configuration file orchestrates the entire test execution, including setting up the test environment, defining test projects, and specifying timeouts and retries.

Option	Description	Source
testMatch	Glob pattern for discovering test files, set to *.spec.ts.	e2e/playwright.config.ts:31
projects	Defines different test runs, including setup projects and browser-specific projects (Desktop Chrome, iPad Mini).	e2e/playwright.config.ts:34-64
timeout	A generous timeout of 120 seconds per test to accommodate for environment setup and complex interactions.	e2e/playwright.config.ts:69
webServer	The central piece of the E2E setup. It launches the entire application stack before tests begin.	e2e/playwright.config.ts:71-100
retries	Tests are retried once on CI to handle potential flakiness.	e2e/playwright.config.ts:104
use.trace	Traces are recorded on the first retry of a failed test, which is invaluable for debugging CI failures.	e2e/playwright.config.ts:105

Sources: [e2e/playwright.config.ts](https://github.com/crt25/collimator/blob/HEAD/e2e/playwright.config.ts)

Test Environment Startup

The webServer configuration in playwright.config.ts is responsible for starting all necessary services for the E2E tests. It executes the e2e-testing.ts script, which sets up a template database and starts the backend, frontend, and a mock OIDC server.

// e2e/playwright.config.ts:71-74
webServer: {
  // Run your local dev server before starting the tests.
  command: `tsx ./scripts/e2e-testing.ts`,
  url: frontendUrl,

The following diagram illustrates the components launched by the webServer.

This diagram shows the test environment startup process initiated by Playwright.

Sources: [e2e/playwright.config.ts:71-100](https://github.com/crt25/collimator/blob/HEAD/e2e/playwright.config.ts#L71-L100)

Test Execution Flow

The test execution is divided into two main phases: a one-time setup phase and the actual test execution phase, which runs in parallel workers.

1. Setup Phase

Before any tests are run, Playwright executes "setup projects". These are defined in playwright.config.ts and are dependencies for the main test projects.

1. authenticationProjectName: This project runs setup/authentication.ts to log in as predefined users (e.g., admin) and saves the authentication state. This state is stored in the template database that will be cloned by test workers.
2. setupFinishProjectName: A project that signals the end of the setup phase.

This setup phase creates a "golden" template database that serves as a clean, pre-authenticated starting point for all subsequent tests.

This diagram shows the initial setup flow before test workers are launched.

Sources: [e2e/playwright.config.ts:34-55](https://github.com/crt25/collimator/blob/HEAD/e2e/playwright.config.ts#L34-L55)

2. Test Worker Phase & Isolation

After the setup phase, Playwright starts parallel test workers. Each worker gets a completely isolated environment managed by custom fixtures in e2e/helpers.ts.

• Database Isolation: The workerConfig fixture creates a unique clone of the template database for each worker using CREATE DATABASE ... TEMPLATE .... This ensures that tests running in parallel do not interfere with each other's data. The cloned database is dropped after the worker finishes.
• Server Isolation: The workerConfig fixture also starts a new backend and frontend instance for each worker on dynamically allocated free ports.
• Test File Isolation: The _databaseSeed fixture automatically resets the worker's database to its initial cloned state whenever the test execution moves to a new file. This provides a clean slate for each test suite while avoiding the overhead of resetting the database between tests in the same file.

// e2e/helpers.ts:101-105
// create a new database based on the template database
await client.connect();
await client.query(`DROP DATABASE IF EXISTS "${uniqueDbName}"`);
await client.query(
  `CREATE DATABASE "${uniqueDbName}" TEMPLATE "${postgresConfig.database}"`,
);

The following diagram illustrates the process for a single test worker.

This diagram shows the lifecycle of a test worker, including database cloning and resetting.

Sources: [e2e/helpers.ts:79-204](https://github.com/crt25/collimator/blob/HEAD/e2e/helpers.ts#L79-L204)

Component Testing

In addition to E2E tests, the project uses Playwright for Component Testing. This allows for testing React components in isolation.

• Configuration: Component testing is configured in frontend/playwright-ct.config.ts. It uses Vite as the development server and bundler.
• Coverage: The Vite configuration is augmented with vite-plugin-istanbul to instrument the code for coverage collection.
• Coverage Collection: A custom test fixture extension in frontend/src/tests/component-testing.ts intercepts the test context to collect coverage data. It exposes a function collectIstanbulCoverage to the browser, which is called before the page unloads, and writes the coverage data to a JSON file in the .nyc_output directory.

Sources: [frontend/playwright-ct.config.ts](https://github.com/crt25/collimator/blob/HEAD/frontend/playwright-ct.config.ts), [frontend/src/tests/component-testing.ts](https://github.com/crt25/collimator/blob/HEAD/frontend/src/tests/component-testing.ts)

Scratch App Testing

The embedded Scratch application, located in apps/scratch, has a unique testing challenge because it runs within an iframe and communicates with the main application via postMessage.

The tests for the Scratch app mock this parent-child communication layer.

• Mocking postMessage: In the test.beforeEach block, the native window.parent.postMessage function is replaced with a mock that pushes messages into a window.postedMessages array.
• Simulating Parent Events: To trigger functionality within the Scratch app, tests use page.evaluate to dispatch a mocked MessageEvent, simulating a message coming from the parent window.
• Assertions: Tests then assert on the contents of the window.postedMessages array to verify that the Scratch app sent the correct response back to the parent.

// apps/scratch/src/pages/__tests__/Edit.spec.tsx:21-27
// @ts-expect-error - we mock the parent window
window.parent = {
  postMessage: (message, options) => {
    window.postedMessages.push({ message, options });
  },
};

The interaction flow is shown in the sequence diagram below.

This diagram shows how tests for the Scratch app simulate and verify postMessage communication.

Sources: [apps/scratch/src/pages/__tests__/Edit.spec.tsx:15-53](https://github.com/crt25/collimator/blob/HEAD/apps/scratch/src/pages/__tests__/Edit.spec.tsx#L15-L53)

Running Tests

The tests can be executed using yarn scripts defined in frontend/package.json.

Command	Description
yarn test	Runs both Jest and Playwright tests.
yarn test:playwright	An alias for yarn test:ct.
yarn test:ct	Runs Playwright component tests using playwright-ct.config.ts.
yarn test:ct:debug	Runs component tests in debug mode.

The full E2E tests are not directly mapped to a simple script but are run via the playwright test command within the e2e directory, which uses e2e/playwright.config.ts.

Sources: [frontend/package.json:10-22](https://github.com/crt25/collimator/blob/HEAD/frontend/package.json#L10-L22)