Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• cypress.config.ts
• cypress/e2e/api-team.cy.ts
• cypress/e2e/api-user.cy.ts
• cypress/e2e/api-website.cy.ts
• cypress/e2e/user.cy.ts
• cypress/e2e/website.cy.ts
• cypress/support/e2e.ts
• package.json
• scripts/seed-data.ts
• src/app/(main)/console/[websiteId]/TestConsolePage.tsx

The project employs a comprehensive testing strategy primarily centered around End-to-End (E2E) testing using Cypress. This approach covers both user interface (UI) workflows and direct API endpoint verification. The strategy is supported by a set of custom helper commands to streamline test creation and maintain consistency. Additionally, a data seeding script is available to populate the database with realistic sample data for development and testing purposes.

The testing framework is integrated into the project's workflow via npm scripts, allowing for easy execution of the test suites. While Jest is configured as a test runner, the provided files focus heavily on Cypress for E2E validation.

Tooling and Execution

The project relies on Cypress for E2E testing and Jest for other testing needs. The execution of these tests is managed through scripts defined in package.json.

Sources: package.json:30-32,41-43

Test Scripts

The following scripts are available to run the test suites:

Script	Description
test	Runs the Jest test suite.
cypress-open	Opens the Cypress Test Runner UI for interactive testing.
cypress-run	Executes all Cypress tests in headless mode.

Sources: package.json:41-43

End-to-End (E2E) Testing

E2E testing is the cornerstone of this project's quality assurance, validating complete user flows and API functionality from start to finish. Cypress is the exclusive tool used for this purpose.

UI Testing

UI tests simulate user interactions within the application to ensure that front-end components and user workflows behave as expected. Tests are organized by feature, such as user management and website management.

A typical UI test follows this flow:

1. Login: Authenticates a user for the test session using a custom cy.login() command.
2. Navigate: Visits the relevant page within the application.
3. Interact: Clicks buttons, types into input fields, and selects from dropdowns using data-test attributes for robust element selection.
4. Assert: Verifies that the UI updates correctly and displays the expected information.
5. Cleanup: Deletes any data created during the test to maintain a clean state.

The diagram below illustrates the flow for adding a new website.

Sources: cypress/e2e/website.cy.ts:9-28

API Testing

API tests directly target the application's backend endpoints to validate their functionality, including data creation, retrieval, updates, and deletion (CRUD). These tests ensure the reliability and correctness of the API layer, independent of the UI.

Tests are structured to cover the full lifecycle of a resource (e.g., User, Team, Website). A before block often handles initial setup, such as creating a prerequisite resource.

The following sequence diagram shows the API test flow for managing a user resource.

Sources: cypress/e2e/api-user.cy.ts

The API tests cover the following endpoints:

Resource	Method	Endpoint	Description
User	POST	/api/users	Creates a new user.
User	GET	/api/admin/users	Retrieves all users (admin only).
User	GET	/api/users/{userId}	Retrieves a specific user.
User	POST	/api/users/{userId}	Updates a specific user.
User	DELETE	/api/users/{userId}	Deletes a specific user.
Team	POST	/api/teams	Creates a new team.
Team	GET	/api/teams/{teamId}	Retrieves a specific team.
Team	POST	/api/teams/{teamId}	Updates a specific team.
Team	DELETE	/api/teams/{teamId}	Deletes a specific team.
Team User	POST	/api/teams/{teamId}/users	Adds a user to a team.
Team User	DELETE	/api/teams/{teamId}/users/{userId}	Removes a user from a team.
Website	POST	/api/websites	Creates a new website.
Website	GET	/api/websites	Retrieves all websites for the user.
Website	GET	/api/websites/{websiteId}	Retrieves a specific website.
Website	POST	/api/websites/{websiteId}	Updates a specific website.
Website	DELETE	/api/websites/{websiteId}	Deletes a specific website.
Website	POST	/api/websites/{websiteId}/reset	Resets a website's data.

Sources: cypress/e2e/api-user.cy.ts, cypress/e2e/api-team.cy.ts, cypress/e2e/api-website.cy.ts

Test Support and Helpers

To improve test readability and reusability, a suite of custom Cypress commands is defined. These commands encapsulate common actions like logging in, creating test data, and cleaning up.

Sources: cypress/support/e2e.ts

Custom Commands

The main custom commands are defined in cypress/support/e2e.ts.

Command	Parameters	Description
getDataTest	value: string	Selects a DOM element by its data-test attribute.
logout	-	Logs the current user out via the UI.
login	username: string, password: string	Logs a user in via an API call and caches the session for efficiency.
addWebsite	name: string, domain: string	Creates a new website via an API call.
deleteWebsite	websiteId: string	Deletes a website via an API call.
addUser	username: string, password: string, role: string	Creates a new user via an API call.
deleteUser	userId: string	Deletes a user via an API call.
addTeam	name: string	Creates a new team via an API call.
deleteTeam	teamId: string	Deletes a team via an API call.

The login command is particularly important as it uses cy.session() to cache the authentication state, speeding up test execution by avoiding repeated UI logins.

// cypress/support/e2e.ts:15-28
Cypress.Commands.add('login', (username: string, password: string) => {
  cy.session([username, password], () => {
    cy.request({
      method: 'POST',
      url: '/api/auth/login',
      body: {
        username,
        password,
      },
    })
      .then(response => {
        Cypress.env('authorization', `bearer ${response.body.token}`);
        window.localStorage.setItem('umami.auth', JSON.stringify(response.body.token));
      })
      .its('status')
      .should('eq', 200);
  });
});

Sources: cypress/support/e2e.ts:15-28

Test Data Management

A consistent approach to test data is crucial for reliable tests. This project uses a combination of Cypress fixtures for small, static data and a dedicated seeding script for generating larger, more complex datasets.

Cypress Fixtures

API tests load request bodies from fixture files (e.g., users.json, teams.json). This separates the test data from the test logic.

// cypress/e2e/api-user.cy.ts:12-13
cy.fixture('users').then(data => {
  const userCreate = data.userCreate;
  // ...
});

Sources: cypress/e2e/api-user.cy.ts:12-13

Data Seeding Script

For local development and more extensive testing, the seed-data.ts script can be used to generate realistic analytics data for demo websites. This helps in testing the application with a populated database.

The script is executed via npm run seed-data and accepts several arguments to customize its behavior.

Argument	Description	Default
--days <number>	Number of days of historical data to generate.	30
--clear	Clears existing demo data before generating new data.	false
--verbose or -v	Shows detailed progress during execution.	false
--help or -h	Displays the help message.	-

Sources: scripts/seed-data.ts:16-43

Manual Test Console

The application includes a TestConsolePage component, which serves as a dedicated UI for manually triggering various tracking events. This page is essential for debugging the analytics tracker (script.js) and verifying that different types of events (page views, custom events, identify calls, revenue tracking) are correctly processed. It contains buttons and links that call window.umami.track() and window.umami.identify() with various data payloads.

Sources: src/app/(main)/console/[websiteId]/TestConsolePage.tsx