Extending the Test Runner with Mock Functions

Relevant source files

The following files were used as context for generating this wiki page:

• testgen/src/components/TestRunner.tsx
• README.md
• testgen/src/pages/Dashboard.tsx
• testgen/src/lib/backend.ts
• testgen-backend/main.go
• testgen/IMPLEMENTATION.md

The TestRunner component is an advanced test execution engine responsible for running the AI-generated test cases directly in the browser. A core feature of this engine is its built-in mock function library, which simulates backend operations like database interactions, user authentication, and API calls. This allows for the immediate, self-contained execution of generated tests without requiring a live backend, database, or other external services.

This system provides a sandboxed JavaScript environment where test code can be safely executed. It validates test outcomes by comparing the actual results from the mock environment against the expected results defined in the AI-generated test case. The TestRunner is a key component of the "Execute" tab in the main application dashboard.

Sources: README.md, testgen/src/components/TestRunner.tsx, testgen/src/pages/Dashboard.tsx:216-222

Test Execution Architecture

The test execution is orchestrated by the TestRunner React component. When a user initiates a test run, the component iterates through the provided test cases and executes each one within a dynamically created, isolated function scope. This prevents tests from interfering with the main application state.

Sources: testgen/src/components/TestRunner.tsx:34-40

The executeTest Function

The heart of the test runner is the executeTest function. For each test case, it performs the following steps:

1. Records a start time for performance measurement.
2. Constructs a JavaScript code string that includes the entire library of mock functions.
3. Appends the specific code from the current test case to this string.
4. Uses the new Function() constructor to create a sandboxed execution environment. This function takes the test case's input as an argument.
5. Within this environment, it uses a "Smart Function Detection" mechanism to invoke the correct mock function.
6. Captures the result or any errors from the execution.
7. Calculates the total execution duration.
8. Returns the result, duration, and any error message.

This diagram illustrates the internal logic of the executeTest function.

Test Execution Sequence

The following diagram shows the sequence of events when a user runs tests from the dashboard.

Mock Function Library

The TestRunner contains a pre-defined set of JavaScript functions that mock common backend and utility operations. These functions are injected into the execution environment for every test.

Sources: testgen/src/components/TestRunner.tsx:47-194, README.md

Available Mock Functions

The table below details the mock functions available within the test execution environment.

Function	Description	Example Input
CreateUser(userData)	Simulates creating a user. Includes validation for input, username, password length, and duplicate usernames.	{ "Username": "test", "Password": "password123" }
Login(credentials)	Simulates user authentication. Checks for a mock user and validates the password.	{ "Username": "test", "Password": "TestPassword123!" }
GetUserProfile(currentUser)	Simulates fetching a user profile for an authenticated user.	{ "ID": 1 }
validateEmail(email)	Validates an email address format using a regular expression.	"test@example.com"
calculateSum(a, b)	A basic math utility to add two numbers.	{ "a": 5, "b": 10 }
calculateProduct(a, b)	A basic math utility to multiply two numbers.	{ "a": 5, "b": 10 }
fetchData(url)	Mocks an asynchronous API call, always resolving with mock data.	"https://api.example.com/data"
saveToDatabase(data)	Simulates saving data to a database, returning a success status and a random ID.	{ "name": "John Doe" }
deleteUser(id)	Simulates deleting a user by ID.	123
updateUser(id, data)	Simulates updating a user's data.	{ "id": 123, "name": "Jane Doe" }
getUserById(id)	Simulates fetching a single user by their ID.	123
authenticateUser(username, password)	Simulates authentication, returning a mock token for correct credentials.	{ "username": "admin", "password": "password" }

Sources: testgen/src/components/TestRunner.tsx:51-193, README.md

Smart Function Detection

To execute the correct logic for a given test case, the runner does not simply eval the code. Instead, it employs a "Smart Function Detection" mechanism to identify which mock function should be called.

1. A regular expression is used to match the name of one of the available mock functions within the test case's code property.
2. If a match is found, a switch statement is used to call the corresponding mock function with the test case's input.
3. This ensures that the correct, validated mock implementation is used, rather than relying on the AI to generate a perfect, executable function call.

// File: testgen/src/components/TestRunner.tsx:201-204
// Extract the function name from the test case code
const code = JSON.stringify(testCase.code);

// Try to find function calls in the code
const functionCallMatch = code.match(/(CreateUser|Login|...|authenticateUser)\\s*\\(/);

// File: testgen/src/components/TestRunner.tsx:207-231
if (functionCallMatch) {
  const functionName = functionCallMatch[1];
  
  // Call the appropriate function based on the test case
  switch(functionName) {
    case 'CreateUser':
      return CreateUser(input);
    // ... other cases
    default:
      return input; // Fallback
  }
}

This approach provides a robust bridge between the declarative test cases generated by the AI and the imperative code needed to execute them.

Sources: testgen/src/components/TestRunner.tsx:201-238

Extending with New Mock Functions

To extend the test runner's capabilities, a developer can add a new mock function by following these steps:

1. Add the Function Implementation: Define the new JavaScript mock function inside the mockFunctions template literal in testgen/src/components/TestRunner.tsx.
2. Update the Detection Regex: Add the new function's name to the functionCallMatch regular expression.
3. Add a case to the switch Statement: Add a new case to the switch block to handle calling the new function.
4. (Optional) Update AI Prompt: To encourage the AI to generate tests for the new function, consider updating the prompt in testgen-backend/main.go or testgen/src/lib/backend.ts to mention the new capability.

This modular design allows the test runner to easily evolve and support a wider range of test scenarios.

Sources: testgen/src/components/TestRunner.tsx:47-238, testgen-backend/main.go:420-456, testgen/src/lib/backend.ts:468-508

Summary

The TestRunner's mock function library is a critical feature that enables TestGen AI to be a self-contained, powerful tool for generating and validating test cases. By simulating a variety of backend operations within a sandboxed browser environment, it provides immediate feedback on the quality and coverage of the AI-generated tests. The "Smart Function Detection" mechanism ensures reliable execution, and the system is designed to be easily extensible with new mock capabilities as the project grows.