Prompt Engineering

Relevant source files

The following files were used as context for generating this wiki page:

• testgen-backend/main.go
• testgen/src/lib/backend.ts
• README.md
• testgen/src/pages/Dashboard.tsx
• testgen/IMPLEMENTATION.md
• testgen/CLIENT_BACKEND.md

Prompt engineering in TestGen AI is the systematic process of constructing a detailed and structured input for the Google Gemini AI model. The primary objective is to guide the AI to generate comprehensive, accurate, and consistently formatted test cases in JSON format. This process involves aggregating source code from a repository, combining it with explicit instructions, defining the desired output schema, and setting generation guidelines to ensure the quality and relevance of the AI-generated tests.

Prompt Architecture

The prompt sent to the Gemini API is a carefully crafted template that combines multiple components to produce the desired output. It defines the AI's role, provides the necessary code context, specifies the output format, and lists clear guidelines for the generation task.

The following diagram illustrates the high-level architecture of the prompt.

Context Generation

A critical part of the prompt is the codeContext, which contains the source code to be analyzed. This context is generated by fetching, filtering, and concatenating files from a specified GitHub repository.

Context Generation Flow

The system follows these steps to create the code context:

1. Parse URL: The GitHub URL is parsed to extract the owner and repository name.
2. Fetch Files: The repository's files are fetched. The Go backend clones the repo, while the client-side implementation uses the GitHub API.
3. Filter Files: Files are filtered to exclude irrelevant content. A predefined list of excludePatterns (e.g., node_modules, .git, *.md) is used to ignore directories and files that are not source code. File size limits are also applied.
4. Group and Format: The remaining files are grouped by type (e.g., Go, Configuration, Other) and concatenated into a single string, with each file's content preceded by a // File: <path> header.

File Grouping

The generatePromptContext function organizes the fetched files to improve the AI's understanding. The order of grouping is typically:

1. Primary language source files (e.g., .go files)
2. Configuration files (e.g., go.mod, .json, .yaml)
3. Other relevant files

This structured approach helps the model prioritize the most important code.

Sources: testgen-backend/main.go:284-302, testgen/src/lib/backend.ts:380-392

Gemini Prompt Structure

The final prompt string is a multi-part template that instructs the Gemini model on its task. Both the Go backend and the client-side TypeScript implementation use an identical prompt structure.

Sources: testgen-backend/main.go:498-539, testgen/src/lib/backend.ts:494-534

Prompt Template Breakdown

Section	Description
Role Definition	Sets the persona for the AI: You are an expert software testing engineer.
Code Context	A placeholder for the concatenated source code from the repository.
Additional Requirements	An optional placeholder for user-provided instructions from the UI, such as the repository name.
JSON Format	Explicitly defines the required JSON output structure, including a testCases array and a summary object.
Guidelines	A list of rules for the AI to follow to ensure test quality.
Final Instruction	A strict command to ensure only JSON is returned: Return only valid JSON, no additional text or markdown formatting.

Generation Guidelines

The prompt includes specific guidelines to direct the AI's test generation strategy.

#	Guideline
1	Generate comprehensive test cases covering normal cases, edge cases, and error scenarios
2	Include both positive and negative test cases
3	Test boundary conditions and edge cases
4	Include error handling tests
5	Make test names descriptive and clear
6	Ensure test inputs are realistic and meaningful
7	Focus on the main functionality of the code
8	Generate at least 5-10 test cases for good coverage

Sources: testgen-backend/main.go:529-538, testgen/src/lib/backend.ts:524-533

Example Prompt Snippet

This is the core template sent to the Gemini API.

const prompt = `
You are an expert software testing engineer. Analyze the provided code and generate comprehensive test cases.

Code Context:
${request.codeContext}

${request.additionalPrompt ? `Additional Requirements: ${request.additionalPrompt}` : ''}

Please generate test cases in the following JSON format:
{
  "testCases": [
    {
      "id": "unique_id",
      "name": "descriptive_test_name",
      "description": "detailed_description_of_what_this_test_does",
      "input": "input_data_for_the_test",
      "expected": "expected_output_or_result",
      "code": "the_function_or_code_being_tested",
      "testType": "unit|integration|edge-case|error-handling",
      "priority": "high|medium|low"
    }
  ],
  "summary": {
    "totalTests": "number",
    "unitTests": "number",
    "integrationTests": "number",
    "edgeCases": "number",
    "errorHandlingTests": "number"
  }
}

// ... (Guidelines) ...

Return only valid JSON, no additional text or markdown formatting.`;

Sources: testgen/src/lib/backend.ts:494-534

API Request and Data Flow

The generated prompt is sent to the Gemini API via a POST request. The overall flow involves the client, the application backend (either Go or client-side TS), the GitHub API, and the Gemini API.

End-to-End Data Flow

This sequence diagram shows the complete workflow from a user cloning a repository to receiving the generated test cases.

Request Parameters

The request to the backend and subsequently to the Gemini API is encapsulated in a GeminiRequest object.

Parameter	Type	Description	Source File
apiKey	string	The user's Google Gemini API key.	testgen-backend/main.go:59-63
codeContext	string	The concatenated string of source code files.	testgen-backend/main.go:59-63
additionalPrompt	string	Optional user-defined instructions.	testgen-backend/main.go:59-63

The API call also includes generation configuration to control the model's output behavior.

Parameter	Type	Value	Description
temperature	float	0.7	Controls randomness. Higher values mean more creative output.
topK	int	40	The model considers the top K most likely tokens at each step.
topP	float	0.95	The model considers tokens with a cumulative probability of P.
maxOutputTokens	int	8192	The maximum number of tokens to generate in the response.

Sources: testgen-backend/main.go:554-559, testgen/src/lib/backend.ts:544-549

Summary

The prompt engineering strategy in TestGen AI is a cornerstone of its functionality. By providing a highly structured, multi-part prompt, the system effectively constrains the powerful but non-deterministic nature of the Gemini LLM. This ensures that the generated test cases are not only relevant and comprehensive but also adhere to a strict JSON format that the application can reliably parse and utilize. The combination of role-playing, detailed context, explicit formatting rules, and clear guidelines is key to transforming raw source code into a structured and actionable test suite.