Context File Generation
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Context File Generation is a core feature of the TestGen AI platform, responsible for analyzing a given code repository and compiling its relevant source files into a single, structured text block. This "context" is then provided to the Google Gemini AI to generate meaningful test cases. The system is designed with two distinct implementations for this process: a Go-based backend server that clones the repository locally, and a client-side TypeScript implementation that fetches repository contents directly via the GitHub API. Both approaches aim to intelligently filter files to create a concise yet comprehensive overview of the codebase for the AI.
The primary goal is to transform a repository URL into a prompt-ready context that accurately represents the project's logic, functions, and methods. This process involves parsing the URL, fetching files, applying exclusion filters, and formatting the content into a standardized structure.
Sources: README.md:16-25, testgen-backend/main.go:310-313, testgen/src/lib/backend.ts:303-306
Architecture Overview
The system provides two parallel paths for context generation, one server-side and one client-side. The Go backend uses a traditional git clone approach, while the client-side implementation leverages browser APIs to interact with the GitHub REST API. This dual architecture offers flexibility in deployment. The live application, for instance, uses the client-side approach to simplify hosting.
Sources: README.md:16-25, testgen/CLIENT_BACKEND.md:5-11
This diagram illustrates the high-level data flow from user input to the final generated context.
Go Backend Implementation
The Go backend handles context generation via the /api/clone-repo endpoint. It performs a full, shallow clone of the specified public GitHub repository into a temporary local directory.
Sources: testgen-backend/main.go:340, testgen-backend/main.go:163-172
Data Flow
The process is orchestrated within the cloneRepoHandler function.
This flowchart shows the step-by-step process executed by the Go backend to generate a context file.
Sources: testgen-backend/main.go:340-394
Key Functions
| Function | Description | Source File |
|---|---|---|
cloneRepoHandler | The main HTTP handler that orchestrates the cloning and context generation process. | testgen-backend/main.go:340-403 |
parseGitHubURL | Parses a GitHub URL string to extract the owner and repository name. | testgen-backend/main.go:124-152 |
cloneRepository | Executes a git clone --depth 1 command to download the repository. | testgen-backend/main.go:154-172 |
readRepositoryFiles | Walks the cloned directory, reads files, and filters them based on exclusion rules and size (< 1MB). | testgen-backend/main.go:174-241 |
shouldExcludeFile | Checks if a given file path matches any of the predefined exclusion patterns. | testgen-backend/main.go:96-113 |
generatePromptContext | Assembles the content of filtered files into a single string, grouped by type (Go, Config, Other). | testgen-backend/main.go:243-294 |
Client-Side (TypeScript) Implementation
The client-side implementation, defined in testgen/src/lib/backend.ts, replicates the backend's functionality within the browser. It avoids a local git clone by fetching repository contents directly from the GitHub API. This is the approach used by the live application to enable serverless deployment.
Sources: testgen/CLIENT_BACKEND.md:5-11, testgen/src/lib/backend.ts:1-4
Sequence Diagram
This diagram shows the interaction between the user, the dashboard component, the ClientBackend service, and the GitHub API.
Key Methods in ClientBackend
The ClientBackend class encapsulates the logic for fetching and processing repository files.
| Method | Description | Source File |
|---|---|---|
cloneRepository | Orchestrates the entire process of fetching files from the GitHub API. | testgen/src/lib/backend.ts:223-298 |
parseGitHubUrl | Extracts owner and repo from a URL. | testgen/src/lib/backend.ts:100-128 |
fetchFilesRecursively | Recursively traverses repository directories via the GitHub API, fetching file content while respecting a maxFiles limit (15) and file size limit (< 50KB). | testgen/src/lib/backend.ts:150-221 |
shouldExcludeFile | Checks if a file path should be excluded based on EXCLUDE_PATTERNS. | testgen/src/lib/backend.ts:131-147 |
generatePromptContext | Formats the fetched file contents into the final context string. | testgen/src/lib/backend.ts:301-351 |
File Filtering and Selection
Both implementations use a nearly identical set of patterns to exclude irrelevant files and directories. This ensures that the generated context is focused on source code and important configuration.
Exclusion Patterns
The following patterns are used to exclude files and directories from the context.
| Pattern | Purpose |
|---|---|
node_modules, .git | Exclude dependency and version control directories. |
dist, build, coverage | Exclude build artifacts and test coverage reports. |
*.log, *.tmp | Exclude log and temporary files. |
*.min.js, *.min.css | Exclude minified assets. |
package-lock.json, yarn.lock | Exclude lock files. |
.env*, .vscode, .idea | Exclude environment and IDE-specific files. |
*.md, LICENSE, README* | Exclude documentation. |
__tests__, test, docs | Exclude testing and documentation directories. |
Sources: testgen-backend/main.go:76-94, testgen/src/lib/backend.ts:41-62, [testgen/src/lib/github.ts:12-44]
Implementation Differences
| Feature | Go Backend (main.go) | Client-Side (backend.ts) |
|---|---|---|
| Method | git clone locally | GitHub REST API |
| File Size Limit | 1 MB | 50 KB |
| File Count Limit | No hard limit | 15 files |
| Source Identification | File extension list and important filenames (Dockerfile, go.mod, etc.) | File extension list |
| Dependencies | git command-line tool | Browser fetch API |
Sources: testgen-backend/main.go:200-203, 206-231, testgen/src/lib/backend.ts:152, 172, 174
Context File Structure
The final output of the generation process is a single string formatted for the Gemini AI. The structure is consistent across both implementations. It starts with a header, followed by the content of each file, clearly demarcated.
The context groups files into logical sections: GO SOURCE FILES, CONFIGURATION FILES, and OTHER FILES to provide better organization for the AI model.
Sources: testgen-backend/main.go:243-294, [testgen/src/lib/backend.ts:301-351]
Example Format
// Source: testgen-backend/main.go:243-294
=== REPOSITORY CODE CONTEXT FOR TEST GENERATION ===
This context contains all source code files from the cloned repository.
Generate comprehensive test cases based on the functions, methods, and logic found in these files.
=== FILES ===
=== GO SOURCE FILES ===
// File: main.go
package main
import (
// ...
)
// ... file content ...
---
// File: another.go
// ... file content ...
---
=== CONFIGURATION FILES ===
// File: go.mod
module github.com/strikertushar19/testgenai/testgen-backend
go 1.21
---
=== END OF CONTEXT ===
Generate comprehensive test cases for the functions and methods found in the above code.
This snippet demonstrates the structured format of the generated context, including headers and file separators.
Sources: testgen-backend/main.go:246-293, testgen/src/lib/backend.ts:318-350
Conclusion
The Context File Generation module is a critical preparatory step in the TestGen AI workflow. By intelligently selecting, fetching, and formatting source code from a GitHub repository, it provides the Gemini AI with a clean and focused foundation to produce high-quality test cases. The dual-implementation architecture (Go backend vs. client-side TypeScript) showcases a flexible design, allowing the application to run in different environments, from a traditional client-server model to a fully serverless static deployment.