Supporting New Programming Languages
Relevant source files
The following files were used as context for generating this wiki page:The TestGenAI system is designed to analyze code from various programming languages to generate test cases. Support for a language is primarily determined by the system's ability to identify, read, and process its source files into a context for the AI model. The architecture is flexible, allowing for new languages to be added by modifying configuration lists within the source code.
This document outlines the current language support and provides a guide for extending it. The system has two parallel implementations for fetching and processing repository files: a Go backend and a client-side TypeScript implementation. Both must be considered when adding a new language.
Architecture for Language Support
The project contains two distinct backends that handle repository analysis. Both implementations rely on filtering files by their extension to determine which files to include in the AI's context.
This diagram shows the two parallel paths for processing a repository. A request from the UI can be handled by either the Go backend via an API call or directly in the browser using the TypeScript ClientBackend. Both paths result in a context string sent to the AI.
Sources: testgen-backend/main.go, testgen/src/lib/backend.ts, testgen/src/pages/Dashboard.tsx
File Identification and Filtering
The core of language support lies in identifying relevant source files within a repository. This is achieved by checking file extensions against a predefined list of supported extensions.
Go Backend (testgen-backend/main.go)
The Go backend uses a slice of strings named sourceExts to identify source code files.
Supported Extensions (Go Backend):
| Category | Extensions |
|---|---|
| Web & Scripting | .js, .jsx, .ts, .tsx, .php, .rb, .py, .sh, .bat, .ps1 |
| Compiled | .java, .cpp, .c, .cs, .go, .rs, .swift, kt |
| Frontend Frameworks | .vue, .svelte |
| Markup & Styling | .html, .css, .scss, .sass, .less |
| Configuration | .json, .yaml, .yml, .toml, .ini, .env |
| Database | .sql |
Sources: testgen-backend/main.go:221-222
The filtering logic is implemented in the readRepositoryFiles function.
// testgen-backend/main.go:220-228
ext := strings.ToLower(filepath.Ext(path))
sourceExts := []string{".js", ".jsx", ".ts", ".tsx", ".py", ".java", ".cpp", ".c", ".cs", ".php", ".rb", ".go", ".rs", ".swift", ".kt", ".vue", ".svelte", ".html", ".css", ".scss", ".sass", ".less", ".json", ".yaml", ".yml", ".toml", ".ini", ".env", ".sql", ".sh", ".bat", ".ps1"}
isSourceFile := false
for _, sourceExt := range sourceExts {
if ext == sourceExt {
isSourceFile = true
break
}
}
Sources: testgen-backend/main.go:220-228
Client-Side Backend (testgen/src/lib/backend.ts)
The client-side TypeScript backend mirrors the Go backend's logic for identifying source files within the fetchFilesRecursively method.
Supported Extensions (Client-Side Backend):
The list of extensions in backend.ts is identical to the one in the Go backend.
| Category | Extensions |
|---|---|
| Web & Scripting | .js, .jsx, .ts, .tsx, .py, .php, .rb, .sh, .bat, .ps1 |
| Compiled | .java, .cpp, .c, .cs, .go, .rs, .swift, kt |
| Frontend Frameworks | .vue, .svelte |
| Markup & Styling | .html, .css, .scss, .sass, .less |
| Configuration | .json, .yaml, .yml, .toml, .ini, .env |
| Database | .sql |
Sources: testgen/src/lib/backend.ts:173-174
// testgen/src/lib/backend.ts:172-175
const ext = item.name.split('.').pop()?.toLowerCase() || '';
const isSourceFile = ['.js', '.jsx', '.ts', '.tsx', '.py', '.java', '.cpp', '.c', '.cs', '.php', '.rb', '.go', '.rs', '.swift', '.kt', '.vue', '.svelte', '.html', '.css', '.scss', '.sass', '.less', '.json', '.yaml', '.yml', '.toml', '.ini', '.env', '.sql', '.sh', '.bat', '.ps1'].includes('.' + ext);
const isNotExcluded = !this.shouldExcludeFile(itemPath);
const isReasonableSize = item.size < 50000; // Less than 50KB
Sources: testgen/src/lib/backend.ts:172-175
Discrepancy in github.ts
An older or alternative file fetching implementation exists in testgen/src/lib/github.ts. This file contains a more limited list of source file extensions. While the main application flow appears to use backend.ts, this file is important to be aware of.
Supported Extensions (github.ts):
.js, .jsx, .ts, .tsx, .py, .java, .cpp, .c, .cs, .php, .rb, .go, .rs, .swift, .kt
Sources: testgen/src/lib/github.ts:105-107
Context Generation
After identifying the relevant files, the system concatenates their content to form a single context string. Both the Go and TypeScript backends give special treatment to Go files by grouping them under a === GO SOURCE FILES === header, indicating a potential focus on the Go language.
The following diagram shows the process of generating the context string.
This flowchart illustrates how files are categorized and appended to the context, with special priority given to Go files.
Sources: testgen-backend/main.go:261-313, testgen/src/lib/backend.ts:353-405
How to Add Support for a New Language
To add support for a new programming language, for example "Rust" with the .rs extension (which is already present but serves as a good example), you would need to ensure its file extension is included in the source file lists.
Step 1: Update Go Backend
Modify the sourceExts slice in testgen-backend/main.go to include the new extension.
// File: testgen-backend/main.go
// Add ".newlang" to the list
sourceExts := []string{".js", ".jsx", ".ts", ".tsx", ".py", ".java", ".cpp", ".c", ".cs", ".php", ".rb", ".go", ".rs", ".swift", ".kt", ".vue", ".svelte", ".html", ".css", ".scss", ".sass", ".less", ".json", ".yaml", ".yml", ".toml", ".ini", ".env", ".sql", ".sh", ".bat", ".ps1", ".newlang"}
Sources: testgen-backend/main.go:221-222
Step 2: Update Client-Side Backend
Modify the isSourceFile check in testgen/src/lib/backend.ts to include the new extension.
// File: testgen/src/lib/backend.ts
// Add ".newlang" to the array
const isSourceFile = ['.js', /* ... */, '.ps1', '.newlang'].includes('.' + ext);
Sources: testgen/src/lib/backend.ts:173-174
Step 3: (Optional) Update github.ts
For consistency, update the isSourceFile check in testgen/src/lib/github.ts as well.
// File: testgen/src/lib/github.ts
// Add the new extension
const isSourceFile = ['.js', /* ... */, '.kt', '.newlang'].includes(
item.path.split('.').pop()?.toLowerCase() || ''
);
Sources: testgen/src/lib/github.ts:105-107
Step 4: (Optional) Add Language-Specific Context Grouping
If the new language requires special grouping similar to Go, you can update the generatePromptContext function in both testgen-backend/main.go and testgen/src/lib/backend.ts to handle it.
Summary
The TestGenAI platform supports a wide array of programming languages out-of-the-box. Its architecture, which relies on simple file extension filtering, makes it straightforward to extend support for new languages. By updating the extension lists in both the Go and TypeScript backend files, developers can easily enable the system to analyze and generate tests for additional languages. While code analysis is broadly supported, it is important to note that the actual test execution environment is currently limited to JavaScript/TypeScript.
Sources: README.md:241, testgen/IMPLEMENTATION.md:65