Temporary File Management

Relevant source files

The following files were used as context for generating this wiki page:

• testgen-backend/main.go
• README.md
• testgen/src/lib/backend.ts
• testgen/src/pages/Dashboard.tsx
• testgen/CLIENT_BACKEND.md
• testgen/src/lib/github.ts

Introduction

Temporary file management in the TestGenAI project is a critical backend process for handling external source code. When a user provides a GitHub repository URL, the system needs a transient storage mechanism to clone the repository, analyze its contents, and generate a consolidated context file for the AI. This process involves creating temporary directories and files, which are then cleaned up after the context has been successfully generated and stored.

The project features two distinct approaches to this problem: a Go backend that uses the local filesystem for temporary storage, and a client-side TypeScript implementation that manages file data in-memory. This document details the architecture and workflow of both implementations.

Sources: testgen-backend/main.go, README.md:31, testgen/CLIENT_BACKEND.md:1-12

Backend (Go) Implementation

The Go backend provides a server-side solution for managing temporary files. It uses a dedicated repos/ directory on the local filesystem to temporarily store cloned repositories and the generated context files.

Sources: testgen-backend/main.go, README.md:31

Directory Structure and Initialization

On startup, the Go application ensures that a directory named repos exists in its root. This directory serves as the primary location for all temporary file operations.

// testgen-backend/main.go:616-618
func main() {
	// Create repos directory
	if err := os.MkdirAll("repos", 0755); err != nil {
		log.Fatal("Failed to create repos directory:", err)
	}
//...

Sources: testgen-backend/main.go:616-618

The structure within the repos directory is dynamic:

Path	Description	Persistence
repos/	The root directory for all temporary data.	Created on server start.
repos/<owner>-<repo>/	A temporary directory where a specific GitHub repository is cloned.	Ephemeral (deleted after processing).
repos/<owner>-<repo>-context.txt	The final context file containing the concatenated source code for the AI.	Persists until the server is restarted or the file is manually deleted.

Sources: testgen-backend/main.go:377-378, 396-397, 408

Repository Cloning and Processing Workflow

The entire process is handled by the cloneRepoHandler function, which is triggered by a POST request to the /api/clone-repo endpoint.

This diagram illustrates the lifecycle of temporary files and directories in the Go backend.

1. Create Clone Path: A unique path for the repository is created inside the repos/ directory, formatted as repos/<owner>-<repo>.
   Sources: testgen-backend/main.go:377-378
2. Clone Repository: The cloneRepository function executes a git clone --depth 1 command to download the repository into the designated temporary path. Any existing directory at that path is removed first.
   Sources: testgen-backend/main.go:170-182, 382
3. Read Files: The readRepositoryFiles function recursively walks the cloned directory, filtering for relevant source files and excluding ignored patterns and files larger than 1MB.
   Sources: testgen-backend/main.go:184-247, 389
4. Generate and Save Context: The content of the filtered files is concatenated into a single string by generatePromptContext and saved to repos/<owner>-<repo>-context.txt.
   Sources: testgen-backend/main.go:249-301, 396-402
5. Cleanup: After the context file is successfully written, the entire cloned repository directory (repos/<owner>-<repo>/) is deleted using os.RemoveAll. The smaller context.txt file is retained for potential retrieval via the /api/context/{owner}/{repo} endpoint.
   Sources: testgen-backend/main.go:408

Sequence Diagram: /api/clone-repo

This sequence diagram shows the interactions between the client, the Go backend, and the filesystem during the cloning process.

Client-Side (TypeScript) Implementation

As an alternative to the Go backend, the project also includes a client-side implementation that performs repository analysis entirely within the browser. This approach avoids the need for a local filesystem and instead manages file data in-memory.

Sources: testgen/src/lib/backend.ts, testgen/CLIENT_BACKEND.md

In-Memory File Handling

Instead of cloning a repository to disk, the ClientBackend.cloneRepository method uses the GitHub API to fetch a list of files and their contents directly.

1. Fetch File Tree: It recursively calls the GitHub Contents API to get the repository's file structure.
2. Filter and Fetch Content: It filters files based on exclusion patterns, file extensions, and size limits (under 50KB). For each valid file, it fetches its content.
3. Store in Memory: The fetched files, including their path, content, and size, are stored in an array of FileContent objects in the application's state.

This diagram shows the client-side data flow, where "temporary files" exist only as JavaScript objects.

The FileContent interface defines the structure for these in-memory files.

// testgen/src/lib/backend.ts:9-14
export interface FileContent {
  path: string;
  content: string;
  size: number;
}

Sources: testgen/src/lib/backend.ts:9-14

Fallback Mechanism

If the client-side logic fails to find any suitable source files in a repository, the createFallbackResponse function generates a temporary, hardcoded example file (example.js). This ensures that the application can still proceed with a demonstration even if a repository is empty or contains no supported file types. This fallback file exists only in memory and is never written to disk.

Sources: testgen/src/lib/backend.ts:392-441, testgen/src/lib/github.ts:303-352

Comparison of Implementations

Feature	Go Backend	Client-Side (TypeScript)
Storage Location	Local filesystem (repos/ directory)	Browser memory (JavaScript objects)
Primary Tool	git clone command	GitHub REST API (fetch)
File Source	Cloned repository on disk	Direct API calls to api.github.com
Cleanup	os.RemoveAll deletes the cloned repo dir	Garbage collection (when state changes)
Persistent Artifact	...-context.txt file	None (context is held in component state)
Limitations	Requires git installed on the server	Subject to GitHub API rate limits and CORS

Sources: testgen-backend/main.go, testgen/src/lib/backend.ts, testgen/CLIENT_BACKEND.md:28-32

Summary

The TestGenAI project employs two effective but different strategies for temporary file management. The Go backend uses a traditional server-side approach with a temporary directory on the filesystem, offering robustness and independence from external API rate limits during file processing. In contrast, the client-side TypeScript implementation provides a serverless alternative by leveraging the GitHub API and managing file data entirely in-memory, simplifying deployment at the cost of being subject to API and browser limitations. Both systems effectively isolate the temporary data, ensuring that only the essential generated context persists for use by the AI.