File & Batch Generation

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_tests/_utilities/_files/_batch.py
• src/flext_tests/_utilities/_files/_creation_parts/creation_part_02.py
• src/flext_tests/_utilities/_files/_creation_parts/creation_part_03.py
• src/flext_tests/_models/_filesystem_parts/filesystem_part_01.py
• src/flext_tests/_utilities/_files/_info.py
• tests/unit/_files_parts/batch_create_in.py

The File and Batch Generation system in flext-tests provides a robust infrastructure for creating, managing, and inspecting filesystem artifacts during testing. It supports automated format detection, complex data serialization (including Pydantic models and results), and batch operations with configurable error handling.

This system is primarily exposed through the FlextTestsFiles utility group, which leverages mixins to handle specific concerns such as single file creation, batch processing, and comprehensive metadata extraction via FileInfo models.

Sources: src/flext_tests/_utilities/_files/_batch.py, src/flext_tests/_utilities/_files/_creation_parts/creation_part_03.py, src/flext_tests/_models/_filesystem_parts/filesystem_part_01.py

Core File Creation Logic

The primary entry point for individual file generation is the create method. It accepts raw content or Result wrappers and automatically determines the target format and serialization strategy based on the content type or file extension.

Content Serialization & Format Detection

The system uses a multi-step process to transform high-level Python objects into disk-level bytes. It supports str, bytes, dict (JSON/YAML), list (CSV), and Pydantic models.

The diagram shows the logical flow from high-level input to physical file writing, including the intermediate payload normalization phase.
Sources: src/flext_tests/_utilities/_files/_creation_parts/creation_part_03.py:22-86, src/flext_tests/_utilities/_files/_creation_parts/creation_part_02.py:23-57

Creation Parameters

File creation is governed by the CreateParams model, which defines how the file should be written and handled.

Parameter	Type	Default	Description
content	FileContentPlain	N/A	The raw data to be written.
name	str	test.txt	The filename (normalized by stripping whitespace).
directory	Path | None	None	Target directory for the file.
fmt	FileFormat	auto	Explicit format override (json, yaml, csv, bin, txt).
enc	str	utf-8	File encoding.
indent	int	2	Indentation for JSON/YAML files.
readonly	bool	False	If true, applies read-only permissions after creation.

Sources: src/flext_tests/_models/_filesystem_parts/filesystem_part_01.py:91-125, src/flext_tests/_utilities/_files/_creation_parts/creation_part_03.py:22-35

Batch Operations

The batch_files method allows for performing bulk operations—specifically CREATE, READ, and DELETE—on a collection of files. This is particularly useful for setting up complex test environments involving multiple configuration files.

Batch Execution Logic

Batch processing involves iterating through an items mapping or sequence and applying a specific operation to each. It features a configurable ErrorMode.

• STOP: Immediately fails the batch and returns an error result if any operation fails.
• COLLECT: Continues processing all items, aggregating successes and errors into a final BatchResult summary.

The sequence illustrates the batch processing loop where individual operation failures are handled based on the configured error mode.
Sources: src/flext_tests/_utilities/_files/_batch.py:21-118, tests/unit/_files_parts/batch_create_in.py:35-46

Batch Result Structure

The outcome of a batch operation is encapsulated in the BatchResult model, providing metrics and per-file status.

Field	Type	Description
total	int	Total number of files processed.
succeeded	int	Count of successful operations.
failed	int	Count of failed operations.
results	dict[str, Result]	Mapping of filename to individual operation results.
errors	dict[str, str]	Mapping of filenames to specific error messages.

Sources: src/flext_tests/_utilities/_files/_batch.py:110-117, tests/unit/_files_parts/batch_create_in.py:48-56

File Inspection & Metadata

Beyond creation, the system provides an info() utility to extract comprehensive metadata from files on disk. This is used to verify the state of the filesystem after operations.

FileInfo Model

The FileInfo object contains technical details about a file, including its size, line count (for text), detected format, and security hashes.

# Model Definition Summary
class FileInfo(m.Value):
    exists: bool
    path: Path | None
    size: int
    lines: int
    encoding: str
    fmt: str
    sha256: str | None
    content_meta: ContentMeta | None

Sources: src/flext_tests/_models/_filesystem_parts/filesystem_part_01.py:19-61

Content Metadata Extraction

When parse_content=True is passed to info(), the system performs format-specific parsing to extract internal statistics:

• JSON/YAML: Returns key_count (for dicts) or item_count (for lists).
• CSV: Returns row_count and column_count.
• Pydantic: Validates the content against a provided validate_model and reports model_valid status.

Sources: src/flext_tests/_utilities/_files/_info.py:102-156, src/flext_tests/_models/_filesystem_parts/filesystem_part_01.py:64-89

Summary

The File & Batch Generation module provides a high-level abstraction for filesystem interactions in tests. By automating format detection and providing structured results for both single and batch operations, it ensures that test setup is both expressive and verifiable. The integration with FileInfo allows for deep inspection of generated artifacts, including content-level metadata and schema validation.