Image Preprocessing Flow

Relevant source files

The following files were used as context for generating this wiki page:

• mb_capcha_ocr/predict.py
• train_model/train.py
• train_model/core.py
• README.md
• mb_capcha_ocr/__init__.py

The Image Preprocessing Flow in this project is a critical pipeline designed to transform raw CAPTCHA images into a standardized format suitable for both training a ResNet-based neural network and performing inference using ONNX Runtime. This flow ensures that images from various sources are normalized in terms of color space, dimensions, and numerical representation.

The pipeline is implemented across two main contexts: the training phase (utilizing PyTorch transforms) and the prediction phase (utilizing numpy and PIL). Both paths share a common goal of producing a 1-channel (grayscale) tensor or array with dimensions $160 \times 50$ pixels.

Preprocessing Pipeline Architecture

The preprocessing logic is decoupled into two primary implementations to support the different requirements of the training environment and the production inference environment.

Inference Preprocessing (ONNX)

In the prediction module, images are processed using PIL and numpy to prepare them for the onnxruntime session. This involves converting the image to grayscale ("L" mode), resizing, and normalizing pixel values to a range of $[0, 1]$.

Training Preprocessing (PyTorch)

During training, the DatasetLoader class utilizes torchvision.transforms to apply a consistent set of transformations. This ensures that the training data matches the input constraints of the OcrModel, which is built on a modified ResNet18 architecture.

Key Transformation Components

The following table summarizes the specific operations performed during the preprocessing stage:

Operation	Implementation	Description	Source
Color Conversion	.convert("L")	Converts input images to single-channel luminance (grayscale).	core.py:58, predict.py:19
Resizing	(160, 50)	Standardizes all CAPTCHA images to 160px width and 50px height.	predict.py:20, train.py:28
Normalization	/ 255.0	Scales integer pixel values (0-255) to floating point values (0.0-1.0).	predict.py:21
Dimension Expansion	np.expand_dims	Adds necessary batch and channel dimensions for the model input (1, 1, 50, 160).	predict.py:22-23

Data Flow in Dataset Loading

The DatasetLoader class manages the transition from disk files to processed tensors. It extracts labels from filenames and applies transformations to the image data.

Implementation Detail: Label Encoding

While the image is being preprocessed, the corresponding label is extracted from the filename and encoded into a one-hot format. This is performed by the encode function which maps characters to their indices in the chars list (digits + ascii letters).

def encode(a):
    onehot = [0] * chars_len
    idx = chars.index(a)
    onehot[idx] += 1
    return onehot

Sources: train_model/core.py:17-22

Input Requirements for Models

The preprocessing must satisfy the specific input layers of the OcrModel. The model's first convolutional layer is modified to accept a single input channel instead of the standard three channels found in ResNet18.

• Input Shape: (Batch, 1, 50, 160)
• Kernel Size: (50, 160) in the initial Conv2d layer.
• Data Type: float32.

Sources: train_model/core.py:72-74, mb_capcha_ocr/predict.py:21

Conclusion

The image preprocessing flow is a unified pipeline that ensures data consistency between the training phase and the production inference phase. By standardizing images to a $160 \times 50$ grayscale format and normalizing pixel values, the system provides a stable input for the ResNet18-based OCR engine to accurately predict CAPTCHA text.