Introduction & Project Home

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• mb_capcha_ocr/predict.py
• train_model/core.py
• train_model/train.py
• setup.py
• mb_capcha_ocr/init.py
• requirements.txt
• train_model/train_requirements.txt

The MBBank CAPCHA OCR project is a specialized library designed for the training and deployment of an Optical Character Recognition (OCR) model focused on MBBank CAPTCHA images. It provides a complete pipeline from dataset loading and model training using PyTorch to high-performance inference using ONNX Runtime.

The repository is structured into two primary modules: mb_capcha_ocr for consumption of the trained model in production environments and train_model for the underlying development and training logic.

Sources: README.md:1-10, setup.py:15-20

System Architecture

The project employs a ResNet-18 backbone modified to handle single-channel (grayscale) CAPTCHA images. The system workflow transitions from raw image data to a multi-label classification output representing the characters in the CAPTCHA.

Core Components

Component	Responsibility	File Path
OcrModel (Training)	PyTorch implementation based on ResNet-18 with modified input and fully connected layers.	train_model/core.py
DatasetLoader	Manages CAPTCHA image loading, shuffling, and one-hot encoding of labels.	train_model/core.py
OcrModel (Inference)	Wrapper for ONNX Runtime to perform predictions using the exported model.	mb_capcha_ocr/predict.py
Training Script	Orchestrates the training loop, loss calculation, and ONNX export.	train_model/train.py

Sources: train_model/core.py:65-74, mb_capcha_ocr/predict.py:11-33, train_model/train.py:10-50

Training Workflow

The training process involves loading images from a local directory where the filename serves as the ground-truth label. The model is trained to predict a sequence of characters, typically limited to a maximum length of 6.

The diagram shows the data flow from raw image files through preprocessing and training to the final model export.
Sources: train_model/train.py:22-55, train_model/core.py:27-63

Model Implementation Details

Architecture Specifications

The model utilizes a resnet18 architecture. Key modifications include:

• Input Layer: conv1 is replaced with a Conv2d(1, 64, ...) to support grayscale input (1 channel) instead of RGB.
• Output Layer: The fully connected layer (fc) is modified to output a tensor of size chars_len * max_capcha, which is then reshaped to (-1, 6, 62) to represent 6 characters across a 62-character character set (digits + uppercase/lowercase letters).

Sources: train_model/core.py:12-16, train_model/core.py:65-76

Character Mapping

The character set used for encoding and decoding consists of sorted digits and ASCII letters:

chars = list(string.digits + string.ascii_letters)
chars.sort()
# Result: ['0', '1', ..., '9', 'A', 'B', ..., 'z']

Sources: train_model/core.py:12-14, mb_capcha_ocr/predict.py:13-17

Prediction & Inference

For production use, the model is executed via onnxruntime to ensure compatibility and performance. The OcrModel class in the prediction module handles the image preprocessing (grayscale conversion, resizing to 160x50, and normalization) and decodes the resulting tensor back into human-readable text.

Inference Sequence

This sequence illustrates the transformation of a raw PIL image into a string prediction using the ONNX model.
Sources: mb_capcha_ocr/predict.py:19-33

Technical Configuration

The project relies on specific dependencies for training and inference, as outlined in the configuration files.

Environment	Key Dependencies
Inference	onnxruntime, pillow, numpy
Training	torch, torchvision, onnx, matplotlib

Sources: requirements.txt:1-5, train_model/train_requirements.txt:1-7

The model expects images to be resized to a specific dimension of 160 pixels (width) by 50 pixels (height) during both training and inference.
Sources: train_model/train.py:23, mb_capcha_ocr/predict.py:21

Summary

The MBBank CAPCHA OCR project provides a streamlined solution for breaking image-based CAPTCHAs. By leveraging a modified ResNet-18 architecture and providing both a training pipeline in PyTorch and a lightweight inference module in ONNX, it bridges the gap between deep learning development and practical application.