Troubleshooting Prediction Errors
Relevant source files
The following files were used as context for generating this wiki page:
This wiki page provides technical guidance for identifying and resolving issues related to prediction errors in the MBBank CAPTCHA OCR system. It covers the end-to-end flow from image preprocessing and model inference to character decoding, highlighting potential failure points in the OCR pipeline.
The system utilizes a PyTorch-based ResNet18 architecture for training, which is then exported to the ONNX format for efficient inference. Troubleshooting efforts should focus on the alignment between training-time data transformations and inference-time preprocessing, as well as the integrity of the character mapping.
Sources: train_model/train.py:1-60, mb_capcha_ocr/predict.py:11-33, README.md:1-15
Prediction Pipeline Overview
Understanding the data flow is critical for diagnosing where a prediction might deviate from the expected output. The prediction process follows a rigid sequence of image conversion, normalization, and tensor reshaping before reaching the ONNX inference session.
The diagram shows the transformation steps required to prepare an image for the OCR model.
Sources: mb_capcha_ocr/predict.py:20-33
Preprocessing Mismatches
A common source of prediction error is a mismatch between how images are processed during training versus inference. The model expects specific dimensions and color channels.
Image Dimensions and Color Space
The model architecture is specifically modified to accept a single-channel (grayscale) input with fixed dimensions.
- Target Size: 160x50 pixels (Width x Height).
- Color Mode: Grayscale ("L").
- Normalization: Pixels are scaled to a range of [0.0, 1.0] by dividing by 255.
| Parameter | Training Value | Prediction Value | File Reference |
|---|---|---|---|
| Resize Dim | (50, 160) PyTorch (H, W) | (160, 50) PIL (W, H) | train.py:24, predict.py:22 |
| Color Mode | .convert("L") | .convert("L") | core.py:73, predict.py:21 |
| Normalization | transforms.ToTensor() | / 255.0 | train.py:25, predict.py:23 |
Sources: train_model/train.py:23-26, mb_capcha_ocr/predict.py:21-23, train_model/core.py:73
Model Architecture and Output Failures
The OcrModel uses a ResNet18 backbone where the first convolutional layer and the final fully connected layer have been modified. Errors in output shape typically stem from these modifications.
Architecture Structure
The model reshapes the final output to (-1, 6, 62) representing the batch size, the number of CAPTCHA characters, and the character set length.
Sources: train_model/core.py:81-93
Logical Logic for Prediction Errors
If the predict() method returns gibberish or incorrect lengths, verify the max_capcha and chars constants:
- Max CAPTCHA Length: 6 characters.
- Character Set: 62 characters (0-9, a-z, A-Z), sorted alphabetically.
Sources: train_model/core.py:16-18, mb_capcha_ocr/predict.py:15-18
Inference Environment Issues
The prediction logic relies on onnxruntime to execute the exported model.onnx.
Loading the Model
The OcrModel class looks for model.onnx in its local directory by default. If the file is missing or corrupted, the InferenceSession will fail to initialize.
# From mb_capcha_ocr/predict.py
def __init__(self, model_path: Optional[str]=None):
self.chars.sort()
self.session = onnxruntime.InferenceSession(model_path or f"{dir_path}/model.onnx")
Sources: mb_capcha_ocr/predict.py:17-18
Dependency Verification
Ensure all runtime dependencies are installed as specified in the project requirements. Missing libraries like PIL (Pillow) or numpy will prevent the preprocessing steps from executing.
| Dependency | Purpose |
|---|---|
onnxruntime | Executing the trained model |
Pillow | Image loading and resizing |
numpy | Matrix operations and normalization |
Sources: requirements.txt:1-3, setup.py:18-30
Troubleshooting Data Labeling
Errors in prediction accuracy often trace back to inconsistent labeling during the training phase. The system derives labels directly from filenames in the dataset/ directory.
If images are improperly named (e.g., containing extra dots or incorrect characters), the DatasetLoader will generate incorrect labels, leading to a model that predicts incorrectly even if the loss appears low during training.
Sources: train_model/core.py:68-71, train_model/train.py:32
Summary
Troubleshooting prediction errors in the MBBank CAPTCHA OCR requires verifying the image preprocessing pipeline (Grayscale, 160x50, normalization), ensuring the ONNX model file is accessible to onnxruntime, and confirming that the character set mapping matches the one used during the training of the ResNet18-based architecture. Consistency between the DatasetLoader encoding logic and the predict.py decoding logic is the foundation of accurate text recovery.