Captcha OCR Solver

Relevant source files

The following files were used as context for generating this wiki page:

• mbbank/capcha_ocr.py
• mbbank/sync/mbbank.py
• mbbank/aio/mbbank.py
• mbbank/base/mbbank.py
• pyproject.toml

The Captcha OCR Solver is a specialized module within the MBBank library designed to automate the recognition and resolution of visual CAPTCHAs required during the authentication process. It acts as an intermediary between the raw image data provided by the MBBank API and the login credentials required to establish a session.

This system is built to be extensible, allowing developers to either use the built-in ONNX-based solver or implement their own custom processing logic. By default, it leverages the mb-capcha-ocr package to provide high-accuracy character recognition specifically tuned for the bank's captcha format.

Architecture and Components

The OCR system is structured around an abstract base class that defines the interface for image processing, ensuring compatibility with the main library's authentication loop.

Core Classes

• CapchaProcessing: An abstract base class that defines the process_image method. It serves as a blueprint for any captcha solver implementation.
• CapchaOCR: The default implementation that inherits from CapchaProcessing. it uses an ONNX model to predict text from image bytes.

Sources: mbbank/capcha_ocr.py:7-30, mbbank/capcha_ocr.py:32-38

Class Hierarchy

The following diagram illustrates the relationship between the base processing class and the concrete OCR implementation.

The CapchaOCR class initializes an OcrModel which typically takes 1-3 seconds to load.
Sources: mbbank/capcha_ocr.py:46-50

Integration with Authentication

The OCR solver is integrated into both the synchronous (MBBank) and asynchronous (MBBankAsync) clients. It is primarily invoked during the _authenticate workflow when a session is expired or non-existent.

Authentication Logic Flow

1. The client requests a captcha image from https://online.mbbank.com.vn/api/retail-internetbankingms/getCaptchaImage.
2. The API returns a Base64 encoded image string, which the library decodes into raw bytes.
3. The bytes are passed to the ocr_class.process_image() method.
4. The resulting text is included in the doLogin API request.
5. If the API returns error code GW283 (invalid captcha), the system automatically retries the process until the retry_times limit is reached.

Sources: mbbank/sync/mbbank.py:113-132, mbbank/sync/mbbank.py:165-182, mbbank/aio/mbbank.py:157-175

Authentication Sequence

The sequence below demonstrates how the client interacts with the OCR solver and the MBBank API.

Configuration and Dependencies

The OCR solver relies on specific external libraries for image handling and machine learning inference.

Technical Requirements

Dependency	Version	Purpose
mb-capcha-ocr	0.1.6	Core ONNX model and prediction logic
Pillow	12.2.0	Image opening and byte stream manipulation

Sources: pyproject.toml:13-20

Configuration Options

The solver's behavior can be tuned via the main client constructor:

• ocr_class: Allows injecting a custom instance of CapchaProcessing. Defaults to CapchaOCR().
• retry_times: The maximum number of attempts to solve the captcha if the bank rejects the text. The default is 30 attempts to account for potential OCR misreads.

Sources: mbbank/base/mbbank.py:60-67, mbbank/sync/mbbank.py:48-51

Custom Implementation

Developers can implement their own captcha solving logic (e.g., using a third-party API or a different local model) by inheriting from the base class.

from mbbank.capcha_ocr import CapchaProcessing

class MyCustomSolver(CapchaProcessing):
    def process_image(self, img: bytes) -> str:
        # Custom logic here
        return "solved_text"

# Usage
mb = MBBank(username="user", password="pass", ocr_class=MyCustomSolver())

Sources: mbbank/capcha_ocr.py:10-18, mbbank/base/mbbank.py:61-64

Error Handling

The system distinguishes between CAPTCHA-specific errors and general API failures.

• GW283: Specifically caught to trigger a retry of the OCR process.
• CapchaError: Raised if the number of retries exceeds the retry_times limit.

Sources: mbbank/sync/mbbank.py:177-182, mbbank/aio/mbbank.py:170-175

Summary

The Captcha OCR Solver provides a robust, automated way to handle visual security challenges in the MBBank authentication flow. By abstracting the recognition logic into the CapchaProcessing interface, the library maintains a clean separation between network communication and image processing while providing a high-performance default solver using ONNX.