Probing Handwritten Manchu Word Recognition Foundation Model

Currently, there are several handwriting recognition models available that effectively address the challenge of Handwritten Text Recognition (HTR). However, majority of tasks require downstream training tailored for specific handwritten datasets. Relying solely on the foundation model may lead to subpar recognition performance. This problem is also encountered in Manchu handwritten text recognition. To overcome this challenge, researchers have been exploring different methods, such as data augmentation through image transformations. However, the existing fine-tuning datasets do not provide sufficient coverage of characters, which can result in weaker feature extraction and limit the improvement of the model. Additionally, creating a large-scale Manchu handwritten dataset is cost-prohibitive and challenging. In light of these constraints, we propose a novel approach that involves training a foundation model using existing large-scale datasets. This is achieved by freezing certain feature layers and conducting probing with the specific characteristics of Manchu handwritten scripts. This approach aims to enrich character features and improve diverse feature extraction capabilities of the model, which can enhance the generalization capabilities of the model to small-scale Manchu handwritten datasets. The experimental results demonstrate that our probing strategy achieves superior performance compared to the state-of-the-art AMRE foundation model. Furthermore, we conducted an ablation study and additional analyses to examine the effectiveness of the probing approach and uncover the underlying modeling patterns.