Research on Multimodal Chinese Spelling Correction Task with Integrated Contrastive Learning Strategies

The Chinese Spelling Correction (CSC) task aims to detect and rectify erroneous characters in user-generated Chinese text. Chinese spelling errors often arise from the misuse of characters with similar semantics, pronunciation, or visual appearance. Previous methods have recognized this issue and attempted to address it by leveraging similarity relationships. However, most of these methods rely on manually curated confusion sets to predict the correct characters, making them susceptible to context-guided error predictions. In this paper, we propose MCLSpell, a Chinese spelling correction program that directly utilizes multi-modal information of Chinese characters and employs a contrastive learning strategy. By leveraging a pre-trained model optimized with contrastive probabilities, MCLSpell captures semantic, phonetic, and visual information of input characters and selectively integrates these cues to predict the correct outputs. The experimental results on the SIGHAN benchmark dataset validate the efficacy of the proposed approach, with F1 scores of 85.8% on SIGHAN13, 69.4% on SIGHAN14, and 78.3% on SIGHAN15. These results showcase the superior performance of our proposed method over baseline models, effectively combining the features of pronunciation and visual appearance, and addressing context-guided error predictions, consequently enhancing the accuracy of Chinese spelling correction models.