Sequence-Wise Distillation Method for Efficient Chinese Heritage Language Error Correction

Current advancements in Chinese error correction primarily focus on native speakers and second language learners, neglecting the unique needs of overseas Chinese heritage language learners. These learners often acquire Chinese in non-systematic, partial linguistic environments and face challenges with non-standard inputs that make existing correction methods ineffective. To address this gap, this paper introduces an artificial intelligence approach tailored for heritage language correction, aimed at improving the Chinese proficiency of overseas Chinese. Leveraging the overseas Chinese heritage corpus accumulated over the years by the College of Chinese Language and Culture at Jinan University, we manually selected and annotated a set of correct and incorrect sentence datasets suitable for overseas Chinese heritage learners. This preparatory step enhances the model's training process, ensuring it is finely tuned to the unique linguistic characteristics and needs of overseas Chinese heritage learners. Furthermore, we refined the powerful pre-trained model, MacBERT, to develop two specialized models for Chinese heritage language error detection and correction tasks, addressing specific challenges faced by heritage learners: a larger model, CHLBERT-390M, and a more compact model, CHLBERT-43MB. Finally, we introduced a strategy known as Sequence-wise Distillation (SwD) to reduce deployment costs by enhancing the performance of the smaller model. The SwD method dynamically modulates knowledge distillation weights according to each sequence's relative accuracy, thereby more effectively enhancing the small model's performance. Our work not only advances the development of knowledge distillation technology but also provides effective support for overseas Chinese in learning Chinese.