Chinese Spelling Error Correction Based on Adaptive Pre-training and Multi-Task Learning

With the rapid development of computer application technology, more and more information is transmitted through the text through the network. The number of network texts grows fast, and it is difficult to guarantee the quality of the text. Low-quality information will pollute the network environment and spread error information. The traditional manual text check method has been unable to deal with today's massive data. Therefore, spelling error correction methods based on the deep learning model have attracted the attention of many experts and scholars in recent years. This paper proposes a Chinese spelling error correction model based on task adaptive pre-training and multi-task learning. Firstly, based on the Bert, this paper proposes an optimized mask strategy for the pre-training method to make the pre-training model better match the spelling error correction task. Secondly, based on the language model obtained by task adaptive pre-training, this paper combines spelling error detection and spelling error correction by multi-task learning, Finally, we test our model on the SIGHAN dataset, which is a general benchmark dataset of Chinese spelling error correction. Compared with other models, our model outperforms all competitor methods.