Spelling Error Correction with BERT based on Character-Phonetic

In order to solve the problem of high proportion of erroneous strings caused by spelling errors in the process of official document writing, this paper proposes a Character-Phonetic BERT model based on the structural transformation of BERT. Which uses the BiLSTM network to detect the location of error characters, and then add the pinyin prior knowledge of the error location to the BERT network to achieve end-to-end spelling error detection and correction. The experimental results show that the Character-Phonetic BERT model in this paper improves the effect by about 5% compared with the traditional language error correction model; Compared with the Bert-Finetune model without considering the pinyin information of the error location, Character-Phonetic BERT model increases by 2.1%. It achieves more accurate Chinese text automatically corrects errors.