Chinese Named Entity Recognition Based on Improved K-BERT

Named entity recognition of Chinese text using pre-trained models is the mainstream approach at present, and the proposed K-BERT model overcomes the problem that BERT models do not possess background knowledge. We use the Chinese pre-trained model BERT-wwm and adversarial training based on full-word masking technology on the basis of K-BERT to improve the disadvantages of poor robustness of traditional K-BERT neural network and the existence of WordPiece sequence mask of traditional BERT. The experimental results on three open source datasets, MEDICAL_NER, MSRA_NER and FINANCIAL_NER, show that the evaluation index F1-score is improved after adding the adversarial training model. And the comparison experiments verify that adding adversarial training can elevate the prediction ability and robustness.