ABCM: A Patent Classification Model Based on Pretrained Language Models with Imbalanced Data

The phenomenon of data imbalance in patent classification research is relatively common. When traditional classification algorithms deal with imbalanced data, the classifier prediction tends to the majority class, and the minority class has a large classification error. To solve this problem, we propose the ABCM model. It is a method based on pre-training language model to solve the problem of sample imbalance in patent classification. It first uses unsupervised patent data to pre-train the language model, and then combines BiGRU and capsule network to abstract patent information in higher dimensions. The paper also modifies the loss function to make it more suitable for training on sample imbalanced tasks, and finally uses supervised data to train the classifier. The experimental results show that this method can effectively improve the classification performance of imbalanced data.