Multi-label Text Classification Based on BiGRU and Multi-Head Self-Attention Mechanism

Multi-label text classification (MLTC) is widely used in natural language processing. There are correlations among labels in most MLTC tasks. But traditional multi-label text classification algorithms tend to pay more attention to the information of the text itself, and the correlations among labels are not sufficiently extracted, which largely affects the classification performance of the model. Aiming at the above problems, this paper proposes a multi-label text classification method based on BiGRU and multi-head self-attention mechanism. The method preprocesses text information by word embedding tools, and extracts contextual and global feature information of text using BERT and BiGRU models. The dependencies of different subspace labels are also integrated with a multi-head self-attention mechanism to model the correlation among labels. Label prediction is performed by combining label feature information and text feature information to realize the multi-label classification task. Experimental results on AAPD and Reuters-21578 datasets show that the method can effectively improve the results of text classification.