Enhancing Multi-Class Text Classification with BERT-Based Models

Document classification serves a diverse range of practical applications, significantly enhancing various processes. For instance, it can expedite the identification of categories assigned to research reports submitted by students. By subsequently aligning these categories with the particular interests of university staff members, the reports can then be evaluated by those who possess the most relevant expertise. In this paper, we developed and evaluated several models for carrying out multi-label and multi-class text classification. Our approach revolves around the pre-trained BERT models. We endeavour to augment the efficacy of classification by leveraging latent information drawn from the output and hidden layers of the BERT architecture. We seek to achieve good accuracy and F1 score of the classification process. Our model utilizing the attention mechanism and LSTM to process information generated by BERT outperforms all other models based on our evaluation.