Long Text Classification Method of News Based on Improved BERT

Text classification is an essential research area in Natural Language Processing (NLP) and has received much attention from scholars in recent years. However, real online news text data has the problems of long text, high information content, and complex structure, which reduces the accuracy of news long text classification. Specifically, pre-trained language models like BERT can effectively process short texts with less than 510 tokens. Nevertheless, existing open-source pre-trained language models such as BERT cannot be directly fine-tuned to meet the requirements when confronted with input texts longer than 510 tokens. To address this issue comprehensively, this paper proposes a solution based on Pooling technology that aims to explore and optimize the processing methodology for long text classification problems.