News Article Classification using a Transfer Learning Approach

The increasing proliferation of electronic files in recent decades has been addressed by significant improvements in Natural Language Processing. As the text data generated is quite large, employing sophisticated algorithms that can process massive data within seconds is needed. Machine learning algorithms seem to be promising in the case of text processing and classifying articles. Still, it lacks automated feature extraction. Moreover, the advancements lead to deep learning algorithms automatically extracting relevant and descriptive features. The rapid evolution of these technologies has resulted in a profusion of strategies for converting natural language into machine-understandable data. This paper proposes the transfer learning approach, i.e., to utilize Bidirectional Encoder Representations from Transformers (BERT) and RoBERTa, pre-trained models for news article classification. The paper's focus is to fine-tune the hyperparameters of the pre-trained models to obtain a higher performance in classifying news articles. Pre-processing techniques like tokenization and extracting labels are performed before training the data on the models. The dataset utilized is an open-source dataset from Kaggle BBC news classification. Furthermore, the models are assessed using standard performance metrics like accuracy, cross-entropy loss, and time required for training. The results depict that both the models perform nearly the same by achieving a training accuracy of 100% and a validation accuracy of 99.1%. Nevertheless, the time required for training the RoBERTa model is more than the BERT model.