SWIFT: Tailored Autoregression using Semantic Word Insight for Transformers

In the current era of online publication, the publication rate of online news articles has grown exponentially. The classification of news articles is crucial for easier comprehension of online content. Without proper categorization, readers often struggle to navigate through the vast amounts of news available to them. This paper is devoted to the single-label semantic classification on the combined dataset of BBC All Time and Reuters News. The paper conducted comparative analyses between SWIFT and established classifiers like BERT, alBERT, and DistilBERT. The experiments demonstrate that SWIFT surpasses these generic models on the English News Dataset, yielding superior classification performance, particularly in the categorization of news titles. Comparison with state-of-the-art research shows that SWIFT can be considered as the basis for future predictions and explorations of news dataset analysis.