Cross-Lingual Text Classification by using Bidirectional Encoder Representations from Transformers based Transfer Learning

In recent days, the growing demand for automated language processing had brought Cross-Lingual Text Classification (CLTC) into focus as powerful approach for categorizing text across multiple languages which enabled users to classify documents in different languages. Traditional approaches for CLTC had faced several issues which include translation errors, dependency on parallel data and language representation gaps. Therefore, this research proposes Bidirectional Encoder Representations from Transformers (BERT) based Transfer Learning. Initially, data is collected from AG news, preprocessed by using Punctuation removal and Lemmatization which reduces irregularities in text data. After that, feature extraction is performed by using Term Frequency-Inverse Document Frequency (TF-IDF) which calculates word frequencies across documents by creating a high-dimensional feature vector representation. Further, features like semantic and contextual features are selected by using Mutual Information (MI). Finally, classification is done by using BERT-based transfer learning. The proposed BERT-based Transfer Learning achieved better results in terms of accuracy (95.5%) when compared with existing Indian Sentence BERT (IndicSBERT) model.