A Study to Classify Virus Genome Through Analyzing DNA Sequences Using Transformer Model

In a generic computational environment for biological data processing, classifying DNA sequences is an important task. In recent times, a range of machine learning methods have been successfully applied to implement this task. Still, the hardest part of the process is choosing the features. The most popular representations exacerbate the massive dimensionality issue, because sequences lack unique features. Due to the pathogenic virus's high rate of transmission, early detection and accurate identification are essential for the treatment. It is challenging to predict though, because this virus's polymorphic nature enables it to adapt and survive in a variety of environment. Deep learning (DL) models have recently developed the ability to automatically extract characteristics from the input sequences. Previously, CNN-Bidirectional LSTM and CNN-LSTM structures with Label and K-mer encoding, Random Forests and Gradient Boosting Method, and Linear Discriminant Analysis have been utilized for DNA sequence classification. In this paper, we investigate the impact of a popular and well-established language model, namely Transformer (BERT) which uses self-attention mechanism, to alter the procedure to understand underlying hidden patterns of the virus DNA sequences and correctly classify them.