Gastrointestinal Insights Redefined: An Integrated Hybrid Model Fusing Vision Transformer and Transfer Learning

Gastrointestinal (GI) problems affect millions of people globally and pose a severe hazard to public health. Treatment for these disorders must begin as soon as feasible, with accuracy, in order to prevent complications. The classification of GI illnesses is dependent on a manual examination, which is time-consuming. In the past few years, deep learning (DL) emerged as a feasible tool for automatically classifying gastrointestinal diseases. In this study, a novel technique for GI disease classification is proposed employing Vision Transformers (ViT) and DL. To extract significant features from patient data, the recommended method takes utilization of a hybrid developed transfer learning (TL) model that is analogous to Xception. The attributes acquired are subsequently put into a ViT model, which contains a Swin transformer, for illness categorization. As a result of evaluating accuracy curves, classification reports, and confusion matrices, the proposed approach outperformed other SOTA ViT models on a large dataset of GI disease cases. According to our results, we acquired Compact Convolutional Transformer(CCT) with an accuracy of 74.30%, External Attention Transformer(EANet) with an accuracy of 74.85%, and our proposed hybrid model (Swin+Xception) with an accuracy of 87.23%. In addition to improving clinical diagnostic accuracy and efficiency, the proposed approach can potentially automate the classification of GI diseases.