Microscopic Fungi Classification Using Vision Transformer Guided by Transfer Learning Approach

The diagnosis of fungal infections is quite challenging because of their wide-ranging symptoms and characteristics. Different fungi species are typically categorized using manual microscopical techniques, which are ostensibly prone to some degree of subjectivity. Deep learning models have been employed in computer-assisted diagnostic systems in recent years for quick and precise mycological diagnosis. This study represents an automatic detection of fungi species in microscopic images using the vision transformer (ViT) architecture and transfer learning-based technique. The dataset in this study includes microscopic image samples of five distinct fungal species. First, a vision transformer network is trained and validated on the dataset, achieving 83.12% accuracy with decent precision, recall, and F1 score. The concept of transfer learning is then utilized by adding pre-trained models—VGG16, InceptionV3, and ResNet50, separately in parallel with the ViT network, where features extracted using the transfer learning technique fine-tune and improve the overall performance of the ViT network. Our study finds the ViT with ResNet50 model as the top-performing model, with an enhanced performance of 90.13% accuracy on the same dataset with 91.22% precision, 89.70% recall, and 90.91% F1 Score. These findings demonstrate the effectiveness of our approach in precisely categorizing fungi species.