Plant Species Identification using Vision Transformer and weed identification using YOLO model

The vast diversity of plant species on Earth makes their identification and classification a highly complex task, even for experts. The challenge is further compounded by visually similar yet biologically distinct species, making accurate classification difficult. While Convolutional Neural Networks (CNNs) have shown promise in addressing these challenges, their performance is often limited, and computational complexity increases with larger datasets and higher precision requirements.In this study, we explore Vision Transformers (ViTs), which have demonstrated state-of-the-art performance in image classification. Although ViTs generally have higher computational demands than CNNs, their ability to capture long-range dependencies makes them well-suited for fine-grained classification tasks as there are plants that look very similar yet are very different. To improve accuracy, we employed a three-branch CrossViT model for plant species classification but since it did not provide the required accuracy for the available dataset we leveraged the Swin Transformer V2 (Swin2), a cutting-edge ViT model, along with YOLO (You Only Look Once), an efficient real-time object detection model,for detecting the presence of weeds in the image. Our approach aims to enhance classification accuracy while optimizing performance for large-scale datasets.With this research paper, we aim to contribute to the field of machine learning as we have introduced a novel 3-branch CrossViT architecture, compared it with the SwinV2 model, and determined how our fine-tuned SwinV2 model gave better accuracy for the plant species detection task which had a very large and imbalanced dataset.