Comparative Analysis of CNN ResNet and Vision Transformers for Plant Type and Disease Recognition for Generalist Farmland Robots

Large Language Models (LLM) are revolutionizing robotics. Vision-Language-Models (VLMs) and Vision-Language-Action (VLA) have gained wide acceptance in general robot applications. Recently, robots are built with the ability to convert text prompts to actions. In this study the efficient deep-learning model to build a foundational backbone for a farmland-based VLM for generalist robot was engaged, we evaluated the performance of Convolutional Neural Networks (CNN) models including ResNet34, ResNet50, ResNet101, and Vision Transformers (ViT) model. Three plants were selected (Soybean, Groundnut, and Tomato) and a plant disease Tomato Tuta Absoluta was considered for experimentation. The models can be further extended and fine-tuned using transfer learning. The pre-trained versions of these models were passed raw images of these plants for validation. The accuracy was 0% in all cases because ImageNet or CIFAR-10 were used for the pretraining instead of datasets oriented to the concerned disease detection. Transfer learning was then done on the pre-trained models to accommodate our plant dataset. We found that the training from pretrained models improved the performance. Each of the four models in consideration was evaluated to select the best performer on our plant dataset. Among the considered methods, ViT model performed best with 82.1% training accuracy and 72.9% validation accuracy. ViT model had a higher generalization capacity, which is a very important characteristic that is needed while building a VLM backbone that will be extended into a generalist robot VLA. The research drew motivation from OpenVLA, and future research is to extend OpenVLA with farmland scenarios.