Multimodal Pest Detection Using Swin Transformer and Gated Recurrent Unit in Agricultural Fields

In the modern era, precise agriculture has experienced significant growth to the automation of pest detection systems. However, existing Multimodal Fine-Grained Transformer (MMFGT)-based systems faced challenges such as a lack of ability to model temporal data, heavy reliance on static images, and simple fusion techniques that failed in real-world field conditions. To address these challenges, a novel pest detection framework that combines a Swin transformer with a Gated Recurrent Unit (Swin-GRU) is proposed. Initially, input data is taken from fields of agriculture, including Red Green Blue (RGB) images and environmental metadata. Next, the collected data is preprocessed by sequence construction and normalization. Subsequently, the preprocessed data are passed to the Swin transformer, which performs spatial feature extraction. Sequentially, the extracted sequence of features through the Swin transformer is fed to the Gated Recurrent Unit (GRU), which captures pest dynamics over time. Finally, the visual and environmental features are combined using a cross-attention-based fusion module, and the final classification is achieved using the softmax layer. The proposed Swin-GRU achieved a higher accuracy of 99.23% and outperformed the existing model.