PlantFusion: A Novel Multimodal Dataset and Hybrid Multimodal Deep Learning Framework

Accurate plant disease detection is crucial for improving agricultural practices and ensuring food security. Traditional methods often rely on uni-modal datasets, which might focus solely on visual symptoms captured through images or on environmental and sensor data. However, these approaches can be limited in scope and effectiveness. Integrating multiple data sources provides a more comprehensive understanding of plant health. This paper addresses the need for a more holistic approach by introducing PlantFusion Dataset - a novel multimodal dataset that combines visual, textual, and environmental data. This integrated dataset allows for the development of advanced models that leverage the strengths of each modality, ultimately leading to more accurate and generalizable plant disease detection. We also propose a hybrid deep learning model PlantFusion that leverages this multimodal data, enhancing disease prediction accuracy and model interpret-ability as we achieved accuracy of 0.998 and as similar high F1 score which is outperforming some of the latest works done.