Rice-Fusion: A Multimodality Data Fusion Framework for Rice Disease Diagnosis

Rice leaf infections are a common hazard to rice production, affecting many farmers all over the world. Early detection and treatment of rice leaf infection are critical for promoting healthy rice plant growth and ensuring adequate supply for the fast-growing population. Computer-assisted rice leaf disease diagnoses are hampered due to strong image backgrounds. Popular Convolutional Neural Network (CNN) architecture extracts the features from images and diagnoses the disease to address the issues above. However, this method is best suitable for segmented images and gives low accuracy with real-time images. In this case, the Internet of Things is a paradigm shift that collects agro-meteorological information that effectively helps diagnose rice diseases. Motivated by the usefulness of CNN models and agricultural IoT, a novel multimodal data fusion framework named Rice-Fusion is proposed to diagnose rice disease. Rice disease diagnosis based on a single modality may not be accurate, and hence the fusion of heterogeneous modalities is essential for robust and reliable disease diagnosis. This gives a new dimension to the domain of rice disease diagnosis. The dataset was collected manually with 3200 rice health category samples using two modalities, namely agro-meteorological sensors and a camera. The Rice-Fusion framework initially extracts the numerical features from agro-meteorological data collected from sensors. Next, it extracts the visual features from the captured rice images. These extracted features are further fused using a concatenation layer followed by a dense layer, which provides single output for diagnosing the rice disease. The testing accuracy of Rice-Fusion is 95.31% as opposed to other unimodal framework accuracies of 82.03% and 91.25% based on CNN and Multi-Layer Perceptron (MLP) architectures, respectively. Experimental results analysis demonstrates that the proposed Rice-Fusion multimodal data fusion framework outperforms the outcome of unimodal frameworks.