Holistic Self-Distillation with the Squeeze and Excitation Network for Fine-grained Plant Pathology Classification

Fine-grained plant pathology classification is an important task for precision agriculture, but at the same time, it is challenging due to the subtle difference in plant categories. Variances in the lighting conditions, position, and stages of disease symptoms usually lead to degradation of classification accuracy. Knowledge distillation is a popular method to improve the model performance to deal with the indistinguishable image classification problem. It aims to have a well-optimised small student network guided by a large teacher network. Existing knowledge distillation methods mainly consider training a teacher network that needs a high storage space and considerable computing resources. Self-knowledge distillation methods have been proposed to distil knowledge from the same network. Although self-knowledge distillation saves time and space compared with knowledge distillation, it only learns label knowledge. In this paper, we propose a novel self-distillation method to recognize the fine-grained plant category, which considers holistic knowledge based on the Squeeze and Excitation Network. We label this new method as holistic self-distillation because it captures knowledge through spatial features and labels. The performance validation of the proposed approach is performed on two public fine-grained plant datasets: Plant Pathology 2021 and Plant Pathology 2020 with the accuracy of 98.22% and 90.72% respectively. We also present experiments on the state-of-the-art algorithm (ResNet-50). The classification results demonstrate the effectiveness of the proposed approach with respect to accuracy.