Invisible damage classification of crispy pears based on the combination of hyperspectral and thermal imaging using COGhost_ShuffleNet

Identifying and classifying invisible damage in crispy pears is vital for ensuring high-quality grading. Despite the demonstrated potential of hyperspectral imaging (HSI) and infrared thermal imaging (IR) for fruit identification, the performance of damage recognition is constrained by the low resolution of HSI images and the spectral redundancy. Additionally, IR exhibits suboptimal performance in identifying multiple types of damage. In this study, we developed a crispy pear damage identification method using a combination of multiple data inputs and an improved ShuffleNetV2. Firstly, this paper registers HSI and IR using the radiation-insensitive feature transform (RIFT) algorithm, obtaining the damaged areas of crisp pears through registration. Secondly, it extracts the HSI spectra from the damaged areas and visualizes the spectra through Gramian Angular Fields (GAF) two-dimensional imaging. Next, it investigates the optimal combination obtained under the fusion of multiple data, aiming to achieve the highest accuracy under the most compatible ShuffleNetV2 model. The optimal combination identified is IR-Band1-Band2-Band3-GAF. Finally, facing the multiple data inputs, reducing network parameters and Flops is crucial for the rapid and efficient damage identification of crisp pears. Therefore, we propose optimizations for ShuffleNetV2.The Ghost module was used to replace the 1 × 1 convolution in the two basic unit modules of ShuffleNetV2, and Ghost_ShuffleNet used convolution optimization (CO) to construct the COGhost_ShuffleNet classification model. We used the optimal data combination as input for identification and classification on this model. COGhost_ShuffleNet achieved the best recognition performance with an ACCP of $98.93\%$, 1.158M parameters, and 0.086G FLOPS, outperforming other classification methods and data input combinations. Overall, this method accurately and reliably identifies crispy pear damage and can be extended to analyze other quality indicators of invisible damage in various fruits.