An Automatic Labeling Method for the Recognition of Micro Rice Ears from Cross Modal in UAV Images

Rice yield estimation is an important component of national food security systems and the number of rice ears per acre is an important factor in the yield composition. Although unmanned aerial vehicles (UAVs) can be used to quickly obtain rice ear images, this also bring out the small features of the target object in the images captured, and it is difficult to label rice ear images for deep learning. In this study, an automatic labeling method for the recognition of small rice ears in UAV images is proposed by synergizing color index and deep learning method based on the essential features of rice ear recognition. Firstly, an optimal color index for the color feature of rice ears was selected for the edge extraction of rice ears in UAV images, which would be regarded as the label edge images in deep learning for rice ear recognition. Secondly, these extracted rice ear edge images were used to cut the original UAV images, in which the cut image contained the color, texture, shape, edge, and spatial relationship features of rice ears. Thirdly, these cut images, defined as automatic labeling datasets, were used to train the feature parameters of the deep learning model, and a hook and visualization toolbox was employed to visualize the features of the rice ears during the training process. The main characteristic parameters in the deep learning system were separated from the visualized feature data using a principal component analysis (PCA). The results indicate that the main feature parameter recognized in deep learning is the color feature of rice ears, which matches the most obvious visual features in the rice ear definition system in the field of agriculture. In this way, this cross-modal method for rice ears recognition not only uses color index to extract edge images as a label prototype, saving the hard work of labeling for small target object in deep learning systems, The entire process achieved the recognition of small targets in the cross modal formed a new deep learning framework contain automatic annotation and network training and prediction. Finally, a comparison of the rice ear recognition in the same deep learning system with the automatic labeling data and manual labeling data was performed on UAV images, and the results showed that the accuracy of automatic annotation has increased by 3.68% compared to manual annotation and the iteration speed of automatic annotation can quickly reach the level of manual annotation. Thus, the results of this study demonstrated that Automated labeling systems can be used to extract the main parameters of large area of rice yield estimation from UAV image.