A Multi-head Two-level Attention-based Network for Plant-part Segmentation on 3D Point Cloud

The automated and accurate segmentation of structural plant parts from 3D point cloud has becoming a promising approach for non-destructive and high-throughput measurement of sought-after phenotypic traits in plant phenotyping. However, previous studies towards automatic segmentation of plants relied heavily on environmental circumstances, vulnerability toward occlusion, sensor set-up, hand-crafted features, as well as manually tuned thresholds. The purpose of this study was to establish a complete framework for plant-part segmentation of the point cloud in 3D space. To this end, we built an image capturing system to capture Caladium bicolor images for 3D reconstruction, and then hand-labeled the point cloud data. Furthermore, based on attention mechanism, we proposed a Muti-head Two-level Attention-based Network (MTANet) to hierarchically capture geometric features and predict per-point semantic annotations directly on the basis of the fully labelled point cloud data. The experimental results outperformed other main-stream point-based deep learning architectures with Overall Accuracy (OAcc), mean Accuracy (mAcc), and the mean Intersection over Union (mIoU) reached 98.73%, 94.67% and 92.58%, respectively. In addition, we also conducted ablation experiments to evaluate its efficiency and examine specific decisions of the network design. This study may contribute to automated measurements of plant phenotypic traits in breeding programs.