Plant Organelle Segmentation using 3D Point Clouds and PointNet

Monitoring crops is an essential task in maintaining the life of plants. State-of-the-art agricultural systems are capable of capturing 3D data of individual crops and reconstructing digital twins. A prominent issue in the field is extracting information about the subject plant from the point cloud data, particularly classifying organelle. To address this problem, a deep neural network with a PointNet-like architecture is proposed to identify organelle of the subject plant. Using datasets of plant images taken from an RGB-camera attached to a robotic arm, 3D point cloud data is generated with structure-from-motion photogrammetry techniques, which is used to train a network to segment local regions of the model. A proof-of-concept segmentation network is presented, trained using a large multi-view dataset of avocados images to identify organelle such as the seed, interior pulp, and exterior skin. The results collated evaluate the model throughout training using metrics such as RMSE of training and validation datasets.