Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey of Datasets and Methods

3D semantic segmentation is a fundamental task for robotic and autonomous driving applications. Recent works have been focused on using deep learning techniques, whereas developing fine-annotated 3D LiDAR datasets is extremely labor intensive and requires professional skills. The performance limitation caused by insufficient datasets is called data hunger problem. This research provides a comprehensive survey on the question: are we hungry for 3D LiDAR data for semantic segmentation? The studies are conducted at three levels. First, a broad review to the main 3D LiDAR datasets is conducted, followed by a statistical analysis on three representative datasets to gain an in-depth view on the datasets’ size, diversity and quality, which are the critical factors in learning deep models. Second, an organized survey of 3D semantic segmentation methods is given with a focus on the mainstream of the latest research trend using deep learning techniques, followed by a systematic survey to the existing efforts to solve the data hunger problem. Finally, an insightful discussion of the remaining problems on both methodological and datasets’ viewpoints, and the open questions on dataset bias, domain and semantic gap are given, leading to potential topics in future works. To the best of our knowledge, this is the first work to study the data hunger problem for 3D semantic segmentation using deep learning techniques, which are addressed in both methodological and dataset review, and we share findings and discussions through a comprehensive dataset analysis.