DAPS3D: Domain Adaptive Projective Segmentation of 3D LiDAR Point Clouds

LiDARs are one of the key sources of reliable environmental ranging information for autonomous vehicles. However, segmentation of 3D scene elements (roads, buildings, people, cars, etc.) based on LiDAR point clouds has limitations. On the one hand, point- and voxel-based segmentation neural networks do not offer sufficiently high speed. On the other hand, modern labeled datasets primarily consist of street scenes recorded for driverless cars and contain little data for mobile delivery robots or cleaners that must work in parks and yards with heavy pedestrian traffic. This article aims to overcome these limitations. We have proposed a novel approach called DAPS3D to train deep neural networks for 3D semantic segmentation. This approach is based on a spherical projection of a point cloud and LiDAR-specific masks, enabling the model to adapt to different types of LiDAR. First of all, we have introduced various high-speed multi-scale spherical projection segmentation models, including convolutional, recurrent, and transformer architectures. Among them, the SalsaNextRecLSTM architecture with recurrent blocks showed the best results. In particular, this model achieved the 83.5% mIoU metric for the SemanticKitti dataset with joint categories. Secondly, we have proposed several original augmentations for spherical projections of LiDAR data, including FoV, flip, and rotation augmentation, as well as a special T-Zone cutout. These augmentations increase the model’s invariance when dealing with changes in the data domain. Finally, we introduce a new method to generate synthetic datasets for domain adaptation problems. We have developed two new datasets for validating 3D scene outdoor segmentation algorithms: the DAPS-1 dataset, which is based on the augmentation of the reconstructed 3D semantic map, and the DAPS-2 LiDAR dataset, collected by the on-board sensors of a cleaning robot in a park area. Particular attention is given to the performance of the developed models, demonstrating their ability to function in real-time. The code and datasets used in this study are publicly available at: github.com/subake/DAPS3D.