Hybrid CNN-LSTM Architecture for LiDAR Point Clouds Semantic Segmentation

The semantic segmentation of LiDAR in the outdoor environment is still an open problem in the field of automatic driving. Although the emerging development and technological advancements, it remains challenging for three reasons: 1) uneven distribution of LiDAR Points in 3D space; 2) the same object can be represented by different point clouds sequences; 3) no unique geometric information compared with the pixels of 2D RGB images. In order to tackle all the challenges mentioned above, we propose a new LiDAR point clouds semantic segmentation algorithm, Hybrid CNN-LSTM, which is composed of an efficient point clouds feature processing method and a novel neural network structure. Inspired by representing the point clouds as a fixed-length vector in PolarNet, we first convert the 3D point clouds into pseudo image. In order to better represent the features of small objects, we input the pseudo image into long short-term memory (LSTM) network according to the spatial filling curve. We design a new neural network structure, which combines the features of different channels generated by convolutional neural network and long short-term memory network. Experiments show that our method has higher semantic segmentation accuracy in comparison with state-of-the-art algorithms on SemanticKITTI dataset. Theoretically, we provide an analysis to understand why our network can better segment the small objects with sparse point clouds features.