SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud

4D LiDAR semantic segmentation classifies the semantic category of each LiDAR point and detects whether it is dynamic, a critical ability for tasks like obstacle avoidance and autonomous navigation. Existing approaches often rely on computationally heavy 4D convolutions or recursive networks, which result in poor real-time performance. In this paper, we introduce SegNet4D, a novel real-time 4D semantic segmentation network, offering both efficiency and strong semantic understanding. SegNet4D addresses 4D segmentation as two tasks: single-scan semantic segmentation and moving object segmentation, each tackled by a separate network head. Both results are combined in a motion-semantic fusion module to achieve comprehensive 4D segmentation. Additionally, instance information is extracted from the current scan and exploited for instance-wise segmentation consistency. Extensive experiments on the SemanticKITTI and nuScenes datasets demonstrate that our method outperforms the state-of-the-art in both 4D semantic segmentation and moving object segmentation. Through detailed runtime analysis, our method shows greater efficiency, enabling real-time operation. Besides, its effectiveness and efficiency have also been validated on a real-world robotic platform. The implementation of our method has been released at https://github.com/nubot-nudt/SegNet4D Note to Practitioners—This paper focuses on enhancing the efficiency of 4D semantic segmentation for applications in autonomous driving or navigation. Existing LiDAR-based 4D semantic segmentation methods fall short of real-time processing capabilities, thereby severely limiting their practicality for autonomous vehicles and robotic systems. To tackle these issues, we design a high-efficiency 4D semantic segmentation network that not only performs real-time operations on real-world robotic systems but also delivers superior performance, validating its practical utility. Future work can further leverage the instance information we introduced to improve the network’s functionality by achieving panoptic segmentation or 4D panoptic segmentation. Furthermore, some studies may use the 4D semantic labels predicted by our approach to reinforce tasks associated with robotic autonomy.