ME-Seg&DLS-Net: A Dataset and a Network for Autonomous Driving Based on Multi-Element Semantic Segmentation of Pavement

The segmentation of multiple elements of the road pavement is an essential task of the autonomous driving perception system, particularly for the accurate recognition of drivable areas, lane lines, and road signs. Currently, publicly available datasets for research in this field are lacking, and state-of-the-art algorithm models can only identify individual pavement elements with specificity. Based on this, we constructed the ME-Seg dataset for pavement multi-element semantic segmentation, which comprises three training labels: drivable areas, lane lines, and road traffic signs. This dataset captures various traffic, lighting, and weather conditions. Next, we propose a real-time semantic segmentation algorithm model for autonomous driving on road surfaces called DLS-Net based on visual sensors. This network adopts a dual-branch network structure and can accurately identify drivable areas, lane lines, and road traffic signs simultaneously. Within the network, we designed a Multi-Scale Fusion Module (MSFM) to expand the receptive field and include large objects within the field of view. Additionally, we designed a Cross-Guided Aggregation Module (CGAM) to provide guidance information for deep and shallow feature maps while compensating for information loss in small objects. The experimental results based on ME-Seg demonstrate that the mIoU of the proposed network reaches 75.61% at an inference speed of 49.5 fps. The results of real-world vehicle experiments demonstrate that the model exhibits good performance and robustness in actual autonomous driving scenarios.