SDS-SLAM: VSLAM Fusing Static and Dynamic Semantic Information for Driving Scenarios

Visual semantic SLAM integrates geometric measurements with semantic perception, making it widely applicable in autonomous driving and robotics. Semantic-assisted localization and dynamic object perception are two critical tasks in visual semantic SLAM. However, many existing state-of-the-art methods address only one of these tasks in isolation. To address issues of functional limitations and insufficient information utilization in a single framework, we propose a unified visual semantic SLAM framework, SDS-SLAM, which tightly couples static and dynamic semantic information to handle the motion estimation of both the camera and observed objects in driving scenarios. A multi-task network for driving perception is employed to extract semantic information, including drivable areas, lanes, and vehicles. Based on various information obtained, we propose semantic local ground manifolds (SLGMs) to represent the geometric structure and semantic features, enabling the online generation of a lightweight semantic map. Subsequently, we integrate SLGM-based constraints such as lane alignment and planar motion to promote camera and object pose estimation. We evaluated our method on the public KITTI dataset and self-collected real-world data. The results demonstrate that our method effectively perceives both dynamic and static semantic elements in driving scenarios, achieving high accuracy in estimating the poses of the camera and objects. Note to Practitioners—This work aims to utilize semantic information to assist ego-localization and enhance dynamic object motion perception in a unified visual SLAM framework, which mainly serves decision-making and planning for ground-based intelligent robots such as autonomous-driving vehicles. Existing methods typically deal with semantic-assisted localization or dynamic object tracking in isolation, thus limiting their ability to fully leverage the semantic richness of the environment and ignoring the mutual constraints between semantic information. By introducing semantic local ground manifolds, which couple road planes and lane lines for traffic scene representation, this work improves camera pose estimation accuracy through joint optimization with additional geometric constraints, including lane alignment, coplanarity, and planar support. Moreover, the planar motion constraints provided by SLGMs effectively reduce cumulative drift in object pose estimation, enabling more robust and accurate object tracking and motion perception over the long term.