STS-SLAM: Joint Visual SLAM and Multi-Object Tracking Based on Spatio-Temporal Similarity

Visual Simultaneous Localization and Mapping (SLAM) is a critical technique for intelligent vehicles and autonomous driving. Most SLAM systems assume the environment to be static or treat dynamic features as outliers for better localization performance. However, clear information about dynamic objects is crucial for localization and decision-making in complex environments. This paper proposes STS-SLAM, a tightly-coupled visual simultaneous localization and multi-object tracking system in dynamic scenes. It can synchronously optimize the motion of the ego-vehicle and objects and estimate object velocity without any prior information about the object. To accurately cluster features, we design a feature metric based on spatio-temporal similarity (STS), which considers the intrinsic properties and current state of the features. All static features are employed for ego-vehicle localization, and features with high STS on dynamic objects are robustly tracked. Furthermore, we propose an adaptive scaling covariance kernel (ASCK) algorithm based on STS to deal with perceptual noise and outliers, which avoids manual optimization of kernel parameters. The STS-SLAM problem is modeled as a dynamic constraint factor graph for joint optimization of dynamic and static structures. Finally, evaluation results on the KITTI tracking dataset, Oxford multi-motion dataset, and real-world scenarios show that the proposed algorithm obtains higher localization accuracy and smaller tracking errors than other state-of-the-art SLAM algorithms.