Multimodal Features and Accurate Place Recognition With Robust Optimization for Lidar–Visual–Inertial SLAM

Lidar–visual–inertial simultaneous localization and mapping (SLAM) (LVINS) provides a compelling solution for accurate and robust state estimation and mapping, integrating complementary information from the multisensor data. However, in the front-end processing of existing LVINS systems, methods based on the visual line feature matching typically suffer from low accuracy and are time consuming. In addition, the back-end optimization of current multisensor fusion SLAM systems is adversely affected by feature association outliers, which constrains further enhancements in localization precision. In the loop closure process, the existing lidar loop closure descriptors, relying primarily on 2-D information from point clouds, often fall short in complex environments. To effectively tackle these challenges, we introduce the multimodal feature-based LVINS framework, abbreviated as MMF-LVINS. Our framework consists of three major innovations. First, we propose a novel coarse-to-fine (CTF) visual line matching method that utilizes geometric descriptor similarity and optical flow verification, substantially improving both efficiency and accuracy of line feature matching. Second, we present a robust iterative optimization approach featuring a newly proposed adaptive loss function. This function is tailored based on the quality of feature association and incorporates graduated nonconvexity, thereby reducing the impact of outliers on system accuracy. Third, to augment the precision of lidar-based loop closure detection, we introduce an innovative 3-D lidar descriptor that captures spatial, height, and intensity information from the point cloud. We also propose a two-stage place recognition module that synergistically combines both visual and this new lidar descriptor, significantly diminishing cumulative drift. Extensive experimental evaluations on six real-world datasets, including EuRoc, KITTI, NCLT, M2DGR, UrbanNav, and UrbanLoco, demonstrate that our MMF-LVINS system achieves superior state estimation accuracy compared with the existing state-of-the-art methods. These experiments also validate the effectiveness of our advanced techniques in visual line matching, robust iterative optimization, and enhanced lidar loop closure detection.