A Real-Time, Robust, and Versatile Visual-SLAM Framework Based on Deep Learning Networks

In this article, we investigate the paradigm of deep learning techniques to enhance the performance of visual-based simultaneous localization and mapping (vSLAM) systems, particularly in challenging environments. By leveraging deep feature extraction and matching methods, we propose a robust, versatile hybrid visual SLAM framework, Rover-SLAM, aimed at improving adaptability in adverse conditions, such as dynamic lighting conditions, areas with weak textures, and camera jitter. Building on excellent learning-based algorithms of recent years, we designed from scratch a novel system that uses the same feature extraction and matching approaches for all SLAM tasks. Our system supports multiple modes, including monocular, stereo, monocular-inertial, and stereo-inertial configurations, offering flexibility to address diverse real-world scenarios. Through comprehensive experiments conducted on publicly available datasets and self-collected data, we demonstrate the superior performance of our Rover-SLAM system compared to the state-of-the-art (SOTA) approaches. We also conducted an in-depth analysis of the integration of visual SLAM with deep learning methods quantitatively to provide insights for future research endeavors in this domain. The experimental results showcase the system’s capability of achieving higher localization accuracy and robust tracking performance.