A Deep Learning-Based Semantic Filter for RANSAC-Based Fundamental Matrix Calculation and the ORB-SLAM System

The estimation of a fundamental matrix (F-matrix) from two-view images is a crucial problem in epipolar geometry, and a key point in visual simultaneous localization and mapping (VSLAM). Conventional robust methods proposed by the data calculation space, such as Random Sample Consensus (RANSAC), encounter computational inefficiency and low accuracy when the outliers exceed 50%. In this paper, a semantic filter-based on faster region-based convolutional neural network (faster R-CNN) is proposed to solve the outlier problem in RANSAC based F-matrix calculations. The semantic filter is trained using semantic patches tailored by inliers, providing different semantic labels in various image regions. First, the patches classified into the top three bad labels are filtered out during the pre-processing phase. Second, precise and robust correspondences are determined by the remaining high-level semantic contexts. Finally, the inliers are assessed using RANSAC to produce an accurate F-matrix. The proposed algorithm can improve the accuracy of F-matrix calculations, as low-quality feature correspondences are effectively decreased. Experiments on KITTI and ETH sequences illustrate that the 3D position error can be reduced by applying the semantic filter to the ORB-SLAM system. Further, indoor and real environment experiments demonstrate that an effective lower trajectory error is yielded with the proposed approach.