Stereo Visual Inertial SLAM with Semi-direct Residual

The existing feature based simultaneously localization and mapping (SLAM) systems such as ORB-SLAM only use sparse keypoints as visual constraints in pose optimization. However, the rich photometric information that may be useful for pose optimization is discarded. In order to improve accuracy, we introduce a novel semi-direct residual into both the front-end and back -end of a stereo visual inertial SLAM system. The semi-direct residual is constructed in triangular sub-image patches formed by local keypoints. It reflects the stable local photometric information between two aligned images. The evaluations on the EuRoC MAV dataset and our indoor parking lot dataset prove that the semi-direct residual is able to improve system accuracy. The results also show that the proposed system achieves highly competitive accuracy, while having better realtime performance than ORB-SLAM3.