A Proposal to Integrate ORB-Slam Fisheye and Convolutional Neural Networks for Outdoor Terrestrial Mobile Mapping

SLAM methods, such as ORB-SLAM, can build a map of an unknown environment (sparse point cloud) with optical images. The sensor motion provides image sequences over which keypoints are extracted and matched, enabling the simultaneous computation of sensor locations and 3D coordinates of points. In the last years, enormous progress has been done to solve the SLAM problem, especially focusing on computational efficiency and accurate sensor trajectory estimation. However, the auto-detection of incorrect or undesired match points (outliers) to support the auto-decision of include or not an image observation in the estimation process is still an open problem. ORB-SLAM fisheye is applied in this study to estimate sensor trajectory based on dual-fisheye images acquired with Ricoh Theta S omnidirectional camera in a terrestrial mobile mapping system carried by a backpack. This preliminary study demonstrated the possible effects of image observation outliers in the sensor trajectory estimation (planimetric and altimetric accuracy of 0.381m and 0.26m, respectively). A proposal to combine semantic segmentation using CNN in the photogrammetric process workflow to cope with this problem and detect potential image observation outlier areas is presented.