FusePose: A Monocular Camera Position Estimation Network Based on Multi-Information Fusion

Current deep learning-based camera position estimation methods fail to fully mine and utilize the effective information in the scene and are susceptible to motion blur and point-of-view variations, resulting in low accuracy of the estimated camera position. To solve the above problems, we design a new camera position estimation network—FusePose. FusePose realizes the fusion of odometry information and semantic information through the temporal-aware localization module (LSVO) and the semantic information-aware module (SEMA), which effectively achieves a more lightweight and high-precision pose estimation. Specifically, the LSVO module employs an improved double-helix network architecture to extract odometry information while incorporating a long short-term memory (LSTM) submodule to mitigate trajectory drift in visual odometry. The SEMA module utilizes a quad-branch network to extract semantic features from segmentation maps. It adopts an adaptive weighted fusion approach to integrate multiscale semantic features, thereby optimizing the information contribution ratio across different feature scales. The experimental results show that the discrete points of the camera localization trajectory of FusePose on the 7-Scenes dataset are further reduced, the trajectory is clearer, and the errors are further reduced, including the translation error by 31.6% and the rotation error by 27%. It is worth noting that the 7-Scenes dataset does not provide semantic labeling, so we built our own semantic segmentation graph dataset based on the 7-Scenes dataset.