Self-Localization of a Biomimetic Robotic Shark Using Tightly Coupled Visual-Acoustic Fusion

This article proposes a tightly coupled visual-acoustic sensor fusion method for self-localization of a biomimetic robotic shark. To address the decreased localization accuracy of visual-based simultaneous localization and mapping systems employed on a robotic fish in underwater environments, we integrate velocity measurements from the acoustic sensor Doppler velocity log (DVL) into a visual odometry. To fully exploit the local position change information contained in velocity measurements, DVL measurements are fused in two stages of visual tracking. Specifically, we first employ the velocity measurements to improve the initial camera pose estimation during visual tracking, aiming to provide a better initial value for subsequent pose optimization. Thereafter, these velocity measurements are directly employed to constrain the camera position change between two adjacent frames by constructing a DVL residual term, which is optimized jointly with the visual residual to obtain a more accurate camera pose. Extensive experiments are conducted on both self-collected simulated datasets and real-world underwater datasets. Experimental results demonstrate that the proposed visual-acoustic fusion method can effectively improve the localization accuracy for the robotic shark by more than 50% compared to a pure visual system, providing valuable guidance for improving the autonomous localization capability of underwater biomimetic robots.