Neural Visual-Inertial Odometry with Integrated Mechanical Choreography

With the extensive application of deep learning, neural network-based Visual Inertial Odometry (VIO) has demonstrated significantly better performance than traditional geometric VIO. Deep learning-based VIO fuses information from vision and inertial measurement unit (IMU) to learn positional changes in an end-to-end data-driven manner. However, existing studies have generally focused only on the extraction of representations from raw data, ignoring the correlations and interactions between modalities. Moreover, these end-to-end networks are often criticized for their limited interpretability. This study proposes an innovative deep learning VIO model that incorporates mechanical choreography. The model enhances the network with prior positional information through the integration of results from IMU mechanical choreography. It leverages multimodal feature learning concepts and employs a cross-modal co-attention mechanism to effectively merge features extracted from images and IMUs. This approach aims to achieve highly accurate, efficient, and interpretable position estimation tasks. Finally, the trained model is tested on a public dataset and the accuracy is compared with traditional methods and classical neural VIO to verify the effectiveness and feasibility of the proposed method in this study.