TON-VIO: Online Temporal Calibration Network Learning On-the-Fly in Fast-Motion VIO

This article is aimed at addressing online temporal calibration in fast-motion visual-inertial odometry (VIO) systems. Unlike stable motion VIO, accurately modeling the time offset is significantly challenging in fast-motion systems. Fast motion not only enlarges the estimation drift caused by the time offset, but also introduces significant difficulties in time offset modeling. In this scenario, existing classic methods struggle to cope with the unknown time-varying time offset modeling problem. Meanwhile, current deep learning networks and training strategies incur high computational costs and lack generality. To address these issues, we propose lightweight online time offset modeling networks (TONs) for real-time temporal calibration. TON is a module for general VIO systems and improves the accuracy of time offset modeling in three ways: 1) for observation modeling, feature velocity observation networks (FVONs) are proposed, which compute feature velocity under unstable visual tracking conditions without relying on inaccurate inertial measurement unit (IMU) velocity propagation; 2) for prediction modeling, time offset prediction networks (TPNs) are designed, which learn the evolution pattern of time offset to overcome the constant value assumption in traditional approaches; and 3) for network training, an online weakly supervised training strategy is adopted. It uses partial results as weak labels from classic models to supervise TON online. Furthermore, cross-scene/vehicle experiments highlight TON’s effectiveness and generality in fast motion. Through online temporal calibration enhancement, our TON-VIO achieves average positioning accuracy improvements of 15% over optimization-based methods, 42% over filter-based methods on public test sequences, and 32% over optimization-based methods on self-collected datasets with raw timestamps.