Multimodal visual-inertial odometry for navigation in cold and low contrast environment

Multispectral setups have a great potential to tackle problems such as collision avoidance or pedestrian detection. However, multispectral odometry is nowadays less precise than most stereo visible setups, so there is a need to improve such systems. With this work, we investigate the fusion of inertial data with visual information to improve the performance of multispectral setups. Inertial data are included in the process model of an extended Kalman filter to estimate the pose of a vehicle. But inertial measurement units drift rapidly when integrated for a certain period of time. Visual odometry is used to correct the predicted pose and reduce this drift. IMU data are also used to provide a pose estimation when images are too noisy to compute a motion (e.g. motion blur or low contrast). Therefore, we present a new multispectral (visible/LWIR) navigation system able to cope with fast motions and to operate in cold environments, where the contrast of infrared images is reduced. We demonstrate the robustness of the setup on a series of semi-urban datasets acquired from a car. An average error inferior to 3% of the distance traveled between the estimated trajectory and the GPS track is achieved on all the datasets.