Multi-spectral visual odometry for unmanned air vehicles

With the recent increase of interest concerning multi-spectral systems, limitations of classic stereo setups have been overcome to tackle complex navigation problems such as night-time navigation or collision avoidance. However, multi-spectral stereo matching still remains a challenging issue as similarity between stereo pairs is reduced. In this work, we address the problem of visual navigation for multi-modal stereo setups. More precisely, we focus on finding correspondences between visible and long-wave infrared (thermal) images for pose estimation purposes. We present a new visual odometry method for air vehicles using mutual information and phase congruency. Mutual information performs an efficient cross-modality statistical analysis and phase congruency provides a robust spatial information. Hence, both of them are used as criteria for multi-spectral stereo matching. Temporal matching is then performed using feature tracking in each modality separately. This way, cross-modality processing is kept to a minimum, which reduces inaccuracy. This method provides an important number of quad-matches (stereo and temporal), which helps to select the best keypoints for computing a precise motion estimation. In our case, the selection process is performed using a RANSAC scheme. Extensive experimental results show the attractiveness of the proposed technique as a navigation solution. Moreover, results show that tracking features in both modalities makes the technique robust to abrupt navigation changes.