Visible-infrared fusion in the frame of an obstacle recognition system

In this article we propose different fusion schemes using information provided by visible and infrared images for road obstacle SVM-based classification. Three approaches for the fusion of VIS and IR information are presented. The early fusion yields a feature vector integrating at the feature level both visual and infrared information. The obtained bimodal feature vector is used as input to an SVM-based classification scheme. The intermediate fusion, which is performed at the kernel level combines different simple kernels of the SVM classifier in order to obtain a multiple kernel (MK). The late fusion combines matching scores of individual obstacle recognition modules in order to improve the system's final decision. In this late fusion case two methods have been considered to calculate the optimum weighting parameter: an Adaptive Fusion of Scores (AFScores) and a non-Adaptive Fusion of Scores (nAFScores). Comparative results showed that fusion-based obstacle recognition systems outperform monomodal visual and infrared obstacle recognizers. An important advantage of these fusion-based systems is their possibility to adapt to the environmental illumination conditions due to the weighting parameter which can contribute to the adjustments of the system's final decision.