Multi-modal Channel-attention for Vehicle RGB Image Reconstruction

Existing applications of vehicle detection and monitoring using RGB cameras suffer from low-light conditions where the quality of RGB vehicle images is low. In this paper we propose to reconstruct RGB vehicle images using pairs of RGB image and Infrared image captured by a dual RGB-Infrared camera, and present RIVIC, an RGB-Infrared imaged based Vehicle Image reConstruction scheme such that existing solutions on RGB images can still work even in low-light conditions.RIVIC exploits a multi-modal channel-attention mechanism on paired RGB and Infrared image. Specifically, two convolutional neural networks are firstly utilized on RGB images and Infrared images separately to generate 64-channel feature maps, respectively. These features maps are then concatenated and fed into a channel attention module. As a result, the fusion weights for different channels are computed while preserving the useful information from both visible and infrared images. These fused features subsequently undergo downsampling and upsampling processes, and eventually the reconstructed RGB image is generated. Extensive experiments in real-world scenarios show that the reconstructed RGB images have a high-quality as compared with the groundtruth RGB images, in terms of Structure Similarity Index Measure,Mutual Information,the first-order Entropy and the second-order Entropy.