Using thermal intensities to build conditional random fields for object segmentation at night

Surveillance systems often make use of the thermal spectrum for improved computer vision application during night time scenarios. In the proposed work we tackle a similar problem of street object segmentation using multispectral information. We propose an efficient model that utilizes spark modules to reduce network parameters and perform a multispectral feature extraction along with multi-scale feature fusion in an encoder-decoder architecture. The predictions are further refined using a conditional random field based on thermal intensity-based pairwise potentials. Overall the network achieves an average pixel-level accuracy of 0.8185 and a mean IOU of 0.6438 for the night dataset which exceeds competing models by a significant margin. The proposed method also performs well for the combined dataset of day and night images. Additionally, it has been established that the use of thermal intensity-based pairwise potential on average is superior when it comes to images captured during the night.