A real time pixel-level based image fusion via adaptive weight averaging

A novel pixel-level image fusion scheme for thermal and visual images is presented. The image fusion technique rests on physical characteristics of targets deemed of interest in a surveillance scenario. Each picture element (pixel), in both the thermal and visual images, is assigned a weight proportional to the interest associated with it. Interest is defined as "not natural" or "man-made". A weighted average of the intensity images representing the thermal and visual modalities is then performed for every corresponding pair of visual and thermal picture elements to obtain the fused image. For the thermal images, elements that are warmer or cooler than their environment (background) are deemed to be of "interest". To this end, the thermal weights are associated with the divergence of the intensity of these pixels from the image mean intensity. For the visual images, the facts that the "targets of interest" are usually larger than the instantaneous field of view (IFOV) of the visual sensor and have a reflection behaviour that is more specular are used. The visual weight determination is based on the local variance in space and time of the intensity of the visual pixels, The performance of this technique is compared to a number of existing techniques in the literature. The results reveal that the proposed technique performs better than those in the literature. In addition, it also reveals that the proposed technique is more robust than those in the literature.