Image Fusion Using Convolutional Neural Network with Bilateral Filtering

Image fusion is a method of combining source images taken from the same scene. A deep convolutional neural network (CNN) is used in this paper to extract the high frequency details from the two source images. A focus map is generated after the several convolution and max-pooling layers which contains the clarity information of the source images. A fixed threshold is applied to the focus map to generate a binary segmented map which correctly classifies the pixels belonging to the focused regions. The results of binary segmentation contain some mis-classified pixels which is improved by applying a small region removal strategy to get the initial decision map. The proposed bilateral filter is a very efficient edge-preserving filter which smoothen the regions around the boundaries of the obtained decision map. The pixel-wise weighted average strategy is calculated to get the fused image with high visual quality. Experimental results show that the proposed CNN-based method produces more natural effect of the fused image.