The PAN and MS Image Pansharpening Algorithm Based on Adaptive Neural Network and Sparse Representation in the NSST Domain

How to improve the spatial resolution as much as possible while maintaining the spectral information of multi-spectral (MS) image in the field of image fusion is of great significance for practical applications, such as map updating, feature classification, and target recognition. To analyze the coefficients of the subband distribution characteristics, in this paper, we propose a new panchromatic (PAN) and MS image pansharpening model based on an adaptive neural network and sparse representation in the non-subsample shearlet transform (NSST) domain. First, this algorithm is specific to regional directional characteristics in the high-frequency subband of PAN and MS images, and we propose an adaptive pulse coupled neural network (PCNN) model. The model can adaptively calculate the link strength of a neural cell based on the region energy. Furthermore, we apply the model to the high-frequency fusing process with the corresponding fusion rule, and the rule can distinguish the high-frequency coefficients by ignition times, which can more effectively capture the geometric texture information and detailed information in the PAN image, enhancing the spatial resolution of the fused image. Second, because of the low-frequency sub-bands from the PAN image and I component obtained by intensity-hue-saturation (IHS) transformation of the MS images with high similarity to the original image but poor sparsity, we select a set of PAN images for learning, a more targeted over-complete dictionary for low-frequency sub-band sparse representation is obtained. Then, the larger absolute value of the sparse matrix is selected to obtain the low-frequency coefficients for the fusion image while maintaining the MS spectral information effectively, and the representation of characteristic information of low-frequency subband is more effective. A large number of simulation experiments verify the effectiveness of the proposed method.