Unmixing Guided Unsupervised Network for RGB Spectral Super-Resolution

Spectral super-resolution has attracted research attention recently, which aims to generate hyperspectral images from RGB images. However, most of the existing spectral super-resolution algorithms work in a supervised manner, requiring pairwise data for training, which is difficult to obtain. In this paper, we propose an Unmixing Guided Unsupervised Network (UnGUN), which does not require pairwise imagery to achieve unsupervised spectral super-resolution. In addition, UnGUN utilizes arbitrary other hyperspectral imagery as the guidance image to guide the reconstruction of spectral information. The UnGUN mainly includes three branches: two unmixing branches and a reconstruction branch. Hyperspectral unmixing branch and RGB unmixing branch decompose the guidance and RGB images into corresponding endmembers and abundances respectively, from which the spectral and spatial priors are extracted. Meanwhile, the reconstruction branch integrates the above spectral-spatial priors to generate a coarse hyperspectral image and then refined it. Besides, we design a discriminator to ensure that the distribution of generated image is close to the guidance hyperspectral imagery, so that the reconstructed image follows the characteristics of a real hyperspectral image. The major contribution is that we develop an unsupervised framework based on spectral unmixing, which realizes spectral super-resolution without paired hyperspectral-RGB images. Experiments demonstrate the superiority of UnGUN when compared with some SOTA methods.

I. Introduction

Hyperspectral image (HSI) contains abundant spectral information, thus it has been applied in quite a few fields, such as scene classification [1], [2], [3], target detection [4], [5] and segmentation [6]. However, due to the limitations of hardwares, it is hard to obtain HSI with high resolution in spatial domain, which leads to a new research direction: hyperspectral super-resolution. Hyperspectral super-resolution is designed to generate high resolution HSIs from low resolution HSIs or high resolution RGB images. Based on the inputs, the recent methods can be divided into three categories: spatial super-resolution of HSI, spectral super-resolution of RGB image, and fusion-based super-resolution.