Hyperspectral Image Reconstruction From RGB Input Through Highlighting Intrinsic Properties

Dozens of spectral bands of hyperspectral images (HSIs) have been successfully reconstructed from only three color band images using deep neural networks according to their powerful nonlinear mapping capability. However, the existing deep-learning-based approaches tend to directly reconstruct HSIs from RGB inputs without emphasizing the discriminative intrinsic properties of different materials, resulting in certain distortion in reconstructed spectra. In this article, an intrinsic image decomposition (IID)-based spectral super-resolution (SSR) framework is proposed to reconstruct spectra of pixels from their reflectance feature and shading feature separately, by which the intrinsic properties can be emphasized during spectral reconstruction. Specifically, a dual hierarchical regression network (DHRNet) is designed for the proposed IID-based SSR task, in which a shading feature extraction module (SFEM) based on dense structure and a reflectance feature extraction module (RFEM) with attention mechanism are first, respectively, designed to reconstruct spectral information from reflectance feature and shading feature, and a feature enhancement module (FEM) is consequently devised to further improve the coarse combined estimation. Ultimately, a novel hybrid loss combining smooth $\boldsymbol {l}_{1}$ loss, spectral angel mapper (SAM), and gradient prior is also presented to restrain the spectral distortion while enhancing the sharpness of the reconstructed HSI. Experimental results over three datasets demonstrate the superiority of our proposed framework.