Hyperspectral Image Reconstruction with Unseen Material Detection

Reconstruction of hyperspectral images (HSIs) from their RGB measurements is an ill-posed inverse problem. The key to successful reconstruction relies on establishing an effective HSI prior, for which deep learning techniques have achieved impressive performance. However, the scarcity of large-scale HSI datasets poses a significant challenge, limiting the practical application of deep HSI reconstruction methods and often leading to incorrect results when dealing with unseen substances or materials. To tackle this challenge, we propose a deep RGB-to-HSI reconstruction model based on the sparse prior of hyperspectral signals. The network can effectively correlate HSI and RGB features via shared sparse codes, representing the weights of spectral-unique materials. Besides, testing images with unseen materials can be detected by measuring their sparse modeling errors. Experimental results demonstrate that the proposed method achieves promising results on RGB-to-HSI reconstruction. Further, the sparse modeling error evidently demonstrates its efficacy as an indicator for unseen materials.