A Two-step Training Transformer Framework for Recovering Hyperspectral Images from RBG Images

The recovery of hyperspectral images from RGB images is done by discovering the inverse function to extract the hyperspectral information from RGB images. Current deep learning methods mainly establish the relationship between the totals from the data by a neural network, but the unique information of each data is not considered and the generalization is poor. To solve this problem, we propose a two-step training Transformer framework, where the first step is to learn a forward and inverse network by pre-training, and the second step is to fix the inverse network to fine-tune the forward network based on the pre-training. The method can be fine-tuned for each data and thus has better generalization ability for unique data friendly. The experimental results show that the method in this paper does work, and fine-tuning gives better results than the forward-only network, and can be fine-tuned for each data, which is friendly to unique data.