Dual Embedding Transformer Network for Hyperspectral Unmixing

Hyperspectral unmixing is an essential task for achieving accurate perception of hyperspectral remote sensing information, aiming to overcome the limitation of spatial resolution and interpret the distribution of land features. To achieve the spatial and spectral feature representation of hyperspectral images, we propose a dual embedding transformer network (DET-Net) based on an encoder-decoder architecture, which utilizes two transformer modules, including three-view spatial attention(TVA) module with 2D embedding and multi-scale spectral band group feature fusion(BGF) module with 3D embedding to accomplish the task of hyperspectral unmixing. In TVA module, based on 2D embedding, we introduce a three-view attention mechanism to extract more comprehensive spatial features. In BGF module, the transformer embedding is extended to band group spatial-spectral 3D cubed embedding and establishes a series of spectral band groups. A cross-feature fusion mechanism is adopted to achieve multi-scale spatial-spectral feature decoupling. With the collaboration of these two embeddings, DET-Net effectively captures complex spatial and spectral dependencies to decouple the tridimensional unmixing feature representation. Experimental results on synthetic and real datasets demonstrates the generalization performance of the proposed method, and the ablation experiments confirm the effectiveness of the TVA and BGF modules.