ESNet: Anti-Hierarchical Encoder–Decoder Network With Surrounding Expansion and Stripping for Hyperspectral Image Classification

Spatial information is crucial in deep spectral–spatial hyperspectral image (HSI) classification methods. Spatial features can be divided into central features and surrounding features, which have different significance in high-precision HSI classification tasks. However, the existing algorithms ignore the discrepancy between the two attributes, which weaken decisive characteristics and lose spatial structures. To resolve the above issues, a patch-based anti-hierarchical encoder–decoder network is proposed, namely expansion and stripping network (ESNet). First, a surrounding relation module (SRM) is proposed to change the patch scale. Unlike existing scale change modules, the SRM increases the scale in the encoder and decreases the scale in the decoder by controlling the expansion and stripping of peripheral pixels of the patch, which enables ESNet to amply exploit the critical central features while utilizing the surrounding information in the anti-hierarchical structure. Second, we design a spectral–spatial attention module (SSAM) to extract features. To better represent land cover attributes, SSAM employs large- and small-kernel convolutions to generate spatial, spectral, and spectral–spatial weighted features. Finally, a novel skip connection [deep shallow feature fusion module (DSFM)] is designed to promote the model to fuse deep and shallow features while preserving spectral sequences of patches. Combining DSFM with the encoder–decoder structure can more scientifically fuse deep semantics and shallow details. We conduct extensive experiments in six commonly used HSI datasets to demonstrate the superiority of the proposed model. The codes will be available from the website.