Patch-Based Transformer Network Construction With Adaptive Feature-Interaction for Hyperspectral Image Classification

Hyperspectral image classification (HSIC) is widely used in such fields as vegetation classification and fine agriculture. Nowadays, numerous classification models based on Transformer networks are inputting data typically in the form of patches. However, such patch-based input format may weaken the spatial relationships between neighbor pixels, thus limiting the ability to obtain contextual semantic information. Additionally, hyperspectral images (HSIs) generally contain hundreds of bands, which in turn include some redundant information. The original spectral vectors may lead to redundancy of information, thus reducing the classification accuracy remarkably. Therefore, in this article, a patch-based transformer network construction with adaptive feature-interaction (AFi) for HSIC called AFinet is developed for HSIC. As an end-to-end network, AFinet consists of a feature extraction module and an AFi module. For the developed model, the feed-forward neural network (FNN) in the standard Transformer framework suffers from a limited ability in exploiting local contexts. In this article, an expression-enhanced FNN (E2FNN) is introduced, which can incorporate depthwise convolution layers to capture local contextual information, so as to enhance the correlations between neighbor pixels. Moreover, this study designs an AFi module to facilitate the sharing of interaction features among Transformer-extracted features. Subsequently, global spatial feature information can be integrated into each spectral channel. The AFi module also includes a channel attention mechanism to help focus more attention on channels that contain critical information, while paying less attention to channels with little critical information. After the approach proposed in this study was tested on three widely used datasets, the experimental results demonstrate that the AFinet developed in this study can effectively improve classification accuracy.