TriTF: A Triplet Transformer Framework Based on Parents and Brother Attention for Hyperspectral Image Change Detection

Hyperspectral image change detection (HSI-CD) is a technique to accurately detect land cover changes by using HSIs with rich spatial–spectral information. In recent years, the HSI-CD methods based on convolutional neural networks (CNNs) have achieved great success because of their flexible and effective feature extraction ability. However, these methods often take the HSI patches as the input of the networks, which undoubtedly hinders the overall perception of the HSIs. Meanwhile, the valuable temporal information in HSIs is often underutilized. For this end, a triplet transformer framework (TriTF) based on parents-temporal attention (PTA) and brother-spatial attention (BSA) is proposed for HSI-CD. The proposed framework mainly contains the following three parts: 1) transformer-based network backbone, which uses the self-attention to capture the correlation between arbitrarily two pixels in the same patch and extracts the global spatial correlation in the unit of encoded input patches; 2) PTA branch; unlike the previous cross-temporal attention mechanisms of the “
$\text{T}1\leftrightarrow $
T2” mode which only consider the interaction between bitemporal HSIs, this article constructs a novel PTA of the “
$\text{T}1\rightarrow \text{T}3\leftarrow $
T2” mode, which takes the difference-temporal image T3 as the core and the impact of bitemporal HSIs on the land cover changes is more concerned in the PTA; and 3) BSA branch. The most similar patch in the current training batch of each patch is defined as its brother patch. Furthermore, cross-spatial attention is applied to propagate the features of the brother patch to the current patch. Thus, the middle- and long-range dependencies can be utilized and the scope of feature propagation can be extended. In this article, the experiments under low and high sampling rates are conducted and proved the outstanding change detection performance of the proposed TriTF when compared with abundant state-of-the-art (SOTA) CD algorithms. The source code of this article will be released at
https://github.com/zkylnnu/TriTF
.