Combining the Convolution and Transformer for Classification of Smoke-Like Scenes in Remote Sensing Images

Remote sensing (RS) images are used in a wide range of tasks. In the fire detection field, smoke in RS images is considered as an indicator of wildfires. However, smoke-like scenes, e.g., cloud, in RS images increase the difficulty of smoke recognition. Convolutional neural networks (CNNs) have greatly promoted the development of image processing. CNNs are good at capturing local features; however, their ability to capture global features is relatively weak. Recently, the transformer deep learning model has shown strong potential in vision tasks. The transformer model utilizes self-attention modules to extract global features but may lose local details. Recognition of smoke in RS images depends strongly on the combination of both local and global features. Thus, this article proposes the transformer enhanced convolutional network (TECN) to classify RS smoke-like scenes. The proposed hybrid TECN model exploits the advantages of the CNN and transformer techniques at the same time. In TECN, the feature merge and intelligent aggregation modules are used to promote conversion and aggregation between CNN feature maps and transformer patch embeddings. Experiments are conducted on the USTC_SmokeRS dataset, which is developed for the classification of RS smoke-like scenes. The experimental results demonstrate that the proposed TECN achieves a competitive accuracy of 98.39% on this dataset.