Refined Feature Fusion-Based Transformer Network for Hyperspectral and LiDAR classification

Multimodal remote sensing images describe the same surface scene from different viewpoints, and can gain more reliable information about the earth's surface. As a result, fusion classification of multi-modal remote sensing images has gradually become a hot area of research in recent years. There is a large body of work on devising better fusion algorithms. However, if the features of single-modal data sources cannot be thoroughly explored, the classification performance of the network will be significantly degraded. In this paper, a refined feature fusion-based Transformer network(RFFT) for hyperspectral and LiDAR classification is proposed, which effectively compensates for the aforementioned shortcomings. In particular, a shallow specific feature extraction network(SSFE) is designed. It can use a more appropriate feature extractor to obtain the shallow feature of the individual sources. A refined spectral Tokenizer module (RSTM) is then proposed to finely explore richer unique spectral information through grouping and parallel perception. Finally, a spectral intergroup-spatial perception fusion(SISP) is built to capture global multimodal feature dependencies and perform a complete fusion. Experiments results on the public Houston2013 and Trento datasets show that the proposed RFFT performs well.