Learning Local and Global Feature Representation for Hyperspectral and LiDAR Data Fusion

Hyperspectral image (HSI) records plentiful spectral-spatial information of land covers while the light detection and ranging (LiDAR) reflects the elevation and structural elements. HSI and LiDAR data fusion can gain finer image classification result. However, HSI and LiDAR provide different object characteristics, and thus, how to accurately represent the intrinsic characteristic of the input is a challenging issue. In this work, a local and global feature representation method (LGFR) is studied for HSI and LiDAR data fusion. First, a two-branch network architecture is designed to excavate the spectral-spatial attributes of the source images, in which the transformer network is utilized to built the long-range dependency of the spectal data and the convolutional neural network is exploited to acquire the local spatial information. Then, a multilevel fusion strategy is designed to extract the common feature, difference feature, and the intrinsic feature. Finally, the fully connected layer is to carry out on three features to derive class scores followed by a decision fusion scheme to generate the ultimate classification outcome. Experiments on two public hyperspectral and LiDAR data, i.e., Houston 2013 and MUUFL Gulfport scenes show that the proposed LGFR greatly surpasses other advanced multisource data fusion models.