Locality-Constrained Bilinear Network for Land Cover Classification Using Heterogeneous Images

Optical and SAR modalities provide complementary information of land properties, which can lead to outstanding classification performance. Recently, factorized bilinear coding (FBC) as an extension of bilinear pooling in respect of coding-pooling perspective, which extracted compact bilinear fusion features with second-order interaction information in the form of sparse representation, brought the performance improvements on multimodal learning tasks. However, it lost locality attributes among similar samples to be encoded. In this letter, we propose a novel locality-constrained bilinear network (LC-BNet) for land cover classification with heterogeneous remote sensing (RS) images. Specifically, the locality-constrained bilinear coding (LC-BC) introduces locality information to generate compact and discriminative fusion features for land cover classification. Extensive experimental results show superior performances of our work on two broad coregistered optical and SAR datasets.