Mine Diversified Contents of Multispectral Cloud Images Along With Geographical Information for Multilabel Classification

Multispectral multilabel cloud image classification (MSMLCIC) aims to predict a set of labels presented in a multispectral (MS) cloud image, which usually contains more than one cloud type or weather system. However, the exploration of diversified contents reflected by multiple bands of MS image is limited and the consideration of geographical information (time and location information) is insufficient. To cope with the abovementioned problems, this work proposes the multispectral cloud image multilabel classifier with group feature extractor and geo-queries (MS-GoGo). With a group feature extractor, different bands of MS images are processed separately according to the content they reflected, and a group of distinctive yet complementary image features are generated. Geo-queries are responsible for implicitly embedding different labels with time and location information to probe the corresponding similar semantic ingredients. Due to the coarse classification of the existing dataset, a new dataset named LSCIDMR-V2 is generated with fine-grained cloud-type annotation and multichannel data. The experiment shows that, using the group feature extractor and geo-queries, the popular used metric subset accuracy is improved from 40.06 to 42.87 and 44.35, respectively. The proposed method achieves the mean average precision of 82.40, outperforming state-of-the-art methods.