Learning Multiple Information Long-range Dependencies for Cloud Type Classification

Cloud type classification using multiple information mainly relies on traditional hand-crafted approaches and Convolutional Neural Networks (CNN)-based approaches. However, these methods fail to capture long-range dependencies across multiple information, i.e., cloud images and its one-to-one meteorological element information. Transformer-based approaches can model the long-range dependencies of the multiple information with the attention mechanism. In this paper, we propose to compare three Transformer-based multiple information models: Concatenation-based Self-Attention (Concat-SA), Addition-based Self-Attention (Add-SA) and Bidirectional Cross-Attention (Bi-CA). We systematically evaluate these fusion methods for ground-based cloud type classification. We present a detailed comparison and analysis of their performance on a cloud dataset which contains 4000 cloud samples. Furthermore, we evaluate how the number of layers affects the performance of the model.