Recognizing Unknown Disaster Scenes With Knowledge Graph-Based Zero-Shot Learning (KG-ZSL) Model

Unseen category prediction is a common challenge for real-world applications, especially for remote sensing (RS) imagery interpretation. Zero-shot learning (ZSL)-based scene classification methods have made significant progress recently, providing an effective solution for unseen scene recognition with semantic embeddings that link seen and unseen classes in the field of RS. However, existing ZSL methods mainly focus on semantic feature exploration, and they failed to combine image features and semantic features effectively. To address the aforementioned challenges, we propose a novel knowledge graph (KG)-based ZSL model that adeptly integrates both image and semantic features for disaster RS scene recognition. First, we construct an RS KG to generate semantic features of RS scenes, enhancing the reasoning ability from conventional RS scene categories to disaster RS scene categories. Second, we propose an interactive-attention mechanism to integrate image and semantic features, focusing on the most informative regions. Finally, we introduce an RS domain adapter that enables the model to better adapt to RS data, reproject common features into the RS domain, and thus solve zero-shot RS scene classification tasks. To demonstrate the effectiveness of our method, we construct an RS disaster scene dataset, which contains 8700 high-quality disaster scenes. Extensive experiments show that our proposed method outperforms current state-of-the-art (SOTA) methods under zero-shot RS image scene classification settings.