Synergizing Vision and Language in Remote Sensing: A Multimodal Approach for Enhanced Disaster Classification in Emergency Response Systems

As remote sensing capabilities continue to advance, there is a growing interest in leveraging computer vision and natural language processing for enhanced interpretation of remote sensing scenes. This paper explores the integration of textual information with images to augment traditional disaster classification methods. Our approach utilizes a predefined vision-language model to generate descriptive captions for images, fostering a more nuanced understanding of the remote sensing data. Next, we seamlessly integrate the generated textual information with image data through multimodal training, employing a multimodal deep learning method for disaster classification. The system categorizes input data into predefined disaster categories, presenting a comprehensive and accurate approach to emergency response system development. Experimental evaluations conducted on the AIDER dataset (Aerial Image Database for Emergency Response applications) showcase the efficacy of our approach, demonstrating improved accuracy compare to unimodal approach and reliability in disaster classification. This research contributes to the advancement of intelligent emergency response systems by harnessing the synergy between vision and language in the context of remote sensing.