Optical and Synthetic Aperture Radar Image Fusion for Ship Detection and Recognition: Current state, challenges, and future prospects

The detection and recognition of ships holds crucial significance across various domains, including transportation, fishing, smuggling prevention, and rescue operations. To achieve accurate and efficient ship detection and recognition, remote sensing technologies, specifically, optical and synthetic aperture radar (SAR) sensors, have been widely employed. However, each sensor modality presents its own limitations, such as cloud cover in optical images and interference in SAR images, resulting in restricted data availability and false alarms during detection. This article provides a comprehensive summary of recent advancements in the fusion of optical and SAR images for ship detection and recognition. Meanwhile, the fusion of remote sensing images and in situ data, such as automatic identification system (AIS) data, is briefly summarized, considering significant benefits and technical trends. This article highlights the immense potential of integrating these two sensor modalities to overcome individual challenges and enhance overall system performance. One of the primary challenges addressed in this work pertains to accurately matching ship targets between optical and SAR images when ships change positions. However, recently developed multimodal fusion networks and transfer learning techniques have the potential to tackle the challenges, even with unmatched optical and SAR image pairs. In addition to summarizing existing techniques, the article delves into future perspectives on fusion-based ship detection and recognition. In conclusion, this survey highlights the current state of optical and SAR image fusion techniques, pinpoints existing challenges, and outlines future research prospects, emphasizing the significance of fusion-based approaches in shaping the future of maritime applications.