Loading [MathJax]/extensions/MathMenu.js
Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition | IEEE Journals & Magazine | IEEE Xplore

Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition


Abstract:

Detailed and accurate identification of disaster geomorphology is important in the governance, development, and emergency decision-making of shallow water areas in estuar...Show More

Abstract:

Detailed and accurate identification of disaster geomorphology is important in the governance, development, and emergency decision-making of shallow water areas in estuaries. However, traditional disaster geomorphology monitoring processes are constrained by the low efficiency of multisource heterogeneous data fusion and the poor stability of recognition models. In this study, we seamlessly acquired shallow water bathymetry and topographic data using unmanned aerial vehicle (UAV) light detection and ranging (LiDAR) and unmanned surface vehicle (USV) multibeam sonar, and proposed an attention-aware multimodal multiscale cross-fusion geomorphology recognition network called AMMNet. First, we adopted a three-stream encoder–decoder structure and designed an attention mechanism-based multimodal fusion module (AMFM) for riverbank morphology and texture to better utilize the fused enhanced features at multiple extraction stages. Second, we added an atrous spatial pyramid pooling (ASPP) module between the encoder and decoder to aggregate feature information of disaster geomorphology at different scales. Finally, an integrated approach combining gradient-weighted class activation mapping (Grad-CAM) and Shapley additive explanations (SHAPs) was employed to analyze the decision-making process of the model and disaster mechanisms. To evaluate the performance of the AMMNet model, it was compared comprehensively with other typical recognition models using an UAV–USV dataset containing 16 types of morphological and textural features. The results of this study demonstrate the superiority of the AMMNet model in recognizing hazard features compared with other models. Specifically, the intersection over union (IOU) for riverbank collapse, scarps, scour pits, and dunes reached 93.7%, 92.2%, 87.2%, and 94.2%, respectively. Furthermore, this study found that low entropy and a high topographic position index (TPI) were the main triggering factors for riverbank collapse. In addition, low e...
Published in: IEEE Sensors Journal ( Volume: 25, Issue: 7, 01 April 2025)
Page(s): 11545 - 11562
Date of Publication: 19 February 2025

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.