Off-Road Trafficability Assessment with Remote Sensing Imagery and Incomplete Auxiliary Data via a Cross-Modal Channel Feature Fusion Network

Off-road trafficability (ORT) in complex geological environments is crucial for special operations, emergency rescue, and natural resource development. ORT is influenced by a combination of geographical and geological environmental factors. Recent studies primarily employ rule methods for ORT assessment. These approaches are labour-intensive, lack timeliness and objectivity, and are constrained to limited data sources. To address these issues, this work proposes a new method for ORT assessment that combines remote sensing imagery with geographic and geological data (RSI-factors) through a cross-modal rectified fusion network (CRFNet). This network is designed with the feature rectification module (FRM) and feature fusion mixer module (FFMM) to integrate cross-modal features. To address potential missing data in complex geological environments, this work proposes a multi-task and prompt learning strategy to improve model robustness. Experiments conducted on our Asia Dataset and Africa Dataset yielded optimal evaluation results. On the complete Asia and Africa Dataset, the CRFNet model improved overall accuracy (OA) by more than 25% and the kappa coefficient by over 43% compared to rule methods. On the incomplete Asia and Africa Dataset, the CRFNet model improved OA by more than 5.5% and the kappa coefficient by over 8% compared to suboptimal deep learning (DL) models. To the best of our knowledge, this research work is the first in which DL features have been combined with multi-modal RSI-factors data for ORT assessment, paving a new path for research in this field. The source codes of this work will be made publicly available at https://github.com/kangkanghe/CRFNet.