A Guided Deep Learning Approach for Joint Road Extraction and Intersection Detection From RS Images and Taxi Trajectories

Automatic extraction of road information based on data-driven methods is significant for various practical applications. Remote sensing (RS) images and GPS trajectories are two available data sources that can describe roads from a complementary perspective, and fusing them together can improve road detection performance. However, existing studies on the combination of RS images and GPS trajectories do not fully utilize their enhanced information about roads and suffer from road information loss. Moreover, roads and intersections are two crucial elements of road network generation, and they are closely related to each other. Therefore, we propose a multitask and multisource adaptive fusion (MTMSAF) network, which leverages RS images and trajectory datasets to execute road extraction tasks and intersection detection tasks simultaneously. The MTMSAF network is built on three components. First, two encoder steams are designed to capture road features from RS images and trajectories. Then, an adaptive fusion module is created to fuse the individual road features at each level in a guiding fashion. Finally, two specific decoders are proposed: one is used to implement the road region detection task by considering multidirectional geometric road information, and the other is used to perform the intersection extraction task by recovering intersection information with the road region feature. Four state-of-the-art deep learning-based methods, including segmentation networks and road extraction networks, are compared with the proposed approach. The results show the superiority of our approach for both road detection and intersection extraction tasks.