DiffVector: Boosting Diffusion Framework for Building Vector Extraction From Remote Sensing Images

Building vector maps play an essential role in many remote sensing (RS) applications, thereby boosting the deep learning (DL)-based automatic building vector extraction methods. These approaches have achieved pleasant overall accuracy, but their predict-style framework struggles with perceiving subtle details within a tiny area, such as corners and adjacent walls. In this study, we introduce a denoising diffusion framework called DiffVector to generate representations for direct building vector extraction from the RS images. First, we develop a hierarchical diffusion transformer (HiDiT) to conditionally generate robust representations for detecting nodes and extracting corresponding features. The conditions of HiDiT are multilevel boundary attentive maps encoded from input RS images through a topology-concentrated Swin Transformer (TCSwin). Subsequently, an edge-biased graph diffusion transformer (EGDiT) takes extracted node features as conditions to produce new visual descriptors for the adjacency matrix prediction. In EGDiT, we replace the standard self-attention (SA) operation with an edge-biased attention (EBA) to inject edge information for training stabilization. Furthermore, given typical challenges of training difficulty and weak perceptive ability in convectional diffusion paradigms, we conduct an isomorphic training strategy (ITS), ensuring that the training procedures of both HiDiT and EGDiT precisely mirror the inference phase. Quantitative and qualitative experiments have evidently demonstrated that DiffVector can achieve competitive performance compared with existing modern approaches, especially in the metrics assessing topology quality.