DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation

While most existing advanced large-scale point cloud semantic segmentation methods can accurately identify most large-scale objects, there is still room for improvement in the recognition accuracy of small-scale, low-proportion objects. Compared to point clouds, digital orthophoto maps (DOMs) has a more structured data format, allowing for better recognition of small-scale surface features. However, in existing projection-based methods, directly mapping images onto point clouds leads to occlusion issues. If image and point cloud features are simply concatenated, it results in feature blurring. Based on this observation, this article proposes a DAPSS network for point cloud semantic segmentation, assisted by prior knowledge constructed from DOM. The pretrained DOM features can provide a broader receptive field as guidance for learning the local context features of point clouds. Vertical occlusion has an issue, making ray-based mapping methods unsuitable. We propose a method that search for the nearest mapped point cloud in spherical space to fill in the occluded point cloud based on the already mapped point cloud. The traditional approach of directly concatenating point cloud features with image features often leads to feature blurring. Therefore, we propose a plug-and-play multimodal feature adaptive fusion module, which can adaptively select and aggregate features from different modalities to reduce redundant information further. In addition, we designed a cascaded multimodal feature deep fusion module to promote deep fusion between different modal features. Experiments on two large datasets demonstrate that DAPSS outperforms current mainstream methods, achieving mean Intersection-over-Union scores of 65.9% and 82.9% on the SansetUrban and SUM-Helsinki datasets, respectively. DAPSS not only effectively addresses the recognition of small-scale surface features, but also resolves the occlusion problems associated with projection-based methods.