CMDC-PCQA: No-Reference Point Cloud Quality Assessment via a Cross-Modal Deep-Coupling Framework

Point cloud quality assessment (PCQA) aims to provide an objective and unified evaluation of the quality of the point cloud. Existing research in this field lacks a comprehensive exploration of cross-modal perceptual feature fusion, overlooking the intrinsic correlations and complementarities across different modalities. To establish explicit relationships between modal features and capture the importance differences across modalities, we propose a novel method based on the cross-modal deep-coupling framework (CMDC-PCQA) for evaluating objective indicators of point clouds. First, we divide the point cloud into patches of equal size and propose a point cloud feature extraction (SGSA) module based on point space and graph structure, which employs self-attention mechanism for fusion learning. Second, we capture the projected images of point clouds from multiple viewpoints. To improve the perception of 2-D texture and semantic features, we embed attention to spatial and channel information in the local acceptance domain of convolutional neural networks. Third, we propose a method called dual-stage feature attention interaction module (DFAM). It is designed to update the local and global weights of the semantic and geometric features that fully characterize deep coupling. The proposed method is validated on the SJTU-PCQA, WPC, and WPC2.0 databases. Experimental results demonstrate that our method has surpassed the state-of-the-art PCQA methods, which validate its effectiveness in the PCQA task.