PointCU: Multiscale Sparse Convolutional Learning for Point Cloud Color Upsampling

Upsampling a sparse point cloud is a common operation in various applications, such as high-quality reconstruction, rendering. Point-cloud geometry upsampling has been extensively studied for this purpose. However, when a sparse point cloud includes attributes, such as color, their upsampling must also be carried out in addition to geometry. Despite the apparent need of such work, not a lot of effective solutions are developed that can work with real world large scale point cloud. In this work, we propose a novel solution called PointCU, which is a sparse convolution learning-based Point cloud Color Upsampling method that enables high-fidelity dense point color reconstruction from sparse point color. The proposed method first prepares multiple representations of a dense point cloud through voxelization at different scales, then transfers the color from sparse to the newly created point clouds including the dense point cloud itself through devoxelization. Then by learning feature on these multiple point cloud representation through sparse convolution neural network (SparseCNN) while also expanding and fusing the feature to higher scale, PointCU achieves an excellent color super-resolving capability. Our experimental results on four times (4x) and eight times (8x) upsampling tasks demonstrate that the color upsampling performance of the proposed method is superior to the previous known color upsampling schemes by a large margin.