RLGrid: Reinforcement Learning Controlled Grid Deformation for Coarse-to-Fine Point Cloud Completion

Many point cloud completion methods typically rely on two steps: coarse generation and 2D grid deformed fine output. However, in the fine generation, the expansion range (2D grid scale) required by each point cloud sample may be vastly different. For example, if the expansion range for a vessel shape is applied to a table shape, the final output may be blurry or sparse. To this end, we propose the RLGrid, Reinforcement Learning Controlled Grid Deformation. In detail, we firstly obtain two point cloud skeletons by two branches. One is to use an autoencoder, and the other is to convert the randomly generated normal distribution to coarse point cloud by GAN. We choose the one with smaller Chamfer Distance between coarse output and incomplete input as the input of the second stage. Then, a Reinforcement Learning (RL) agent is designed to select the appropriate expansion range based on the feature of each point cloud, and generate a 2D grid. Finally, all the features are concatenated and sent into a Multilayer Perceptron to obtain the detailed complete point cloud. Experimental results show that RLGrid achieves state-of-the-art performance on various datasets. To the best of our knowledge, RL is not widely used in point cloud completion task due to lack of custom environment, and the proposed RLGrid provides an insight on how to formulate 2D grid deformation as a sequential decision making problem. Further, it can also be plug-and-play on any 2D grid features.