Meta Combination Framework for 3D Point Cloud Analysis with Distillation

Recent advancements in 3D point cloud analysis have introduced various network architectures to the field. Nevertheless, the absence of a standard framework for describing these networks makes it difficult to systematically compare and analyze them. Furthermore, the network size of these models tends to be large due to the emergence of Large Language Models. We propose a method called MetaPointDistill that employs unified building blocks and model compression techniques to mitigate the aforementioned issues. To explain, when comparing systems, we need a unified framework structure for fair and convenient comparison. At the same time, the scale of this framework cannot be too large, so model compression technology needs to be used. Specifically, we partition the building blocks into Neighborhood Updater, Neighborhood Aggregator, Point Updater, and Spatial Embedding Generator, and explore their optimal combinations. Additionally, we employ knowledge distillation to obtain a model with improved performance and reduced size. We conduct experiments on multiple tasks to evaluate the efficacy of our framework. The results demonstrate that MetaPointDistill outperforms prior state-of-the-art methods by 1.2% mIoU and 0.4% OA while utilizing only 13% and 35% FLOPs on the S3DIS and ScanObjectNN datasets, respectively. Overall, our proposed framework exhibits significant potential for addressing challenges encountered in practical production processes.