Masked PaCONet: Self-supervised Part-aware Implicit Shape Reconstruction

Localized neural implicit representation methods have recently been proven effective for shape reconstruction. However, while some recent neural implicit representation-based approaches have investigated part awareness, there is still room for improvement in leveraging the rich geometry information contained in parts, which is crucial for accurate reconstruction. This study aims to enhance the accuracy of shape reconstruction by incorporating part awareness. This principle faces a fundamental technical challenge: manually defining parts across various categories is ambiguous and expensive. To address it, we propose a new self-supervised learning paradigm that automatically discovers meaningful parts. Our proposed paradigm has several prominent advantages as compared with the prior arts: (1) It allows masked part modeling that scales well with available data; (2) It is a flexible formulation that allows a variable number of parts; (3) It allows the fusion of multi-scale (global-level and part-level) features at an arbitrarily given coordinate; (4) The semantic consistency of learned parts leads to transferable features. Extensive experiments validate our approach, named Masked PaCONet, showcasing its superiority in qualitative and quantitative results on public benchmarks, even under challenging settings. Codes and models will be released.