A Robust Local Spectral Descriptor for Matching Non-Rigid Shapes With Incompatible Shape Structures

Constructing a robust and discriminative local descriptor for 3D shape is a key component of many computer vision applications. Although existing learning-based approaches can achieve good performance in some specific benchmarks, they usually fail to learn enough information from shapes with different shape types and structures (e.g., spatial resolution, connectivity, transformations, etc.) Focusing on this issue, in this paper, we present a more discriminative local descriptor for deformable 3D shapes with incompatible structures. Based on the spectral embedding using the Laplace-Beltrami framework on the surface, we first construct a novel local spectral feature which shows great resilience to change in mesh resolution, triangulation, transformation. Then the multi-scale local spectral features around each vertex are encoded into a `geometry image', called vertex spectral image, in a very compact way. Such vertex spectral images can be efficiently trained to learn local descriptors using a triplet neural network. Finally, for training and evaluation, we present a new benchmark dataset by extending the widely used FAUST dataset. We utilize a remeshing approach to generate modified shapes with different structures. We evaluate the proposed approach thoroughly and make an extensive comparison to demonstrate that our approach outperforms recent state-of-the-art methods on this benchmark.

1. Introduction

Obtaining detailed 3D shapes has become easy with the advancement of 3D scanning devices and computer vision reconstruction techniques. Accordingly, the importance of 3D shape analysis (e.g., shape matching, segmentation, correspondence, and retrieval) has increased remarkably. Designing local descriptors for surface points is a fundamental problem in computer vision, computer graphics and robotics, and it is a building block of various shape analysis and geometry processing applications.