Towards Pointsets Representation Learning via Self-Supervised Learning and Set Augmentation

Deep metric learning is a supervised learning paradigm to construct a meaningful vector space to represent complex objects. A successful application of deep metric learning to pointsets means that we can avoid expensive retrieval operations on objects such as documents and can significantly facilitate many machine learning and data mining tasks involving pointsets. We propose a self-supervised deep metric learning solution for pointsets. The novelty of our proposed solution lies in a self-supervision mechanism that makes use of a distribution distance for set ranking called the Earth’s Mover Distance (EMD) to generate pseudo labels and a pointset augmentation method for supporting the learning solution. Our experimental studies on documents, graphs, and point clouds datasets show that our proposed solutions outperform baselines and state-of-the-art approaches under the unsupervised settings. The learned self-supervised representation can also be used as a pre-trained model, which can boost downstream tasks with a fine-tuning step and outperform state-of-the-art language models.

1 Introduction

Distance measuring has been used to find the similarity between the query data point and other samples in the dataset. The calculated distance can be applied in various machine learning algorithms to make predictions. However, pointsets, which have been widely used for representing documents [1], graphs (as a set of vertex embeddings) [2], and 3D models (point clouds) [3], require a complex distance function to compute a distance between each sample. In particular, one of the effective distribution distances for pointsets called Earth Mover’s Distance (EMD) [4] requires $O(n^3\log n)$ per one comparison. Deep metric learning can alleviate this high computation cost problem by encoding raw pointsets into new representations in a euclidean space with a lower distance measuring cost. Several studies proved that deep metric learning approaches achieve state-of-the-art results in various tasks, including image retrieval [5], [6], face recognition [7], [8], [9], person re-identification [10], [11], trajectory similarity search [12], and pointsets similarity search [13] in terms of classification accuracy and computation cost. However, these techniques mostly require supervision, which cannot be applied to unlabeled data.