Multi-View Contrastive Self-Supervised Triplet Network for Skeleton-Based Action Recognition

Improving feature discrimination power from different viewpoints has long been a hot topic of interest for skeleton-based action recognition. Most of the existing methods are based on fully supervised approaches for feature representation learning, which need to rely on a large number of labeled samples. However, it is easy to lead to overfitting and insufficient generalization of the model. In this work, we propose a novel Multi-View contrastive self-supervised Triplet network for skeleton-based action recognition, dubbed MVT. The original skeleton is encoded with features from different viewpoints, and mutual information maximization through a triplet network combined with triplet-view loss by contrastive learning. In the finetune stage of the downstream task, we froze the encoder combined with a classifier to output the final classification score. The extensive experiments show that our proposed MVT dramatically outperforms the state-of-the-art self-supervised learning methods for NTU-60, NTU-120 and PKU-MMD benchmarks.