Cross-view Self-supervised Learning via Momentum Statistics in Batch Normalization

A novel cross-view self-supervised learning (CVSSL) method via momentum statistics in batch normalization is presented in this paper. The problem of accuracy degradation in small-batch cases is currently common in self-supervised learning. Our method introduces the cross-view loss and the momentum statistics in batch normalization to solve the accuracy degradation problem in small-batch cases. Experimental results show that our method can drastically outperform the state-of-the-art self-supervised learning method in small-batch cases on the STL-10 dataset.