ViSeR: Visual Self-Regularization

We propose using large set of unlabeled images as a source of regularization data for learning robust representation. Given a visual model trained in a supervised fashion, we augment our training samples by incorporating large number of unlabeled data and train a semi-supervised model. We demonstrate that our proposed learning approach leverages an abundance of unlabeled images and boosts the visual recognition performance which alleviates the need to rely on large labeled datasets for learning robust representation. In our approach, each labeled image propagates its label to its nearest unlabeled image instances. These retrieved unlabeled images serve as local perturbations of each labeled image to perform Visual Self-Regularization (ViSeR). Using the labeled instances and our regularizers we show that we significantly improve object categorization and localization on the MS COCO and Visual Genome datasets.