Comparative learning signals lead to aligned representations in an infant-inspired visual task

Infants develop visual abilities through different kinds of visual experience: for instance, they learn from playing with objects when at home and also when they see objects outdoors. Inspired by this dichotomy of experiences, we use machine learning (ML) to investigate how representations that bridge these two kinds of experiences can be learned. In ML parlance, the differences in the characteristics of these two experiences constitutes a distribution shift. ML research suggests that aligning learned representations across the two distributions can be beneficial for bridging the distribution shift. We use the supervised learning paradigm in this work; we assume we have labeled images from both distributions. In this paradigm, the most popular approach is to train a model to map different examples of a category to the same label, i.e., using a category-based learning signal. In contrast, we propose that comparison of image features across the two distributions can promote alignment, i.e., what we call a comparative learning signal. We propose 2 metrics to measure the cross-distribution alignment of image features. Using these metrics, we show that standard supervised learning using only categorical learning signals does not lead to aligned representations when training a popular neural network architecture (ResNet-18). Additionally, we show that using our comparative learning signal on only a small number of cross-distribution image pairs promotes alignment without affecting accuracy. Finally, we show that the diversity of images that form the image pairs contributes to the effectiveness of comparative learning signals.