Learning Imbalanced Datasets With Maximum Margin Loss

A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the deep model tends to predict the majority classes rather than the minority ones. For better generalization on the minority classes, the proposed Maximum Margin (MM) loss function is newly designed by minimizing a margin-based generalization bound through the shifting decision bound. As a prior study, the theoretically principled label-distribution-aware margin (LDAM) loss had been successfully applied with classical strategies such as re-weighting or re-sampling. However, the maximum margin loss function has not been investigated so far. In this study, we evaluate the two types of hard maximum margin-based decision boundary shift with training schedule on artificially imbalanced CIFAR-10 /100 and show the effectiveness.

1. Introduction

With the advancement of deep neural networks, large-scale datasets have appeared. In general, these real-world large data sets often have shown long-tailed label distributions [1,2,3,4] as depicted in Fig.1. On these datasets, the models [3] have a tendency to perform poorly on the minority classes due to the over-fitting. This tendency has to do with biased predictions, i.e., the trained deep model tends to predict the majority classes rather than the minor ones. Thus, overfitting to minority classes seems to be one of the challenges of generalization.