Visual Memory Transfer for Imbalanced Medical Image Classification

In deep learning of medical image data, skin lesion classification remains a challenging problem due to imbalanced distribution of training data. We propose a visual memory transfer (VMT) method by means of transferring visual knowledge from majority classes to minority classes. As a result, our method enriches feature of minority classes with pre-calculated memory features. In addition, VMT defines a refined feature map to perform fine-grained classification. Our classification results outperform SOTA methods on the largest public available dermoscopic image dataset on averaged F-score and Top-1 classification accuracy.