Rethinking Deep CNN Training: A Novel Approach for Quality-Aware Dataset Optimization

An informativeness of the data has always been of great interest within a machine learning community. Nowadays, with a skyrocketing advancement of the artificial intelligence and massive volumes of noisy data, it becomes even more essential to develop a robust and effective methods for training data optimization. Existing approaches are mostly based on empirical trials and errors, with either stochastic or deterministic data reduction strategies. The key limitation of such solutions is that they do not consider the overall informativeness of the resulting training dataset. In this paper, we propose a novel approach for a quality aware dataset optimization by initial assessment of its informativeness. As a metric of informativeness, we use entropy values calculated over the target dataset. To alleviate the computational complexity, we perform initial clustering of the dataset into many subsets and calculate the entropy of each cluster independently. The dataset is then optimized by dynamic programming to find a sequence of subsets with low overall entropy according to imposed size limitations. The experimental evaluation shows the that proposed approach shows improvement over current best alternatives in terms of accuracy, precision, recall and F1-score metrics. Moreover, the proposed approach provides an excellent interclass discrimination even for a large number of classes.