Adaptive Data Compression for Classification Problems

Data subset selection is a crucial task in deploying machine learning algorithms under strict constraints regarding memory and computation resources. Despite extensive research in this area, a practical difficulty is the lack of rigorous strategies for identifying the optimal size of the reduced data to regulate trade-offs between accuracy and efficiency. Furthermore, existing methods are often built around specific machine learning models, and translating existing theoretical results into practice is challenging for practitioners. To address these problems, we propose two adaptive compression algorithms for classification problems by formulating data subset selection in the form of interactive teaching. The user interacts with the learning task at hand to adapt to the unique structure of the problem at hand, developing an iterative importance sampling scheme. We also propose to couple importance sampling and a diversity criterion to further control the evolution of the data summary over the rounds of interaction. We conduct extensive experiments on several data sets, including imbalanced and multiclass data, and various classification algorithms, such as ensemble learning and neural networks. Our results demonstrate the performance, efficiency, and ease of implementation of the underlying framework.