Generative Dataset Distillation Based on Large Model Pool

Distilling knowledge into generative models is a technique to extract knowledge from a large dataset and embed it into a generative model, which effectively reduces redeployment costs and boosts dataset distillation efficiency. In this paper, we propose a novel approach to enhance the performance of generative models by expanding the size of the model pool. Our approach strives to improve the diversity of models in the model pool so that more distinctive information can be considered when matching the prediction logits. We also verified the proposed method is effective on the CIFAR-10 dataset.