Dataset Distillation with Channel Efficient Process

The success of deep learning is primarily attributed to the vast amount of data used for training, which comes with massive computation costs and storage. Dataset distillation(DD) aims to reduce the dependency on such massive data by learning a small synthetic dataset that preserves most information from the original dataset. Recent work has proposed a new condensation framework that generates multiple synthetic data with a limited storage budget. However, they only focus on the synthetic data’s spatial regularity and ignore the compressible space on the channel. In this paper, we propose a novel channel-efficient process that augments the number of condensed data and trains the synthetic data in a channel information-intensive mode. We design the process as a plugand-play strategy that is portable to any existing DD baseline, and our experiment results demonstrate that it can yield significant improvement on downstream classification tasks compared with previous DD methods.