Confidence Judgement Network: An Efficient Sample Filter for Lifelong Distillation

Unlabeled raw data of the real world collected by the edge computing platforms can be utilized to improve the performance of the lightweight deep learning models by using Lifelong Distillation (LD) method. However, not all the raw data samples are equally beneficial to such a process. In this paper, we propose a novel Confidence Judgement Network (CJN) to refine the raw data and to accelerate the LD processing. First, we design a new CJN model and use it to filter easy samples. Second, we scissor Lite-HRNet to get a slimmed student network—Half-LiteHRNet for knowledge distillation. Finally, we apply the new LD+CJN method to train our new image segmentation network, receiving a 39.4% reduction in network size and a 17.1% reduction in calculation load. Experiments show that, the new CJN model successfully improves LD by a roughly +3.96% increase in AP with a tiny cost, a negligible 0.038% rise in calculation load.