Knowledge Distillation for Object Detection Based on Mutual Information

While pursuing high performance, most currently proposed object detection algorithms are composed of huge and complex neural network and large amount of parameters. As a results, many of them can not be implemented on edge devices due to the limitation of computation resources. To address this, in this paper, we designed a model compression scheme for object detection algorithm based on knowledge distillation. Firstly, a novel loss function based on mutual information have been proposed for similarity measurement between the teacher network and the student network. Then, a new knowledge distillation scheme is proposed for object detection network. The proposed algorithm has been verified on PASACL VOC dataset. The experimental results have demonstrated the effectiveness of our method.