PRRD: Pixel-Region Relation Distillation For Efficient Semantic Segmentation

Current state-of-the-art semantic segmentation methods usually require high computational resources for accurate segmentation. Knowledge distillation has been one promising way to achieve a good trade-off between accuracy and efficiency. However, current distillation methods focus on transferring the spatial relations and ignore the multi-scale context interaction. This paper proposes one novel pixel- region relation distillation (PPRD) to transfer the multi-scale pixel-region relation (PRR) from the teacher to the student. We get the multi-scale regions with pyramid pooling and characterize the multi-scale PRR between the feature and the multi-scale regions. Transferring such PRR from the teacher to the student is beneficial for the student to mimic the teacher better in terms of multi-scale context interaction. Experimental results on two challenging datasets, Cityscapes and Pascal VOC 2012, show that the proposed approach outperforms state-of-the-art distillation methods.