Localized Knowledge Distillation Helps IoT Devices Provide High-performance Visual Services

Deploying high-performance convolutional neural networks (CNNs) on ubiquitous Internet of Things (IoT) devices to provide convenient services has attracted increasing attention. However, most existing studies have two limitations, (i) low performance due to lack of effective learning strategies; (ii) difficulty to measure resource consumption due to lack of real deployment. To this end, this paper proposes a novel localized knowledge distillation (LKD) method to train the resource-efficient CNN and implements a cloud-assisted system to evaluate the on-device performance. The proposed LKD follows the layer-wise heterogeneous information distribution in the CNN and distills the knowledge from features that contains the most crucial knowledge to the resource-efficient CNN. Thus, the knowledge guided to learn by limited parameters is reduced to the crucial part, which is more reliable for the resource-efficient CNN. The cloud-assisted system consists cloud server and IoT devices, which allows the training and deployment of the resource-efficient CNN, and the measurement of the corresponding resource consumption. Experimental results show that the proposed LKD could improve the performance on standard benchmarks close to the counterpart CNN, and the robustness on corrupted data by approximately 11.3% for the resource-efficient CNN. The measurements on the cloud-assisted system also demonstrate the resource efficiency for transmission and on-device running.