ReFrame: A Resource-Friendly Cloud-Assisted On-Device Deep Learning Framework for Vision Services

Cloud-assisted Internet of Things (IoT) device deployment of deep neural networks (DNNs) promotes On-device deep learning to provide users with ubiquitous high-quality services by solving the contradiction between insufficient IoT device resources and intensive demand for high-performance DNN resources. However, most existing methods optimize DNNs by considering one or two terms of transmission, computation, and storage resources, but do not consider all three terms at the same time in cloud-assisted IoT device deployment and updating DNNs. To this end, we propose a non-learnable module-based ResNet and a cloud-assisted on-device deep learning framework, ReFrame, based on the consideration of three indicators: model transmission parameters, computation resources, and storage resources. In the proposed method, we first specify that some parameters in DNNs are non-learnable and randomly initialized, so that, these parameters can be saved and reproduced with a few random seeds. By doing so, the cloud only transmits random seeds and learnable parameters to reduce the number of parameter transmissions. Secondly, we reduce the computation resource consumption of the model by introducing computation-friendly operators, such as pooling, to replace vanilla convolutions. Finally, since random seeds are used to save non-learnable model parameters, on IoT devices we only need to store random seeds and learnable parameters to reproduce the well-trained model. Compared with saving the complete model, our method greatly reduces IoT device storage resource consumption. Experimental results on image classification, object detection, and semantic segmentation tasks demonstrate the effectiveness of the proposed method. Specifically, on the CIFAR-10, our proposed method reduces approximately 89% of FLOPs and 90% of transmitted data in the prototype system compared to ResNet-18.