Deep Learning Video Analytics on Edge Computing Devices

The rapid progress of deep learning-based techniques such as Convolutional Neural Network (CNN) has enabled many emerging applications related to video analytics and running them on mobile devices can help improve our daily lives in many ways. However, there are many challenges for video analytics on mobile devices using multiple CNN models. CNN models are resource hungry, and each model requires a large amount of computational power and occupies a large portion of memory space. Although video processing can be offloaded to reduce the computation time, transmitting large amount of video data is time consuming. Thus, offloading is not always the best option. Moreover, different CNN models have different memory usage and processing time, making the scheduling problem more complex. As a result, besides deciding which task to be offloaded, we must decide which CNN model should reside in the memory and for how long, and which CNN model should be switched out due to memory constraint. In this paper, we propose resource aware scheduling algorithms to address these challenges. We identify the task scheduling problem for running multiple CNN models on mobile devices under resource constraints and formulate it as an integer programming problem. We propose resource-aware scheduling algorithms which combine offloading and local processing methods to minimize the completion time of video processing. We implement the proposed scheduling algorithms on Android-based smartphones and demonstrate its effectiveness through extensive experiments.