OffloaDNN: Shaping DNNs for Scalable Offloading of Computer Vision Tasks at the Edge

Emerging mobile applications often require the execution of computer vision (CV) tasks based on compute-and memory-intensive deep neural networks (DNNs). Although offloading CV tasks to edge servers can decrease resource consumption at the mobile devices, it poses the challenge of handling multiple concurrent tasks with limited computing and memory capacity. In stark opposition with the existing state of the art, we tackle this challenge by jointly optimizing (i) the utilization of resources at the edge, among which memory - so far widely overlooked - and the radio resources used for task offloading; (ii) which and how many offloaded tasks should be executed; and (iii) the structure of the DNNs. First, we formulate the DNN for scalable Offloading of Tasks (DOT) problem, prove that it is NP-hard, and envision a weighted-tree-based heuristic solution, named OffloaDNN, that efficiently solves the DOT problem. We evaluate OffloaDNN through extensive numerical analysis using state-of-the-art image classification ResNet-18, as well as real-world experiments on the Colosseum emulator. The numerical results show that, in small-scale scenarios, OffloaDNN matches the optimum very closely, and, in larger-scale scenarios, increases the number of admitted offloaded tasks by 26.9 % with respect to the state of the art, while saving 82.5 % memory and 77.4% per-inference computing time. The numerical results are confirmed by the real-world validation on Colosseum.