Joint DNN Partitioning and Resource Allocation for Multiple Machine Learning-Based Mobile Applications at the Network Edge

As a core technique of machine learning, deep neural networks (DNNs) have been extensively used in today's mobile applications. However, users' mobile devices (MDs) have limited capabilities to execute computation-intensive DNN inference operations and meet the latency constraints. Offloading part of DNN computations to edge servers (ESs) in a mobile edge computing (MEC) network can mitigate these challenges. However, prior studies on offloading either overlook the varying computation demands and output data sizes for different layers of neural networks or only consider split DNN offloading based on a single user or a single ES. Driven by the question of how to partition multiple parallel DNN inferences and offload multiple partitioned DNN processes in a multi-user multi-ES network, in this paper, we design a distributed scheme that jointly optimizes user-ES association, DNN layer-level partitioning, computing and wireless communication resource allocation, and offloading. Specifically, MDs and ESs make decisions to maximize their own utilities based on a multi-leader multi-follower Stackelberg game by leveraging DNN layer characteristics and taking account of multi- server heterogeneous network environments, computation load, and available resources. The evaluation results show that the proposed joint optimization scheme can significantly improve the performance of DNN inferences, compared to the commonly used benchmarks.