Partition Placement and Resource Allocation for Multiple DNN-Based Applications in Heterogeneous IoT Environments

The evolution of the Internet of Things (IoT) has been driving the explosive growth of deep neural network (DNN)-based applications and processing demands. Hence, edge computing has emerged as a potential solution to meet these processing requirements. However, emerging IoT applications have increasingly demanded to run multiple DNNs to extract multifaceted knowledge, requiring more computational resources and increasing response time. Consequently, edge nodes cannot act as a complete substitute for the previous cloud paradigm, owing to their relatively limited resources. To address this problem, we propose to incorporate nearby IoT devices when allocating resources to multiple DNN models. Furthermore, the optimization of resource allocation can be hindered by the heterogeneity of IoT devices, which affects the delay performance of DNN-based computing. In this context, we propose a DNN partition placement and resource allocation strategy that considers different processing powers, memory, and battery levels for heterogeneous IoT devices. We evaluate the performance of the proposed strategy through extensive simulations. Simulation results reveal that the proposed strategy outperforms other existing solutions in terms of end-to-end delay, service probability, and energy consumption. The proposed solution was further simulated in a Kubernetes testbed consisting of actual devices to assess its feasibility.