FreDDI: Frequency-Driven DNN Partitioning in Distributed Inference

This work is the first to formulate distributed inference in deep neural networks (DNNs) when considering frequency selection of an edge/hub/cloud device as an additional knob of control which can impact the global latency and/or energy. Specifically, each device in the network may have a set of discrete operating frequencies which we utilize to formulate the problem of layer-wise partitioning of a DNN in a distributed execution environment. We first develop a procedure for profiling energy and latency of a device, based on its operating frequencies, to execute a subset of DNN layers. We then propose an Integer Linear Programming (ILP) formulation for distributed inference which incorporates frequency-dependent energy and latency profiles. In our experiments, we demonstrate variations of ILP including energy-, and latency-constrained, when aiming to find the transition layer from the edge to a cloud device. We consider NVIDIA Jetson Nano and LePotato as edge device options to explore frequency selection. We show better energy and/or latency of our ILP, compared to a recent work (JointDNN).