Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach

Edge devices, including, in particular, mobile devices, have been emerging as an increasingly more important platform for deep neural network (DNN) inference. Typically, multiple lightweight DNN models generated using different architectures and/or compression schemes can fit into a device, thus selecting an optimal one is crucial in order to maximize the users’ Quality of Experience (QoE) for edge inference. The existing approaches to device-aware DNN optimization are usually time consuming and not scalable in view of extremely diverse edge devices. More importantly, they focus on optimizing standard performance metrics (e.g., accuracy and latency), which may not translate into improvement of the users’ actual subjective QoE. In this article, we propose a novel automated and user-centric DNN selection engine, called
$\mathsf {Aquaman}$
, which keeps users into a closed loop and leverages their QoE feedback to guide DNN selection decisions. The core of
$\mathsf {Aquaman}$
is a neural network-based QoE predictor, which is continuously updated online. Additionally, we use neural bandit learning to balance exploitation and exploration, with a provably efficient QoE performance. Finally, we evaluate
$\mathsf {Aquaman}$
on a 15-user experimental study as well as synthetic simulations, demonstrating the effectiveness of
$\mathsf {Aquaman}$
.