On-Device Deep Learning for Mobile and Wearable Sensing Applications: A Review

Although running deep-learning (DL) algorithms is challenging due to resource constraints on mobile and wearable devices, they provide performance improvements compared to lightweight or shallow architectures. The widespread application areas for on-device DL include computer vision, image processing, natural language processing, and audio classification. However, mobile and wearable sensing applications are also gaining attention. They can benefit from on-device DL, given that these devices are integrated with various sensors and produce large amounts of data. This article reviews state-of-the-art studies on on-device DL for mobile and wearable devices, particularly from the sensor data analytics perspective. We first discuss the general optimization techniques of DL algorithms to meet the resource limitations of the devices. Then, we elaborate on model update and personalization techniques and review the studies by classifying them according to several aspects, including application areas, sensors, types of devices, utilized DL algorithms, mode of implementation, methods for optimizing DL algorithms for the target devices, training method, implementation toolkit/platform, performance metrics, and resource consumption analysis. Finally, we discuss the open issues and future research directions about on-device DL for mobile and wearable sensing applications.

I. Introduction

Wearable and mobile devices, including smartphones, smartwatches, and smart glasses, have become a part of our daily lives by assisting us for various purposes. In recent years, various machine-learning-based mobile applications have been developed. Particularly the use of deep-learning (DL) algorithms [1] or deep neural networks (DNNs) is emerging: image and face recognition using a camera, speech recognition using a microphone, and activity tracking using motion sensors are only a few examples.