A Survey on Human Behavior Recognition Using Smartphone-Based Ultrasonic Signal

With the rapid progress of the Internet of Things (IoT) technology, human behavior recognition has become an important research topic in the field of ubiquitous computing and has obtained quite a number of research achievements. Accurate human behavior recognition can enhance the quality of human–computer interaction and facilitate the development of various sensing applications. With the popularity of smartphones and the improved performance of sensors such as speakers and microphones built in smartphones, the behavior recognition technique based on ultrasound signal of the smartphone has gained more attention and achieved several research results. In this paper, we first review the common behavior recognition techniques including light, video, sound, and frequency radio and outline their main characteristics. Then, we introduce the fundamental principle of human behavior recognition based on ultrasound signals. Specifically, these systems treat speakers and microphones embedded in smartphones as the transceiver and leverage the received signal changes caused by human movement including phase differences, frequency shift, and time of flight (ToF) to recognize human behavior. Next, we investigate the state-of-the-art studies and applications and analyze the signal processing techniques such as data collection, signal preprocessing, feature description, and behavior recognition approach. Afterward, according to the purpose of these applications, we classify them into five groups and compare them in detail including hand gesture recognition, activity recognition, hand trajectory tracking, vital sign monitoring, and lip reading. Finally, we conclude by discussing the limitations, challenges, and open issues involved in behavior recognition based on ultrasound signal of smartphone.