Internet of Things Assisted Monitoring Using Ultrasound-Based Gesture Recognition Contactless System

In the last decade, we have seen an increase in the use of digital technologies in our daily lives, including advanced systems such as mobile applications, smart digital kiosks for intelligent retailing, touch screens for ordering food, or simple implementations such as the ticket machines in the butchers. However, from the perspective of interaction design, those sensing systems suffer from several constraints and limitations such as, low usability and poor hygiene. For example, the elderly or the disabled normally have huge difficulties when interacting with these types of digital systems. Moreover, the recent SARS-CoV-2 pandemic has made us rethink the way we interact with the digital devices. Hence, in this paper, we present a novel solution for digital interaction through a contactless model. This system can provide human gesture recognition and therefore it can be integrated into others to achieve contactless control. We have implemented a prototype based on cost efficient sensors to validate the idea's feasibility. Moreover, a sequence of real world experiments has also been conducted to evaluate its performance. This system is composed with 1) a 3d printed grid of ultrasound sensors to capture the distance information from a human body; 2) software to analyse the data for gesture predictions. This analysis results will be transformed into control commands to interact with the attached system. The experimental results have shown that the proposed system is capable of providing a contactless Human Computer Interaction (HCI), and also has a great opportunity to replace existing touching interaction manner with a remote control scheme.