TinyDL: Edge Computing and Deep Learning Based Real-time Hand Gesture Recognition Using Wearable Sensor

Offloading data analysis to edge devices by decentralizing processing can be used to decrease bandwidth requirements, latency, and can decrease the total transmission time required in wireless devices. This can be especially useful for compact wearable devices used for health monitoring, human activity recognition, and gesture recognition, where sending raw data over wireless protocols such as Bluetooth can be both time and power consuming. By performing analysis on the wearable device, wireless radio usage can be greatly decreased, reducing a main power consumer on the device. Deep learning (DL) methods, specifically using Tensorflow (TF) and Keras were evaluated for their usage in such a case, in this example gesture recognition. A multilayer long short-term memory (LSTM) model was trained and evaluated off of data (10 gestures, 1000 trials total, balanced) from a finger-worn ring profile device that collected acceleration data, and was found to perform with accuracy from 75-95% per gesture. The attempted conversion of the model into a compressed TF Lite format, to allow for analysis on-device did not succeed, due to current incompatibilities between the different frameworks. Future work may improve the accuracy, and potentially expand the use of neural networks on wearables for health diagnostics or as input devices.