Optical Sensor Based Gestures Inference Using Recurrent Neural Network in Mobile Conditions

In this paper the implementation of recurrent neural network models for hand gesture recognition on edge devices was performed. The models were trained with 27 hand gestures recorded with the use of a linear optical sensor consisting of 8 photodiodes and 4 LEDs. Different models, trained off-line, were tested in terms of inference time on several mobile devices. The impact of different network topologies (different number of neurons and layers) and different effective sampling frequency of a recorded gesture on the inference time were evaluated. Inference time tests performed on the most effective of available platforms, Samsung Galaxy S8 smartphone, present that recurrent neural networks trained on unprocessed (raw) data from the gesture sensor (1-layer model) execute faster than models trained on differently processed features (2 and 3-layer models): 374ms vs 765ms and 1048ms for data recorded with 100Hz. Also, the reduction of effective sampling frequency allows to shorten the inference time, e.g., for raw data from 374ms at 100Hz to 115ms at 25Hz. Presented results confirm the usability of low complexity interfaces like linear gesture sensor in mobile devices with limited computational capabilities, which allows to consider such sensors in wearable electronic devices e.g. smart glasses.