Real Time Hand Tracking Utilizing Neural Networks and Graphical Processing Units

As a state-of-the-art technology and useful method of input for controller free devices, real time hand tracking utilizing Graphical Processing Units (GPUs) is investigated in this research project. It was accomplished by applying traditional object detection and hand landmark detection, based on fully connected neural network methodology. Not many previous works in this field draw performance comparisons of these methodologies across a range of hardware platforms, which can be used to emulate controller free input devices. To more accurately demonstrate the specifications needed to properly support this form of input, this research project aims to deploy two approaches to real time hand tracking across two unique hardware environments. These two approaches are traditional object detection as well as a novel landmark method that trains a model on Google's MediaPipe hand landmark data. Experimental results demonstrated that the proposed novel landmark method could achieve a higher prediction accuracy on average across the American Sign Language (ASL) alphabet classes. In addition, utilizing hand landmarks as a dataset outperformed traditional object detection at runtime due to memory restrictions on lower performing platforms.