A Framework for Realtime Multiview Yoga Pose Detection and Corrective Feedback Using MoveNet and Convolutional Neural Network

Human pose detection and correction is a rapidly developing and important field in today’s world. After the advent of COVID-19, people have taken a keen interest in personal health and fitness. Many systems perform pose detection. But, a major issue is that they do not operate in real-time and have high latencies. Our system provides real-time pose detection and feedback, along with low latency. We achieved this by using a human pose estimation (HPE) model called MoveNet, which is ultrafast and more accurate when compared to other HPE models like Mediapipe, OpenPose, etc. In addition to this, our system uses a convolutional neural network (CNN) for multiclass classification to predict the correctness of the pose. Our system takes real-time video input of the user either from the front view or side view. keypoints detected by the MoveNet are given as input to a trained CNN model on a custom dataset which evaluates the correctness. For incorrect poses, real-time text and audio feedback are provided to the user. The CNN model is trained for three yoga poses multiview Tree Pose, Chair Pose, and Cobra Pose. For better correction, a system must support at least two views. Keeping this in mind, we have included multiview support. Our web application system can be used on both mobile and computer.