Real-Time Mouth Movement Recognition Using Media Pipe and CNN

The requirement for food monitoring systems has increased rapidly for analyzing eating behavior to mitigate overweight, obesity, and related disorders. In response, a novel approach of automatic food intake and mouth movement recognition system is designed to assess the frequency of food intake, meal duration, and mouth movements. Datasets are collected using OpenCV and Mediapipe Holistic for capturing facial landmarks and hand gestures. The model is trained using TensorFlow and Keras, incorporating the CNN architecture for feature extraction and image classification. The trained model is tested on 15 subjects of age group (20–40) in real-time, achieving an impressive accuracy rate of 96%. This helps in understanding and addressing the complexities of eating behavior and its impact on a person's health.