Automatic Mouth Detection for Self-Feeding

Automatic mouth detection can assist in controlling a robotic system with self-feeding of individuals with disability. To address this need we developed and evaluated algorithms that: 1) detect and track the mouth of an individual in real-time, and 2) classify if the mouth is open or closed. A k-nearest neighbors (KNN) clustering algorithm was used to classify and recognize the mouth's posture. The KNN algorithm classified image frames using features extracted by four methods including a histogram of oriented gradients, Harris-Stephens algorithm, maximally stable extremal regions, and local binary patterns. The results of this study indicated a high classification accuracy (~87%) using 10-fold cross validation for three participants without disability. The study has shown that the algorithms can detect the mouth postures of a person in near real-time (<;1s) while they have a robot-assisted meal in a social setting.