Design and Implementation of Multimodal Fatigue Detection System Combining Eye and Yawn Information

Fatigue can harm people's physiology and psychology, and in serious cases it can even endanger people's lives. To meet the needs of real-time detection of human fatigue status, this paper designs and implements an online fatigue detection system which is capable of interactive fatigue status analysis. Based on AdaBoost framework and cascading technology, a classifier which is capable of identifying faces and eyes in video images at the same time is constructed, and the stable tracking of human eyes is realized by KLT algorithm. After extracting the features from the eye area, the eye opening and closing coefficient, or eye aspect ratio (EAR), is obtained. The k-means method is used to determine the thresh value of the eye opening and closing state to achieve the detection of blink frequency. Combined with nodding frequency and yawn frequency, the state of human fatigue is determined. The experimental results of different subjects in different scenarios show that the detection accuracy of this system can reach up to 95%.