Fatigue detection for public transport drivers under the normalization of epidemic prevention

Several studies have shown that fatigue driving is one of the important causes of public transport safety accidents. With the outbreak of the COVID-19, the wearing of masks by public transport drivers presents new challenges for computer-based visual fatigue detection. In order to achieve the goal of accurately capturing the landmark information of the face even when the face is occluded by a large area, we adopt the DNN-based face detection method which has the highest accuracy and the best occlusion resistance. When the driver's face is blocked, the landmark information of the blocked face can be accurately detected by using our optimized face landmark detector. The accuracy rate of landmark recognition can reach 97.80%. On this basis, we calculate the driver's eye information, mouth information and the driver's head deflection angle information in real time as the judgment indicators of the degree of fatigue to comprehensively evaluate the driver's fatigue state. And use mathematical methods to fuse indicators in real time, classify the driver's fatigue state according to the value of the fusion indicators, and adopt different early warning methods for different levels of fatigue. In addition, in order to further improve the accuracy of the detection results and exclude the influence of other facial behaviors on our fatigue judgment indicators, we propose a kinetic energy calculation formula for facial organs based on the improved optical flow method. According to the different kinetic energy of facial organs in different states, which can accurately distinguish the different behaviors of the same facial organs such as blinking and closing eyes, yawning and speaking, which significantly increases the robustness and generalization ability of the detection program. The final experimental results show that the correct rate of the method for determining the degree of fatigue of the driver and passengers can reach 98.40% and 92.30% respectively when the driver does not wear a mask or wears a mask.