Face Masked and Unmasked Humans Detection and Tracking in Video Surveillance

Due to the spread of COVID-19, people wearing face masks became a regular occurrence worldwide. Moreover, there are nations where covering one's face is done for religious or cultural reasons, or even wear face masks for convenience. However, current face detection and tracking systems are hindered by face masks as the full facial features are no longer visible and therefore became less effective. In this paper, it is proposed to improve current face detection and long-term tracking technology by extracting the facial features of the top regions of the face, taking into account the eye, eyebrow, and forehead. The methodology contains two models, the face detector and the long-term object tracker. The face detection model uses a joint dataset from ISL-UFMD and MaskedFace-Net. The dataset is used to train a Keras sequential model. The object detection model uses pre-trained YOLOv4 weights and DeepSORT to identify people and uses the tracking-by-detection method to perform long-term tracking throughout the surveillance video. The final face detection model results show a testing accuracy of 93.33% and a loss of 26.92%, which are up to par and comparable with other state-of-the-art models.