Automated Online Proctoring System Using Gaze View Tracking and Custom Object Detection

Due to COVID-19, there’s been a burst in online examinations. The main integrity safeguard so far is human proctoring, which requires trained supervisors to constantly monitor all test-takers’ videos and audios through webcams. To overcome such costliness and ineffectiveness, we have designed an automated online proctoring system that is effective, nonintrusive, and adaptable to different testing scenarios. Our approach presents a novel combination of (a) a gaze view tracking module using a mathematical 3D gaze motion formula and (b) a configurable cheating classification using a custom-trained object detection model. Our gaze view tracking leverages two cameras (a webcam and a follower-cam) working in tandem, and facial landmark detection to follow the test-taker’s gaze in a real-time and non-intrusive manner. It then feeds to our AI-based cheating classifier, which leverages TensorFlow object detection algorithm with a custom-trained object detection model to identify preconfigured cheating targets. Our end-to-end prototype and trials show effectiveness in tracking the test-taker’s gaze and autodetecting cheating targets. Our system can serve as a great complement to the current online proctor suite and will influence online learning even after pandemics by reducing human toil.