Student Abnormal Behavior Detection Using Dlib Combined With YOLO Models

In the last years, educational technology has advanced tremendously. To better serve their students, schools and universities are moving online. AI-based proctoring solutions have also grabbed the industry, and online proctoring services are rising. Online proctoring systems (OPS) generally use online tools to ensure the examination is conducted with integrity. This study shows how to develop a multi-model system using computer vision to avoid and analyze anomalous student behavior during exams. The system incorporates mouth open/close detection, object identification, head posture assessment, and face detection utilizing Dlib facial landmarks and Yolo models. Our system also proposed a novel way of analyzing student behavior by analyzing the student’s computer screen using a deep neural network model learned from our developed dataset, "AppScreenshotDS." This work achieves good detection accuracy, where the "Tasks Detection" model achieves 0.87 mean Average Precision (mAP) on the developed dataset and the "Behavioral Analyses Model" achieves 0.95 accuracies. We conclude that our proposal provides a novel method for detecting abnormal student behavior by analyzing what is on the student’s computer screen using deep learning.