Enhancing Crowd Safety at Hajj: Real-Time Detection of Abnormal Behavior Using YOLOv9

This study introduces an innovative approach for detecting abnormal behavior in real-time during the Hajj pilgrimage, utilizing the advanced capabilities of the YOLOv9 algorithm. Faced with the challenges of monitoring dense crowds, existing methodologies often fall short in accuracy, efficiency, and cost-effectiveness. Leveraging deep learning, this research accurately identifies features of abnormal behavior from the HAJJv2 dataset, specifically curated and annotated for the Hajj context. Optimization of the YOLOv9 algorithm for this scenario demonstrated superior performance metrics (mean Average Precision (mAP@0.5), Recall, and Precision) when compared with its predecessors (YOLOv4, YOLOv5, YOLOv7, and YOLOv8), highlighting significant improvements in detection accuracy and real-time applicability. A pivotal aspect of this research was the implementation of critical data pre-processing steps to enhance the object detection model’s performance. These included a few-shot data sampling framework and comprehensive data augmentation techniques (flipping, rotation, scaling, and blurring) to increase the training dataset’s variability and representativeness. Such pre-processing measures are instrumental in addressing occlusions and variations in viewpoints, thereby ensuring the model’s robust generalization capabilities in real-world scenarios. The application of these methodologies not only improves safety and security measures during the Hajj but also offers a versatile framework that can be adapted for abnormal behavior detection in various crowded environments. This paper contributes to the field of crowd management and safety by providing a scalable, accurate, and efficient solution for the real-time detection of potential risks, ensuring a safer environment for one of the largest human assemblies worldwide.