Crowd Monitoring of Hajj Pilgrimage using Video Analytics and Deep Learning *

This paper presents developments in crowd analysis, with a specific emphasis on crowd density during the Hajj and Umrah pilgrimages. In recent years, there has been a rising interest in increasing video analytics and video surveillance to enhance the safety and security of pilgrims throughout their stay in Mecca. Crowd surveillance research and development are still in their infancy. Real-time monitoring of crowds is impeded by a lack of distinct crowd scenes or the high intensity of the crowd, which reduces its use. This paper proposed a fully convolutional neural network (FCNN) method for large crowd analysis, and it is able to classify the three classes of crowd density. This is intended to overcome existing technical challenges in video analysis in situations involving the circulation of a large number of pilgrims at a density of seven to eight per square meter. This endeavor intends to build a new dataset based on the Hajj scenario, in part to overcome the aforementioned difficulty. Manual and semi-automatic approaches are now used for video analysis, notably during Hajj. However, it is vital to automatically monitor pilgrim crowd events. Integrating FCNN classification with FCNN regression requires a strategy that is more powerful than the direct approach. Using ResNet50 and Fully Convolutional Neural Networks, a deep learning architecture is constructed to predict the density of certain crowd footage (FCNNs). The new dataset (Hajj-Crowd dataset) and using the recommended strategy, our solution obtained 100 % accuracy in crowd analysis, exceeding the state-of-the-art technology. In addition, the ResNet50 method achieved an accuracy of 98.6% for the developed crowd dataset.