Pulse of the Crowd: Quantifying Crowd Energy through Audio and Video Analysis

Crowd energy can be considered a combination of several factors, including loudness, intensity, and visual cues such as hand waving and dancing. Crowd energy quantification, which refers to measuring the intensity and excitement of a crowd, is crucial and has many applications in various domains like sports, entertainment, and public event monitoring. However, current research in Crowd Analysis does not provide a way to quantify crowd energy in different videos and scenes. This paper introduces a novel approach combining audio and video processing to create a scoring mechanism for effectively quantifying crowd energy in an objective and data-driven way. We propose a novel pipeline that generates an absolute score for an input video using audio and video analysis, where higher scores indicate greater perceived crowd energy. We have created a custom dataset to address some of the shortcomings of current vision-based techniques in localizing excited crowds in a given image. Since there is currently no accepted standard for assessing crowd energy, we have used a subset of a standard crowd analysis research dataset to create a new framework for this purpose. This framework has been used to evaluate the quality of our results and the effectiveness of the proposed scoring mechanism in distinguishing videos with different levels of crowd energy. Additionally, we discuss potential applications of our crowd energy score in various contexts, including political rallies, event management, and other cultural and social gatherings.