PublicVision: A Secure Smart Surveillance System for Crowd Behavior Recognition

Crowd behavior recognition plays a critical role in various domains, including public safety, event management, and urban planning. Understanding crowd dynamics and detecting behaviors based on violence levels are crucial for preventing incidents and maintaining order in crowded environments. However, traditional surveillance methods fall short of providing comprehensive and real-time insights into complex crowd behavior patterns and fail to distinguish different violence levels within crowds that affect proactive decision-making. Moreover, most of the current systems do not provide reliable secure data transmission and are not viable in protecting the privacy of individuals. This paper designs an end-to-end secure and smart surveillance system, namely PublicVision, that transmits CCTV data securely to a remote central hub where a deep learning (DL) model based on Swin Transformer is utilized to identify and analyze crowd behaviors. A novel video dataset was created to train the DL model that identifies crowds based on size and violence level. The proposed system incorporates end-to-end security by creating a Dynamic Multipoint Virtual Private Network (DMVPN) and leverages the property of IP Security (IPSec) and Firewall for confidentiality and integrity during transmission and storage. Experiment analysis and real-time inference using DeepStream Software Development Kit (SDK) proved that the proposed system has significant implications for public safety, security, and crowd management in various contexts, including public spaces, transportation hubs, and large-scale events.