CECL: Context-Embedded Contrastive Learning for Enhanced Recognition of Abnormal Behavior

The widespread deployment of Internet of Things (IoT) devices in intelligent school environments provides abundant data for enhancing video recognition systems with advanced behavior recognition technologies, which is crucial for ensuring campus safety. Traditional models primarily rely on visual cues and often struggle to identify complex behaviors against dynamic backgrounds. This study addresses these challenges by introducing a novel Context-Embedded Contrastive Learning (CECL) approach, which leverages data collected from IoT devices to synergistically integrate textual and visual modalities through contrastive learning, thereby refining behavior recognition capabilities. In addition, the model incorporates a Context-Embedded Transformer Encoder that utilizes temporal and geographic context cues captured by IoT sensors, processed through the proposed Context-Aware Multi-Head Attention mechanism. This enhancement significantly improves the model’s ability to discern subtle behavioral nuances by harnessing the rich contextual data from IoT-based surveillance systems, which conventional methods frequently overlook. Through this integration, the model achieves a more sophisticated understanding and recognition of abnormal behaviors in campus settings, as evidenced by the CABRH8 dataset. The CECL model has demonstrated outstanding performance, achieving a Top-1 accuracy of 80.26% and a Top-5 accuracy of 99.38% on the challenging CABRH8 dataset, thereby outperforming existing models reliant solely on visual data. Its robustness and adaptability have also been validated across additional datasets, including CABR50, UCF-101, and HMDB-51, demonstrating its potential for widespread deployment in various IoT-enabled surveillance applications.