Crime Detection from Pre-crime Video Analysis with Augmented Pose and Emotion Information

This study aims to detect pre-crime events in videos focusing on shoplifting. Our work proposes a novel approach of augmenting human pose information and emotion information to visual features with the aim of understanding pre-shoplifting events. We used a set of CCTV videos in stores in which at the end of some clips customers shoplift and in others they do not. We used these videos as training data with a transformer machine learning architecture. We augmented low level video analysis data with customer pose and emotion information. We experimented our model on visual features only, visual features augmented with pose information, and visual features with pose and emotion information of videos. We get improved accuracy at each step of experiments. The essence of our study’s contribution is hinged upon the utilization of this augmented pose and emotion information, enabling the capture of crucial behaviors and emotions in individuals, such as observing their surroundings, changing direction in aisles and exhibiting distress moments prior to engaging in shoplifting.