Detecting human abnormal behaviour through a video generated model

Detecting human abnormal activities is the process of observing rare events that deviate from normality. In this study, an automated camera-based system that is able to detect irregular human behaviour is proposed. PoseNet and OpenPose, which are pre-trained pose estimation models are used to detect the person in the frame and extract the body keypoints. Such data is used to train two types of AutoEncoders based on LSTM and CNN units in a semi-supervised approach where the goal is to learn a general representation of the normal behaviour. Evaluated on a challenging realistic video dataset, the results show that both types of models were able to correctly distinguish between normal and abnormal data sequences, with an average F-score of 0.93. The results also show that the proposed method outperformed similar work done on the same dataset. Furthermore, it was also determined that pose estimated data compares very well with sensor data. This shows that pose estimated data can be informative enough to understand and classify human actions.