Recognition of Abnormal Behavior from a Thermal Camera using Deep Learning

Recognizing dangerous, abnormal activities in videos is a complicated process. Nowadays, it is a hot topic, and neural networks are most often used for this problem. However, here we encounter the problem of the lack of available datasets. Publicly available data sets in most cases contain human activities such as sports, cooking, etc. However, very few of the available datasets contain videos with non-standard events. If such datasets exist, they are filmed during the day under white light. Therefore, there is a need to create a dataset shot at night. In this article, we deal with the creation of a dataset that focuses on non-standard abnormal activities recorded using a thermal camera. The dataset contains five categories, which include the most common non-standard abnormal events such as Drunkenness, Begging, Normal videos, Harassment, and Robbery. We use the dataset we created for our experiments. The dataset is used for training the neural network ConvLSTM (Convolutional Long Short-Term Memory). However, we also train the neural network on other datasets to compare the dataset quality. We use datasets such as AIRTlab and LoDVP Abnormal activities. The ConvLSTM neural network achieved 84% classification on thermal camera night videos compared to other datasets such as LoDVP-Abnormal activities dataset, the model achieved only 77% and on the Airtlab dataset 90%. These results support the creation of a high-quality dataset usable in the classification of abnormal, non-standard events.