Anomaly Detection in Unsupervised Surveillance Setting Using Ensemble of Multimodal Data with Adversarial Defense

Autonomous aerial surveillance using drone feed is an interesting and challenging research domain. Along with surveillance, an autonomous device should be capable of detecting device malfunction or abnormality in real time. However, the inherent uncertainty embedded within the type and level of abnormality makes supervised techniques unsuitable since the adversary may present a unique anomaly for intrusion. To counter this, in this paper, we propose an unsupervised ensemble anomaly detection system to detect device anomaly of an unmanned drone analyzing multimodal data like images and IMU (Inertial Measurement Unit) sensor data synergistically. We have proposed AngleNet and used autoencoder to analyze image and IMU data, respectively and later ensembled the two pipelines for predicting degree of abnormality of the device. Furthermore, we have applied adversarial attack to test the robustness of the proposed approach and integrated defense mechanism. The proposed method performs satisfactorily on the IEEE SP Cup-2020 dataset with an accuracy of 97.8%.