A Hybrid Multimodal Neural-Network-Based Anomaly Detection Model for UAVs

Uncrewed aerial vehicles (UAVs) are susceptible to malfunctions caused by environmental and human factors. Anomaly detection is a critical technique for ensuring the safe operation of UAVs. However, traditional methods that rely on unimodal sensor data and neural networks often fail to capture the comprehensive characteristics of faulty UAVs, limiting their effectiveness in complex environments. To address these limitations, this article proposes a UAV anomaly detection model that utilizes heterogeneous sensor data and multimodal neural networks [fusion multimodal deep neural networks (FMDNNs)]. First, the FMDNN uses an interpolation-based method for fusing multisource heterogeneous UAV sensor information. This method aligns the sensor data while simulating the distribution of UAV sensor data, enabling the FMDNN to utilize multisensor information to assess the overall state of the UAVs. Second, the FMDNN trains two neural networks of different modalities to extract nonlinear features and key structural features from the fused data by parallel computing using a multilayer perceptron and a convolutional neural network, respectively. Finally, the feature fusion module combines the nonlinear and key structural features to create a fusion vector. The fusion vector is processed by the fully connected layer and used to determine the UAV's state through a Softmax classifier. Experimental results demonstrate that on the ALFA, TLM, and UAVGPSAttack datasets, the FMDNN model achieves higher accuracy than the single-model classifier, by about 2% to 9%, for both the multiclassification task and the binary classification task. At the same time, the FMDNN model has strong robustness under classification tasks with noisy data.