AUC Maximization for Flood Attack Detection on MQTT with Imbalanced Dataset

Several studies have conducted countermeasures against message queue telemetry transport (MQTT) attacks. However, existing research needs to examine further the imbalanced dataset that occurs in flood attacks, where these conditions can affect attack detection performance. Our research aim is to utilize the area under curve (AUC) maximization for flood attack detection on MQTT with imbalanced datasets. We leverage datasets from existing papers for flood attacks and legitimate network data. Then we utilize the AUC maximization methodology as follows: we perform parameter tuning of three machine learning methods: gradient boosting, naive Bayes, and k-nearest neighbor (KNN). Then we measure the performance of each model with the receiver operating curve (ROC) and AUC. We repeated these steps by performing parameter tuning and feature selection until the optimum AUC was found. Our test results show that the f1-score test can be used in the majority class but gives poor results in the minority class. At the same time, AUC shows that gradient boosting is the optimum detection method compared to naive Bayes and KNN, with AUCs of 0.96, 0.86, and 0.77, respectively. Finally, testing with feature selection shows that feature selection can increase AUC and decrease training and testing time by reducing features from 33 to two, revealing that the prominent features in a flood attack are tcp.len and tcp.time_delta.