Ada-Thres: An Adaptive Thresholding Method To Mitigate The False Alarms

In a wide variety of domains, the advanced intrusion detection system consists of a learning-based detection method and a signature-based analysis approach. Such a system scans the incoming data, performs the analytics on it by using an anomaly detection algorithm, and finally transfers the report of suspicious activity for further analysis if found. The major problem of such a current system is the high false-positive rate (FPR), specifically in the case of a highly complex system with a large dataset. Such high FPRs, which are non-crucial, can easily overwhelm the user of the system and can further increase the likelihood of ignoring such indications. Therefore, mitigation approaches aim to develop a technique to reduce high FPR without losing any potential harmful threats. Thus, in this study, we develop an adaptive thresholding algorithm that can mitigate the issue of high FPR. The proposed algorithm applies three scoring mechanisms. They are Anomaly Pruning, Sequence Scoring, and Adaptive Thresholding. The model is trained on sequential data. Anomaly Pruning gives a score to an individual data point. It either rejects or accepts the data points to be considered for Sequence Scoring. This Sequence Scoring will give a score to an individual sequence. Finally, an Adaptive Thresholding is applied to the cumulative score of all the sequences to detect the anomalous nature of the analyzed data. Multiple experiments have been conducted using various optimizers to access our proposed approach. Using the proposed approach, we train a deep learning-based LSTM algorithm widely adopted for sequential data. Furthermore, we validate it with three different datasets of various sizes. From the experimental results, we infer that the proposed approach allows the model to train faster and reach the minimum loss at a faster rate.