ASAD: A Meta Learning-Based Auto-Selective Approach and Tool for Anomaly Detection

Anomaly detection, crucial for identifying issues such as financial fraud or medical malfunctions, has advanced significantly with machine learning (ML) and deep learning (DL). However, a major problem in the field is that no single model works best with diverse datasets and problem domains. To address this, we propose an innovative auto-selective approach and software tool based on meta-learning, called ASAD (Auto-Selective Anomaly Detection), to dynamically select the most appropriate model based on the unique features of a given dataset or problem domain. ASAD trains an ML model to predict the best candidate from a large pool of models by considering the specific characteristics and requirements of the dataset. It is trained using 139 datasets built upon 60 base datasets from 11 diverse domains (finance, healthcare, network security) and 80 ML and DL models composed of 22 base anomaly detection algorithms. It uses meta-features and correlation functions to evaluate 300 features. It selects the best-performing model, evaluates models based on 7 metrics, and delivers significantly better performance than the cutting-edge in this area. ASAD addresses the critical challenge of model generalization and adaptability, aiming to enhance the efficiency and accuracy of anomaly detection across varied application domains. By automating the selection process, the method aims to reduce the reliance on trial-and-error methods, streamline the anomaly detection workflow, and lead to more robust, adaptable, and efficient anomaly detection systems. We provide the methodology, implementation, and evaluation of our approach, offering insights into its potential to revolutionize anomaly detection in this era characterized by vast and complex datasets.