A Novel Dataset and a Hybrid Ensemble Approach for Anomaly Detection in Enterprise-Access-Logs for Anomaly Detection

This work proposes a new access-log based dataset which combines real-world enterprise access logs with infused synthetic anomalies to establish a well-balanced distribution of normal and malicious activities. The dataset can be used to effectively train ML algorithms for classification of identity-based threats such as unauthorized access and privilege escalations. The dataset contains a very comprehensive and diverse set of features, which make the dataset feasible for Machine learning based implementations. The dataset has been made publicly available through Github, and will serve as a reference standard for access-logs based threat detection research in future. Further, this study also introduces an ensemble anomaly detection framework consisting of Quantum Clustering (QIC), Autoencoder, Graph Learning, Hidden Markov Model (HMM), Hyperdimensional Computing and Recurrence Quantification. Through ensemble-based approach combining the mentioned algorithms in a structured flow, the detection accuracy is found to be significantly higher when compared to traditional individual ML algorithms. The ensemble model thus built surpasses conventional ML and DL algorithms by reaching almost 90% accuracy, 94% precision, 92% recall and 93% F1-score. To sum up, this work contributes to the current pool of knowledge in this domain by adding a novel dataset, and proposing an effective ensemble approach. This work finds its applications in critical infrastructure protection and enterprise data security.