An Abnormal Event Classification Model for Big Data Platforms Based on Semi-supervised Learning

With the increasing diversity of network attacks, the security of big data platforms is receiving more and more attention. To solve the problem of detecting and classifying attack events on unlabeled, multi-source heterogeneous big data platform log data, this paper proposes a semi-supervised security event detection and classification identification model based on a time-series detection algorithm and UEBA. First, based on data analysis and processing and security event knowledge base construction, a time series detection algorithm is used to detect anomalies in some log data. Based on the anomaly identification results, a fine-grained analysis guide rule encoding module is conducted to initially label the anomaly results. Then, semi-supervised learning is performed on a small amount of labeled data by the Pu Learning algorithm to train an optimized detection model to achieve anomaly identification of unlabeled data. Finally, based on the classification results, the XGBoost algorithm is further used to train the recognition results for multi-classification to enhance the real-time detection and prediction capability of subsequent related attacks. The experimental results show that the proposed model can effectively identify anomalous intrusion detection sequences and obtain better classification results.