LogContrast: Log-based Anomaly Detection Using BERT and Contrastive Learning

Log data plays a vital role in the maintenance and fault diagnosis of computer systems. However, with the increase of system size and complexity, manual log analysis will take a lot of effort, which makes automated log anomaly detection an important research area. Early rule-based methods can effectively analyze software failures in logs, but require a lot of expert knowledge support. In recent years, with the development of deep learning (DL), automatic detection methods based on learning, such as DeepLog and LogBERT, have improved the efficiency of anomaly detection to a certain extent, but there are still problems such as semantic loss and data imbalance when dealing with massive and diversified log data. In this paper, a novel log anomaly detection model LogContrast is proposed, which is based on BERT pre-training model and realizes semantic analysis of logs through fine tuning. Meanwhile, semi-supervised contrast learning is used to improve the prediction accuracy of the model. Unlike the previous two efforts, LogContrast does not rely on log templates and does not need to map log information to template ids, but instead uses the original log semantic vectors to adapt to changing log content and application scenarios. We construct a semi-supervised learning log dataset by manually selecting a few logs from the data set according to the log level and marking them accurately. Semi-supervised contrast learning can learn richer and more robust feature representations from unlabeled data, while small amounts of labeled data can be used to tune and optimize the model. Compared with existing methods DeepLog and LogBERT, LogContrast achieves the best F1 scores on the HDFS (Hadoop Distributed File System),BGL (BlueGene/L) and ThunderBird datasets (85.39%, 93.67%and 97.73%, respectively).The experiments confirm the effectiveness of the LogContrast model.