Feature-Based Optimization for Log Anomaly Detection Methods

The increasing complexity of distributed software system architectures often leads to various anomaly issues. Since log data can reflect the runtime status of programs, anomaly detection based on log data holds significant research importance for promptly identifying anomalies in software systems. However, current research faces challenges such as log noise interference, imbalanced data distribution, and inefficient inference of deep learning models. To address these issues, this paper proposes a feature-optimized log anomaly detection processing framework. This framework conducts mining and analysis on log feature data, optimizes through methods such as denoising, feature matching, and feature fusion. We have verified that our feature optimization method significantly improves the performance of various anomaly detection models across three public datasets, achieving dataset size reduciton ranging from 65.3% to 99.7%. Additionally, the study indicates that lightweight machine learning models can deliver comparable performance with higher efficiency in anomaly detection tasks, operating 10 to 187 times faster.