MalBET: A Multiclass Malware Detection Method Using Improved Bidirectional Encoders from Transformers

Malware poses a significant threat to cyberspace security, making the classification of malware families crucial for safeguarding computer devices and information systems. Analyzing the API call behavior characteristics of malware is a key method for identifying and detecting malware. However, traditional machine learning and deep learning methods struggle to effectively capture the complex interaction patterns between API calls. In this paper, we propose a multiclass malware detection method named MaIBET, which leverages a multi-layer bidirectional Transformer encoder network to capture the intricate dependencies between API calls and identify malware families. To ensure efficient classification and acceptable accuracy, we have implemented specific improvements to the encoder network. Evaluation results show that MalBET achieves an overall performance with an average accuracy of 95.55% across four different datasets. Additionally, we conducted three sets of experiments to validate the contributions of the implemented improvements.