Transformer-RF Ensemble: A Hybrid Deep Learning Approach for Malware Detection

Detecting malware has become challenging due to the continuous evolution of evasion strategies by malware authors. This study introduces a malware detection methodology that combines transformer-based language models with ensemble learning techniques. Specifically, DistilBERT is utilized to extract semantic features from executable opcodes, which are subsequently classified using ensemble models such as Random Forest, Support Vector Machine, and Logistic Regression. The proposed model achieved an F1-score of 0.939 and an AUC-ROC of 0.970, demonstrating high classification accuracy.