MALUP: A Malware Classification Framework using Convolutional Neural Network with Deep Unsupervised Pre-training

Malware is becoming the main threat during the development of the Internet. Driven by the increasing cost due to the complexity and diversity of malware, machine learning is more and more popular for related tasks. Among these ML methods, convolutional neural network has gradually become the mainstream for its excellent performance. At the same time, the scale of labeled data needed for training a ConvNet is so large that labeling malware samples has become a huge burden which hinders the practical application. Meanwhile, better performance means deeper network which requires more resources and more labeled samples for training. In this paper, we propose a novel and effective framework called MALUP for malware classification of guaranteed performance improvement which uses ConvNet with deep unsupervised pre-training. The framework is implemented with deepCluster as the pre-training method to deal with unlabeled malware images and the pre-trained ConvNet is then fine-tuned with labeled samples. We evaluate the effectiveness of MALUP by measuring the performance on a benchmark provided by Microsoft in different conditions. The experiments demonstrate the superior performance of MALUP compared to the ConvNet of the same architecture, especially for a shallower network. Additionally, we evaluate the impact of the number of clusters, the volume of unlabeled data, a simple balance restriction optimization during pre-training, and the percentage of labeled samples as well. Those variables form the optimization space of MALUP and help the proposed framework outperform the transfer learning model pre-trained with ImageNet in deepCluster.