Detection of Android Malware Based on Deep Forest and Feature Enhancement

Detecting Android malware in its spread or download stage is a challenging work, which can realize early detection of malware before it reaches user side. In this paper, we propose a two-stage detection framework based on feature enhancement and cascade deep forest. This method can detect the traffic generated in the encrypted transmission process of Android malware. The first stage realizes the binary classification of benign and malicious software. The second stage realizes the multi-classification of different categories of malware. To enhance data representation, convolutional neural networks is used to extract benign and malicious features in the first stage, and the principal component analysis method is used to extract the malicious features in the second stage. Theses extracted features are spliced with the payload part of the traffic to form fusion features for classification task. In order to adapt to different scale of samples, especially for the small-scale sample, cascaded deep forest method is proposed to construct the classification model. In this model, many layers that consist of base classifiers are cascaded and the number of layers can be automatically adjusted according to the scale of the samples. With different combinations of base classifiers in each layer, the optima detection accuracy is archived in the two stages. The experimental results on several datasets prove that the proposed method is effective for encrypted transmission detection of Android malware. It is also suitable for the detection of unknown attacks.