Support vector machine integrated with game-theoretic approach and genetic algorithm for the detection and classification of malware

In the modern world, a rapid growth of malicious software production has become one of the most significant threats to the network security. Unfortunately, widespread signature-based anti-malware strategies can not help to detect malware unseen previously nor deal with code obfuscation techniques employed by malware designers. In our study, the problem of malware detection and classification is solved by applying a data-mining-based approach that relies on supervised machine-learning. Executable files are presented in the form of byte and opcode sequences and n-gram models are employed to extract essential features from these sequences. Feature vectors obtained are classified with the help of support vector classifiers integrated with a genetic algorithm used to select the most essential features, and a game-theory approach is applied to combine the classifiers together. The proposed algorithm, ZSGSVM, is tested by using a set of byte and opcode sequences obtained from a set containing executable files of benign software and malware. As a result, almost all malicious files are detected while the number of false alarms remains very low.