Investigation of The Latest Malware Detection Engines and Lightweight Byte n-Gram Methods with Real Custom Malware

Almost all malware detection systems are black boxes to the general users. The simple pattern matching method is the fastest, but it becomes useless if some bytes in a malware file are changed. Therefore, in this paper, we investigated what kind of customization influences the detection rate of the latest malware detection engines. In addition, we also investigated whether byte n-gram and information gain malware detection methods are effective in detecting the custom malware files. These methods have been proposed and improved by some researchers, but the computational cost of information gain was too high to be used in the real world. Therefore, we proposed a lightweight method of the methods and evaluated the custom files. With this method, we could find only 9 to 13 out of 32 to 33 malware files in 5-hold cross-validation, but it was 15 to 21 in original files. This means that the byte n-gram method can work if it detects a malware file once. On the other hand, almost all of the malware files were not detected by almost all of the latest engines. We conclude that the byte n-gram methods are still effective when malware changes itself, and lightweight methods with better detection rates are needed.