Design and development of a new scanning core engine for malware detection

Malware is a man-made evil code, created for manipulative and destructive purpose. The increasing dependence on today's Internet and other communication network has caused a major malware threat to many computer users. The threat can infiltrate computers using a variety of methods, such as hidden functionality in regular programs, drive-by download from unsafe web sites, attack against known software vulnerabilities and more. In this paper, architecture of modern malware scanning engine is proposed and presented. A known packer detector and removal is proposed to build on top of the core engine. Prior begin malware scanning engine, the detection of known packer has to be performed. If any known packer is detected, a dedicated decryption routine will strip out the packer protection. Our malware detection core engine approach is based on the integration of static heuristic, emulator and disassembler. Static heuristic scanner detects malicious program via byte signature identification. It involves static extraction of an executable file and compares the destructive code with dedicated viral signatures. Emulator can execute the arbitrary code of an instance and trace the instance body's code inside the virtual environment. It can be used to combat any protection code, regardless of the complexity of the protection algorithm. Disassembler module will work simultaneously with emulator to analyze the execution code. Fragment of malicious code within the decrypted virus body could be detected via the execution. Through this study, we hope to help security researchers to understand our defense approach and give some directions for future research.