Anti-Reverse-Engineering Tool of Executable Files on the Windows Platform

In response to the problems of the Windows executable being reverse-analyzed easily, we introduce the technology of INT3 breakpoint detection and characteristics detection in the tail of the heap in the software anti-dynamic-debugging. We also applied the multistage security policy which contains self-stored secret key decryption and the UKEY based decryption to the anti-static-analysis. When an attacker is trying to attack the program with the debugger, the detection thread will check the existence of the INT3 breakpoint in the head of the key API and the debugging-characteristics in the tail of the heap. If the analysis behavior is confirmed, the program will be terminated. At the same time, the key function will be hidden in exception handlers and the attacker will be unable to follow in the key function. According to the multilevel security policy, the program can be executed directly under low level policy, and it should be authenticated by the U-KEY to execute under the high level policy. After the experiment of effectiveness and feasibility test, we can conclude that this system is able to detect the execution environment and protect the software from reverse-attacking effectively.