Measurement of Anti-Debugging Techniques on the Windows and Linux Operating Systems for the Intel x86_64 Architecture

Malware authors and software protection frameworks often use anti-debugging techniques to hinder understanding of the underlying code. Companies use anti-debugging techniques to prevent intellectual property, including music, movies, and games, from being stolen or abused by malicious actors. In addition, the encryption of program data or private data can also be classified as an anti-debugging technique because it requires additional effort to reverse engineer or debug it. Malware authors also use these techniques to prevent their software from being analyzed, detected, or blocked. While anti-debugging techniques are not novel, there is minimal research on their performance. This paper conducted novel research on commonly used anti-debugging techniques by measuring their performance on both the Windows and Linux operating system while also creating performance data about the most common anti-debugging techniques, including implementations and statistical measures for six categories of anti-debugging techniques. The statistical results describe a low overhead for the API tests, timing tests, and debug register tests. As expected, the memory encryption test had the highest overhead. Lastly, the paper discusses these results and their statistics per technique and includes code samples for future research.