Assessing Concept Drift in Malware: A Comprehensive Review and Analysis

The rapid evolution of malware has been posing substantial challenges for conventional detection systems due to attackers' continuous adaptations to evade detection. Concept drift, characterized by the changing statistical properties of data over time, is critical in understanding malware dynamics. A comprehensive dataset of Portable Executable (PE) malware samples spanning various families was analyzed to assess the overall drift in malware characteristics. Static features were utilized to delineate the structural and behavioral attributes of malware, ensuring early detection and resource efficiency. Robust clustering techniques were employed to facilitate the identification of evolving patterns and trends from 2017 to 2023. The analysis quantifies the extent of concept drift, highlighting the necessity for adaptive detection mechanisms to effectively combat emerging malware threats. The results indicate significant deviations among malware clusters over the years, highlighting significant concept drift. The findings underscore the dynamic nature of malware across different families and its impact on detection systems. The study emphasizes the necessity for detection frameworks that are capable of adapting to temporal variations and emerging threats.