Analyzing Evolving Trends of Vulnerabilities in National Vulnerability Database

As the world approaches a state of greater dependence on technology, many products face increasing threats from malicious attackers who are attempting to take advantage of vulnerabilities in software design. Most of the known vulnerabilities are already aggregated, stored in text format, and are readily accessible to the public, making such an aggregated database a prime corpus for analysis using data mining methods. A multitude of research efforts have been deployed analyzing individual aspects of such cyber-security corpora to create taxonomies, assess vulnerability impact, and even predict future vulnerabilities. However, minimal effort has been committed to analyze cyber-security corpora to explore correlations between vulnerabilities and study the evolution of a vulnerability from its genesis. In this paper, we propose an integrated data mining framework to automatically lay out how vulnerabilities develop over time and detect the evolution of a specific cyber-security threat. We use (1) a Supervised Topical Evolution Model (STEM), which discovers temporal themes from a text corpus and (2) a diffusion-based storytelling technique that sifts through past vulnerability reports to describe how a current threat evolved. The STEM gives a holistic evolution structure of the vulnerabilities, while diffusion-based storytelling provides the precise genealogy of a specific threat. A considerable series of experiments demonstrate that the proposed framework can discover evolutionary patterns in today's most pressing vulnerabilities with a high degree of precision. As case studies, we explore the development of vulnerabilities in certain products, providing a unique insight into the correspondence between seemingly unrelated vulnerabilities and the impact of that correspondence on overall software security.