Generating Abuse Stories and Misuse Cases using Large Language Models

Security-by-design aims to build in security into software systems during the design stage in order to achieve a higher level of system resilience. However, traditional approaches to integrate security into software design requires non-security experts to manually brainstorm potential ways that attackers can target the software system. With the remarkable reasoning and generative capabilities of Large Language Models (LLMs), we believe that the initial effort to brainstorming attack strategies on a system can be performed by the LLMs to aid the non-security expert. In this paper, we provide two design artifacts — use case scenarios and user stories — that are produced at different stages of software development to an ensemble of five LLMs and instruct those LLMs to generate misuse case scenarios and abuse stories following a strict set of requirements. Our results show that the LLMs are able to produce a large number of abuse stories and misuse case scenarios but requires extensive postprocessing to extract meaningful results. The resultant abuse stories and misuse case scenarios represent a diverse set of attacks and complement the ability of developers to manually create those security scenarios. As such, LLMs are a valuable tool to jumpstart the process of security testing for non-security experts.