Cyberattacks Using ChatGPT: Exploring Malicious Content Generation Through Prompt Engineering

The emergence of ChatGPT within the realm of computing has provided considerable advantages to a diverse array of individuals. However, it has also become a tool employed by adversaries to execute cyberattacks. This research paper examines the implementation of prompt engineering as a means to coerce ChatGPT into generating malicious content that deviates from its ethical boundaries. By leveraging these techniques, cybercriminals can effortlessly create a range of attacks, including phishing attempts, creating and propagating malware, backdoor attacks, and impersonation schemes, often in conjunction with deep fakes. To substantiate these cases, we successfully present concrete evidence by prompt engineering, which enabled the production of convincing phishing emails and code snippets for malware generation such as keyloggers. Additionally, we address the pressing concern of defending against these malicious activities, exploring effective approaches such as AI-generated text detection and system vulnerability detection.