Securing the Weakest Link: Exploring Affective States Exploited in Phishing Emails With Large Language Models

Cyberattacks often begin with a phishing email designed to exploit human factors, widely regarded as the weakest link in cybersecurity defences. Although previous research has identified factors that make individuals susceptible to scams and fraud, it is still unclear how phishing emails exploit these factors to trigger victims’ emotional responses. This paper addresses the following major questions: Which affective states are commonly exploited in phishing emails? Can they be reliably detected by Large Language Models (LLMs), and how do these models compare to human annotators? In this paper, we identify phishing-specific susceptibility factors inspired by interdisciplinary research literature from areas such as security, human-computer interaction (HCI), and psychology to explain the emotional responses (affective states) attackers seek to trigger in phishing emails. We conducted a pilot study and found that affective states can be key differentiators between phishing and non-phishing emails. Affective states commonly characterizing phishing emails, such as fear and greed, are almost absent in non-phishing emails. Additionally, phishing emails evoke higher emotional arousal levels than non-phishing emails in both subjects and bodies. Furthermore, we explore the capability of several LLMs (GPT-4, Llama 3, and Gemini Pro) in detecting affective states in over 5,000 phishing emails targeting six universities. Using reliability statistics, we compared LLMs’ performance with human annotators’ performance, mapped exploited affective states to the valence-arousal Cartesian space, examined LLMs’ performance in detecting phishing emails with special characters, and analyzed LLM hallucinations. We found that attackers commonly exploit curiosity, trust, and urgency in phishing email subjects and fear, urgency, and trust in their bodies. While fear and urgency elevate arousal levels, the valence of the email body generally remains neutral. Key affective state pairings include fear-urgency, urgency-trust, and trust-curiosity. Among LLMs, GPT4 outperforms Llama 3 and GeminiPro for both body and subject affective states, followed by Llama 3 and GeminiPro. These findings highlight the need for further research on affective states in phishing emails to improve detection systems.