Hey ChatGPT, Is This Message Phishing?

This paper investigates the effectiveness of AI-based tools, with a focus on utilizing ChatGPT as a test platform, in email phishing detection, addressing the pressing need to combat this pervasive threat in the digital landscape. With phishing attacks causing substantial financial losses amounting to billions of dollars annually, innovative approaches are essential to mitigate their impact. We propose leveraging ChatGPT for email phishing detection, capitalizing on its machine learning and natural language processing capabilities to classify emails based on nuanced patterns and variations in phishing techniques. Unlike traditional rule-based approaches, AI-based tools offer a promising avenue for enhanced detection accuracy. Through rig-orous testing focusing on binary mail classification, incorporating metrics such as accuracy, specificity, sensitivity, precision, and the F1 Score, ChatGPT demonstrates commendable performance, achieving an overall accuracy of 75.75% and an impressive sensitivity of 98.4 %. These findings underscore the potential of AI-based tools as valuable assets in mitigating phishing threats. Comparison with FortiSandbox, a widely-used solution, reveals promising potential for AI-based tools, albeit with recognition of the need for further development and refinement. Our evaluation, encompassing various aspects of. eml file analysis, highlights both the strengths and limitations of AI-based tools in identifying phishing emails. While this research lays a strong foundation for leveraging AI -based tools in email phishing detection, there exist opportunities for improvement. Addressing identified limitations and exploring future research directions will be crucial in advancing the efficacy of AI-based tools and enhancing email security measures against cyber threats.