Optimizing Phishing URL Detection: A Comparative Analysis of Feature Counts and Training Times

Phishing attacks are the biggest cybersecurity threats in the digital world. Attackers exploit users by impersonating real, authentic web sites to obtain sensitive information such as passwords and bank statements. One common technique used in these attacks is malicious URLs. These malicious URLs mimic legitimate URLs, misleading users into interacting with malicious websites. This practice, URL phishing, presents a big threat to internet security, emphasizing the need for advanced detection methods. This paper presents a method for detecting malicious URLs using statistical features extracted from n-grams. These n-grams are extracted from the hexadecimal representation of URLs. We used Explainable AI (XAI) to explore the extracted features and evaluate their importance and role in phishing detection. A key advantage of our method is its ability to reduce the required features and the training time by using fewer features after applying XAI techniques. Our technique only uses statistical features extracted from n-grams and the n-gram itself, without requiring any high-level features. We applied XAI techniques, SHapley Additive exPlanations (SHAP), and Local Interpretable Model-agnostic Explanations (LIME). Through the explanation provided by XAI methods, we were able to determine the most important features in our feature set, enabling a reduction in feature count. Using fewer features (4, 7, 10, 13, 15), we got good accuracy compared to the 41 features and 87 features used in the original experiment and reduced the models' training times and complexity. This research aimed to enhance phishing URL detection by optimizing feature selection and analyzing the impact on training times. Our findings show the importance of using minimal statistical features to identify malicious URLs effectively and efficiently.