Evaluating the Effectiveness of the CatBoost Classifier in Distinguishing Benign Traffic, FTP BruteForce and SSH BruteForce Traffic

In this study, we enhance information security practices by leveraging the CatBoost classifier within the challenging domain of network intrusion detection. Building upon the foundation of machine learning techniques in identifying FTP and SSH brute force attacks, our investigation introduces a refined CatBoost classifier approach by addressing the escalating sophistication of these attacks, our research meticulously analyzes the comprehensive CSE-CIC-IDS2018 and CICIDS2017 datasets. Through detailed feature selection, hyperparameter optimization, and class balancing with techniques like SMOTENC, our CatBoost model achieves high accuracy, with a rate of 99.9964% and low false alarm rate. Notably, the model demonstrates robust performance metrics, including a high F1 score over iterations, minimal loss, and a near-perfect Matthew's correlation coefficient (MCC), signifying its precision and reliability. A distinctive aspect of our study is the extensive application of SHAP (SHapley Additive exPlanations) values, providing deep insights into the model's decision-making process. These include Mean Absolute SHAP values and SHAP Value Distributions, which elucidate the significant impact and contribution of individual features to the model's predictions. Moreover, the model exhibits impressive computational efficiency, maintaining a high average processing speed of 5 million samples per second, that underscores its suitability for real-time security applications. These findings underscore the effectiveness of the CatBoost classifier in accurately identifying complex network intrusion patterns. Our investigation underscores the practical benefits of applying the CatBoost classifier in information security, contributing to the ongoing enhancement of cyber defense mechanisms with improved efficiency and interpretability.