Online payment fraud monitoring and detection: Performance analysis of tree-based Ensemble Machine Learning models

In today’s world of digital transformation in business, driven by rapid technological advancement, the use of electronic commerce and digital banking is ever increasing. In line with global trends, the Indian banking and financial sector is experiencing a major adoption of digital payments, be it smaller retail payments or high-value corporate payments. With the increasing use of online payments, money laundering, cyberattacks, and other fraudulent activities are increasing. The innovative use of technology by fraudsters, aided by dynamic and irregular patterns of fraudulent activities, poses additional challenges in identifying frauds with high accuracy. Hence, preventing financial fraud requires the implementation of secured payment systems and efficient fraud detection and monitoring systems.This study delves into analyzing the results of applying ensemble-based Machine Learning (ML) methods to digital payment transactions, aiming to enhance the accuracy of fraud detection. A recent study and review of the existing literature show better performance by the Decision Tree algorithm over other ML models. Taking this into account, we used the decision tree algorithm to further build tree-based ensemble models, namely Random Forest, Gradient Tree Boosting, AdaBoost, and XGBoost. We analyze seven evaluation metrics: Precision, Recall, F-score, Accuracy, Misclassification Error, Area Under the Receiver Operating Characteristic Curve, and Cohen Kappa score. The data sets used are the National Electronic Fund Transfer (NEFT) payment transactions. The results show encouraging scores in segregating valid and fraudulent transactions, and thereby suggest the use of tree-based ensemble models as an effective tool for continuous monitoring and early detection of fraudulent payment transactions.