Fraud detection in big data using supervised and semi-supervised learning techniques

This paper addresses the credit card fraud detection problem in the context of Big Data, based on machine learning techniques. In the fraud detection task, typically the available datasets for ML training present some peculiarities, such as the unavoidable condition of a strong class imbalance, the existence of unlabeled transactions, and the large number of records that must be processed. The present paper aims to propose a methodology for automatic detection of fraudulent transactions, that tackle all these problems. The methodology is based on a Balanced Random Forest, that can be used in supervised and semi-supervised scenarios through a co-training approach. Two different schemes for the co-training approach are tested, in order to overcome the class imbalance problem. Moreover, a Spark platform and Hadoop file system support our solution, in order to enable the scalability of the proposed solution. The proposed approach achieves an absolute improvement of around 24% in terms of geometric mean in comparison to a standard random forest learning strategy.