Default Prediction on Commercial Credit Big Data Using Graph-based Variable Clustering

Credit-lending organizations have resorted to the use of machine learning (ML) algorithms in the recent past to predict the probability of the default of a business. Explainability of the decisions made by the traditional statistical algorithms like Logit models brings transparency to every stakeholder involved in the process. On the other hand, machine learning models like XGBoost and Neural Nets have achieved better accuracy scores, but their decisions are not easily comprehensible. In this paper, we propose a graph based variable clustering (GVC) method as a filter based approach to select prominent features while retaining as much variance as possible. Our experiments show that our GVC approach is not only almost 40 times faster than the existing variable clustering methods but retains retains 5% more variance than the existing packages.The feature set from GVC approach has performed better with an increase of 6% accuracy on an average. The predictions on the feature set from GVC were 98% accurate using XGBoost algorithm.