Enhancing Credit Scoring Models: Unveiling the Impact of Data Preprocessing and Feature Selection Techniques: Case Study based on Microfinancial institution in Bosnia and Herzegovina

In recent years, credit scoring has gained significant attention as an essential process in the financial industry. With the growing adoption of machine learning techniques in different stages of model creation, the accuracy and reliability of credit scoring models have greatly improved. However, the quality and characteristics of input data play a pivotal role in the performance of these models. This research explores the critical aspects of data preprocessing and feature selection techniques and their impact on the efficiency of machine learning classification models for credit scoring. Our research represents a comparative analysis of several hybrid credit scoring models trained on 12 datasets engaging different data preprocessing techniques and 9 feature selection methods. In addition, four custom feature selection methods called Any, Majority, Vast majority 7/9, and Vast majority 8/9 were proposed and used to train the models using Logistic Regression, Decision Tree and K-nearest neighbors as main classifiers. The results of 48 different models were evaluated based on five different performance measures: Accuracy, Precision, Recall, F1 Score and Area under the ROC Curve (AUC). While the hybrid models using the proposed feature selector Majority showed the best results for Logistic Regression and Decision Tree, the hybrid model based on the proposed feature selector Vast Majority 8/9 showed the best performance in terms of all performance measures for KNN classifier. Through a comprehensive evaluation and analysis, this paper showed the importance of optimizing data preprocessing and feature selection to achieve higher predictive accuracy and model interpretability in credit scoring applications.