Gradient Boosting-Based Simultaneous Classification and Regression Approach

The recent development of advanced data analytics and machine learning promoted the introduction of diverse learning techniques designed to alleviate challenges related to two major supervised learning tasks: (i) Classification and (ii) Regression. Despite the successful consideration of these tasks to address problems relevant to various applications, the design of dual machine learning approaches that perform both tasks simultaneously remains challenging. This paper proposes a Dual-learning XGBoost Based Approach (DXGBA), a novel gradient boosting-based machine learning approach that concurrently carries out classification and regression tasks. Specifically, the proposed approach aims at learning a supervised learning model that is jointly trained for both classification and regression using a single unified framework. The optimization of this dual task is formulated using a novel joint cost function that minimizes the total error of both classification and regression tasks during the learning phase. The proposed approach was assessed using benchmark datasets and performance measures. The experiments proved its effectiveness compared to native eXtreme Gradient Boosting (XGBoost) models. DXGBA achieved an F1-score of 0.4718 and an MSE of 0.0002 on the bankruptcy dataset, outperforming the single-task XGBoost, which yielded 0.3942 and 0.0003, respectively. Moreover, DXGBA reduced testing time by up to 81.9% compared to baseline methods. It also evidenced better performance relative to conventional machine learning models.