A Bi-Level Evolutionary Model Tree Induction Approach for Regression

Supervised machine learning techniques include classification and regression. In regression, the objective is to map a real-valued output to a set of input features. The main challenge that existing methods for regression encounter is how to maintain an accuracy-simplicity balance. Since Regression Trees (RTs) are simple to interpret, many existing works have focused on proposing RT and Model Tree (MT) induction algorithms. MTs are RTs with a linear function at the leaf nodes rather than a numerical value are able to describe the relationship between the inputs and the output. Traditional RT induction algorithms are based on a top-down strategy which often leads to a local optimal solution. Other global approaches based on Evolutionary Algorithms (EAs) have been proposed to induce RTs but they can require an important calculation time which may affect the convergence of the algorithm to the solution. In this paper, we introduce a novel approach called Bi-level Evolutionary Model Tree Induction algorithm for regression, that we call BEMTI, and which is able to induce an MT in a bi-level design using an EA. The upper-level evolves a set of MTs using genetic operators while the lower-level optimizes the Linear Models (LMs) at the leaf nodes of each MT in order to fairly and precisely compute their fitness and obtain the optimal MT. The experimental study confirms the outperformance of our BEMTI compared to six existing tree induction algorithms on nineteen datasets.