Ensemble Regression Method with Pairwise Complementary Models

In ensemble regression three steps can be distinguished: first, the set of base models is created, next the ensemble of best models is selected and finally predictions of individual models are integrated to receive common prediction for a given test sample. In this paper a novel method for selection of base models in dynamic mode is developed. According to the proposed concept, models are selected in pairs of complementary models, so as to minimize the mutual prediction error in the predefined area around the test object. In order to define the area in which a pair of complementary models is sought, the concept of orthants in the space of independent variables is used. The proposed method was experimentally tested and compared against 6 literature ensemble regression methods using 14 benchmark data bases and 3 homogeneous ensembles of base models: multilayer perceptrons, 5-nearest neighbor models and linear regression models. The experimental results demonstrate that the proposed method leads to the statistically significant smaller values of the relative root mean squared error of the prediction.