Output Kernel Learning for Multi-Output Regression

Multi-output regression is the extension of single output learning, which requires one input sample to predict multiple real-valued output variables simultaneously. Previous studies have shown that multi-output learning yield to a better predictive performance, mostly because of the multivariate nature and the compound dependencies between the multiple feature/target variables. But these relationships are sometimes hard to detect directly, so it is a challenge to reveal the relationship between these input features and output variables. In order to improve the accuracy of the prediction, we propose a new kernel-based multi-output regression method to study the internal relationship between output variables, called KMOR. Our method is searched the framework between multiple functions and output into a reproducing kernel Hilbert space of vector-valued functions. It constructs input kernel by user-defined to represent the relationship between input characteristics and output kernel by auto-learned to represent the relationship between output variable, we also use an efficient alternating optimization to solve the objective function. We have done experiments on fifteen data sets from a variety of interesting application domains and use relative root mean squared error (RRMSE) as performance measurement, which show that this method can improve the accuracy of regression and has very fast convergence speed in comparison with state-of-the-art methods for multi-output regression.