Learning Output Relevant Features by Joint Autoencoder

Classical regression is a supervised task that uses target output to guide the modeling. Generally, the original input contains both output relevant and irrelevant information, whereas the latter may decrease the predictive performance to a certain extent. Without prior information about what information contributes to the prediction, the predictive performance of regression models can be improved based on such output relevant information as input. Thus, a joint autoencoder (JAE) combining the supervised and unsupervised mechanisms is proposed to learn output relevant features from the original input. In this way, both predictive and reconstructive performance are considered to learn the essential characteristics and avoid poor generalized performance on untrained instances. Meanwhile, hierarchy representations can be learned by successive JAEs with a local parameter embedding strategy, which is presented to preserve the predictive performance in this structure. In the experiments, the predictive performance and robustness of the proposal are verified on nine datasets with different sample sizes.