Independent Feature and Label Components for Multi-label Classification

Investigating correlation between example features and example labels is essential to solve classification problems. However, identification and calculation of the correlation between features and labels can be rather difficult for high-dimensional multi-label data. Both feature embedding and label embedding have been developed to tackle this challenge, and a shared subspace for both labels and features are usually learned by existing embedding methods to simultaneously reduce dimensionality of features and labels. In contrast, this paper suggests to learn separated subspaces for features and labels by maximizing the independence between components in each subspace and maximizing the correlation between these two subspaces. The learned independent label components indicates fundamental combinations of labels in multi-label datasets, which thus helps to reveals the correlation between labels. On the other hand, the learned independent feature components lead to a compact representation of example features. The connections between the proposed algorithm and existing embedding methods have been discussed. Experimental results on real-world multi-label datasets demonstrate the necessity of exploring independence components from multi-label data and the effectiveness of the proposed algorithm.