A Complete Canonical Correlation Analysis for Multiview Learning

Canonical correlation analysis (CCA) is an effective feature learning method, which has wide applications in pattern recognition and computer vision. However, CCA considers the correlation only between the one-to-one aligned samples in two views, ignoring the correlation between all the samples sharing the same label. In this paper, we propose a deep complete canonical correlation analysis (Deep Complete-CCA), which learns the relationships between all pairwise correspondences of sample points in the same classes. Unlike CCA, our method can learn discriminant representations that maximize the correlation between the two views while segregating the different classes on the learned space. We test Deep Complete-CCA on handwriting recognition and speech based emotion recognition using two popular MNIST and RAVDESS datasets. Experimental results show that our proposed method can obtain better performances than several related algorithms.