Multilinear Discriminant Analysis Through Tensor-Tensor Eigendecomposition

The current paper presents a new approach to dimensionality reduction and supervised learning for classification of multi-class data. The approach is based upon recent developments in tensor decompositions and a newly defined algebra of circulants. In particular, it is shown that under the right tensor multiplication operator, a third order tensor can be written as a product of third order tensors that is analogous to a traditional matrix eigenvalue decomposition where the "eigenvectors" become eigenmatrices and the "eigenvalues" become eigen-tuples. This new development allows for a proper tensor eigenvalue decomposition to be defined and has natural extension to tensor linear discriminant analysis (LDA). Comparisons are made with traditional LDA and it is shown that the current approach is capable of improved classification results for benchmark datasets involving faces, objects, and hand written digits.