Estimating the Number of Significant Canonical Coordinates

This study proposes a novel model selection criterion for dimensionality estimation in canonical correlation analysis (CCA), which can be used to estimate the number of correlated components between two sets of multivariate vectors, particularly in the context of canonical coordinates. The proposed method has a different form compared to existing model selection criteria, which typically use the Bartlett-Lawley measure as a goodness-of-fit term. Alternatively, we propose to use instead a goodness-of-fit term based on the sum of squares of the smallest canonical correlation coefficients. The asymptotic properties of the proposed goodness-of-fit term have been laid out and an appropriate penalty term with proof of consistency is derived based on these properties. Numerical examples are presented in the context of direction-of-arrival estimation using canonical coordinates for two channel array systems. It is shown that our proposed order selection criterion outperforms conventional criteria and is robust to various noise and interference scenarios.