Rank Detection Thresholds for Hankel or Toeplitz Data Matrices

In Principal Component Analysis (PCA), the dimension of the signal subspace is detected by counting the number of eigenvalues of a covariance matrix that are above a threshold. Random matrix theory provides accurate estimates for this threshold if the underlying data matrix has independent identically distributed columns. However, in time series analysis, the underlying data matrix has a Hankel or Toeplitz structure, and the columns are not independent. Using an empirical approach, we observe that the largest eigenvalue is fitted well by a Generalized Extreme Value (GEV) distribution, and we obtain accurate estimates for the thresholds to be used in a sequential rank detection test. In contrast to AIC or MDL, this provides a parameter that controls the probability of false alarm. Also a lower bound is presented for the rank detection rate of threshold-based detection for rank-1 problems.