Tractable Calculation and Estimation of the Optimal Weighting Matrix for ALS Problems

We study autocovariance least squares (ALS) estimation methods for covariance estimation for linear time-invariant systems. Previous works have posited that calculation of the ALS weighting matrix is intractable unless the number of data points
$N_d$
is small because it requires storage of a matrix whose number of elements scales as
$N_d^4$
. We derive a novel way to compute the weight that avoids this difficulty. In practice, the true optimal weight cannot be calculated because it is a function of the sought covariance matrices. However, our work enables implementation of two novel ALS algorithms that estimate the weight from data. For the purpose of comparison, we also discuss ALS with an arbitrary weight (such as an identity matrix) and present a previously published method for estimating the ALS weight. ALS with an identity weight guarantees unbiased and consistent covariance estimates, but algorithms that estimate the weight from data do not inherit these guarantees. Despite this drawback, we present a numerical example for which the best performing algorithm, iterative estimation of the covariances and the ALS weight, produces covariance estimates with a small amount of bias and a significantly reduced variance compared to all other algorithms.