Transformation of Regressors for Low Coherent Sparse System Identification

One of the main problems in sparse representation in order to achieve accurate parameter estimation is the potentially large correlation between the regressors. The maximum absolute value of this correlation is called
mutual coherence
. The inverse of the mutual coherence controls approximately the number of nonzero parameters that can be recovered exactly from noise-free observations. In system identification, the observations are constructed from time-series data, which leads to that the columns in the regressor matrix may be highly correlated, and therefore, the mutual coherence can be large. This implies that using standard sparse estimation algorithms may lead to large errors. In this article, we derive a bound on the mutual coherence, which shows that directly using popular algorithms such as the orthogonal matching pursuit (OMP) algorithm can lead to large estimation errors. The key idea in this article is to make a coordinate transformation so that the regressor matrix has low mutual coherence in the new coordinates. The proposed method involves solving a nonconvex optimization problem, which is done by alternating minimization and proximal methods. Additionally, we derive a bound on the mutual coherence of the regressor in the new coordinates, which can be used to ensure the correct estimation of the support of the parameters for OMP. For this method, we show also that the criterion used to compute the transformation maximizes a lower bound for the probability of correct estimation of the support. For a limited setting, we also show formally that the transformation can significantly increase the probability of correct estimation of the support. Finally, we show via simulations how the proposed method performs in comparison with state-of-the-art sparse estimation algorithms.