Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices

We study the information-theoretic limits of exactly recovering the support set of a sparse signal, using noisy projections defined by various classes of measurement matrices. Our analysis is high-dimensional in nature, in which the number of observations
n
, the ambient signal dimension
p
, and the signal sparsity
k
are all allowed to tend to infinity in a general manner. This paper makes two novel contributions. First, we provide sharper necessary conditions for exact support recovery using general (including non-Gaussian) dense measurement matrices. Combined with previously known sufficient conditions, this result yields sharp characterizations of when the optimal decoder can recover a signal for various scalings of the signal sparsity
k
and sample size
n
, including the important special case of linear sparsity
(k
= ¿(
p
)) using a linear scaling of observations
(n
= ¿(
p
)). Our second contribution is to prove necessary conditions on the number of observations
n
required for asymptotically reliable recovery using a class of
¿
-sparsified measurement matrices, where the measurement sparsity parameter
¿(n
,
p
,
k
) ¿ (0,1] corresponds to the fraction of nonzero entries per row. Our analysis allows general scaling of the quadruplet
(n
,
p
,
k
, ¿) , and reveals three different regimes, corresponding to whether measurement sparsity has no asymptotic effect, a minor effect, or a dramatic effect on the information-theoretic limits of the subset recovery problem.