Unknown Sparsity in Compressed Sensing: Denoising and Inference

The theory of compressed sensing (CS) asserts that an unknown signal x ϵ Rp can be accurately recovered from an underdetermined set of n linear measurements with n ≪ p provided that x is sufficiently sparse. However, in applications, the degree of sparsity IlxIl
₀
is typically unknown, and the problem of directly estimating IlxIl
₀
has been a longstanding gap between theory and practice. A closely related issue is that IlxIl
₀
is a highly idealized measure of sparsity, and for real signals with entries not equal to 0, the value IlxIl
₀
= p is not a useful description of compressibility. In our previous conference paper [1] that examined these problems, we considered an alternative measure of soft sparsity, IlxIl
₁
²
/IlxIl
₂
²
, and designed a procedure to estimate IlxIl
₁
²
/IlxIl
₂
²
that does not rely on sparsity assumptions. This paper offers a new deconvolution-based method for estimating unknown sparsity, which has wider applicability and sharper theoretical guarantees. In particular, we introduce a family of entropy-based sparsity measures sq(x) = (IlxIl
_q
/IlxIl
₁
)
^(q/1-q)
parameterized by q ϵ [0, ∞]. This family interpolates between IlxIl
₀
= s
_q
(x) and IlxIl
₁
²
/IlxIl
₂
²
= s
₂
(x) as q ranges over [0, 2]. For any q ϵ (0, 2] \ 111, we propose an estimator ŝ
_q
(x) whose relative error converges at the dimension-free rate of 1/√n, even when p/n → ∞. Our main results also describe the limiting distribution of ŝ
_q
(x), as well as some connections to basis pursuit denosing, the Lasso, deterministic measurement matrices, and inference problems in CS.