The Sample Complexity of Weighted Sparse Approximation

For Gaussian sampling matrices, we provide bounds on the minimal number of measurements m required to achieve robust weighted sparse recovery guarantees in terms of how well a given prior model for the sparsity support aligns with the true underlying support. Our main contribution is that for a sparse vector x ∈ R
^N
supported on an unknown set S ⊂ {1, ..., N} with |S| ≤ κ, if S has weighted cardinality ω(S) : = Σ
_j∈S
ω
_j
²
, and if the weights on S
^c
exhibit mild growth, ω
_j
²
≥ γlog(j/ω(S)) for j ∈ S
^c
and γ > 0, then the sample complexity for sparse recovery via weighted ℓ
₁
-minimization using weights ω
_j
is linear in the weighted sparsity level, and m = O(ω(S)/γ). This main result is a generalization of special cases including a) the standard sparse recovery setting where all weights ω
_j
≡ 1, and m = O(k log(N/κ)); b) the setting where the support is known a priori, and m = O(κ); and c) the setting of sparse recovery with prior information, and m depends on how well the weights are aligned with the support set S. We further extend the results in case c) to the setting of additive noise. Our results are nonuniform that is they apply for a fixed support, unknown a priori, and the weights on S do not all have to be smaller than the weights on S
^c
for our recovery results to hold.