The squared-error of generalized LASSO: A precise analysis

We consider the problem of estimating an unknown but structured signal x
₀
from its noisy linear observations y = Ax
₀
+ z ∈ ℝ
^m
. To the structure of x
₀
is associated a structure inducing convex function f(·). We assume that the entries of A are i.i.d. standard normal N(0, 1) and z ~ N(0, σ
²
I
_m
). As a measure of performance of an estimate x* of x
₀
we consider the “Normalized Square Error” (NSE) ∥x* - x
₀
∥
₂
²
/σ
²
. For sufficiently small σ, we characterize the exact performance of two different versions of the well known LASSO algorithm. The first estimator is obtained by solving the problem argmin
_x
∥y - Ax∥
₂
+ λf(x). As a function of λ, we identify three distinct regions of operation. Out of them, we argue that “R
_ON
” is the most interesting one. When λ ∈ R
_ON
, we show that the NSE is D
_f
(x
₀
, λ)/m-D
_f
(x
₀
, λ) for small σ, where D
_f
(x
₀
, λ) is the expected squared-distance of an i.i.d. standard normal vector to the dilated subdifferential λ · ∂f(x
₀
). Secondly, we consider the more popular estimator argmin
_x
1/2∥y - Ax∥
₂
²
. + στ f(x). We propose a formula for the NSE of this estimator by establishing a suitable mapping between this and the previous estimator over the region R
_ON
. As a useful side result, we find explicit formulae for the optimal estimation performance and the optimal penalty parameters λ* and τ*.