Variable Selection for Max-Affine Regression via Sparse Gradient Descent

Sparse max-affine regression is introduced as a solution to variable selection for non-linear learning problems. A non-asymptotic local convergence analysis for Sparse Gradient Descent (Sp-GD) is presented when covariates are independent and identically distributed Gaussian random vectors. If at most $s$ out of $d$ covariates are actively contributing to the explanation of the target variable and the max-affine model combines a fixed number of linear models, then a suitably initialized Sp-GD linearly approaches the optimal solution and the true active covariates given $n=\mathcal{O}(s\log[(n\vee ed)/s])$ noise-free samples. Numerical Monte Carlo results corroborate these theoretical findings on the phase transition of exact parameter recovery.