A Recursive Implementation of Sparse Regression

Sparse regression deals with the problem of representing a dataset using only a few non-zero basis elements. This work presents a recursive implementation of sparse regression, with the dataset being processed sequentially rather than as a batch. The algorithm, named sparse regularized fused recursive least squares (SP-RF-RLS), uses a re-weighting technique and a smooth approximation to deal with the discontinuous $\ell_{0}$ norm and the non-differentiable $\ell_{1}$-norm, standard norms for sparsity. Inspired by fused least absolute shrinkage and selection operator (fused-LASSO), the algorithm aims to capture structures in the locations of the non-zero elements by including a term depending on the difference between the estimated elements. Comparative experiments in both sparse and non-sparse scenarios show that SP-RF-RLS outperforms several state-of-the-art recursive algorithms.