Incorporating Linear Regression Problems Into an Adaptive Framework With Feasible Optimizations

Accompanied with the increasing popularity of linear regression approaches, most of the existing minimization problems are related with several convex measurements, e.g., $\ell _{1}$/$\ell _{2}$/$\ell _{2,1}$-norm of a vector and $L_{1}$/$L_{2,1}$/Frobenius/nuclear norm of a matrix, where the regularized function and the loss function are usually studied for two objective terms case by case, respectively. To address this issue, this work combines these linear regression problems into a unified expression framework by employing an adaptive and flexible function, in which we need to choose different variable elements and adjust an inner parameter, properly. Besides this, they are equipped with some corresponding relationships and their interesting properties. Intuitively speaking, the proposed framework can generalize several traditional linear regression formulations and even more complex ones into an extended representation. For further optimizations, an iteratively re-weighted penalty solution (IRwPS) is devised without any inner loops, making the iteration programming easy to perform. Meanwhile, the theoretical results are provided for guaranteeing that the mathematical convergence analysis is solid and meaningful. Finally, by performing real-world applications in supervised, unsupervised, and semi-supervised tasks, numerical experiments are conducted to validate the theoretical properties and the superiority over some of the state-of-the-art.