Robust Sparse Online Learning through Adversarial Sparsity Constraints

In this paper, we propose a novel robust sparse online learning framework named Adversarial Sparse Online Learning (ASOL) for high dimensional data streams, which is implemented by the adversarial sparse learning over the ℓ
1
-norm and ℓ
12
-norm constraints. Our main idea is to exploit adversarial learning between ℓ
1
-norm based constraint on the current weight vector and ℓ
12
-norm based constraint on the incremental weight matrix, achieving robust sparse solutions of the online weight vectors through such adversarial sparsity constraints. Compared with projection-based or ℓ
p
-norm based sparsity-introducing mechanisms, our proposed ASOL algorithm is to find an equilibrium and achieve stable sparse solutions by simultaneously considering the sparsity structure of the current weight vector through the ℓ
1
-norm constraint as well as the correlations and sparsity structure of previous weight vectors through the ℓ
12
-norm constraint. Extensive empirical studies have demonstrated the effectiveness and efficiency of the proposed method.