Dual Passive-Aggressive Stacking k-Nearest Neighbors for Class-Incremental Multi-Label Stream Classification

Class-incremental multi-label stream classification (class-incremental MLSC) requires learning algorithms to adapt to concept drifts, perform single-pass online learning, and handle emerging new labels in the data stream. Most learning algorithms in the literature can fulfill only a subset but not all of these desiderata. This paper proposes a stacking ensemble method named Dual Passive-Aggressive stacking k-Nearest Neighbors (DPAkNN) to tackle class-incremental MLSC. DPAkNN uses a vanilla Passive-Aggressive classifier (vanilla PA) and a Random Kitchen Sink Passive-Aggressive classifier (RKS PA) for its base model, and a stacking kNN as the meta model - hence the name. The RKS PA complements the vanilla PA in the base model by handling non-linearities in the data stream. To refine predictions and capture label dependencies, predictions from the base model are fed into a streaming kNN named MetakNN to output the stacked predictions. MetakNN includes label-specific fitness for stored instances and a fitness rejuvenating mechanism based on relative performance to the base predictions for enhanced synergy. Our experiment compares DPAkNN against state-of-the-art kNN-based MLSC algorithms on 30 multi-label benchmark datasets. Results demonstrate the superior performance of DPAkNN in MLSC tasks - overall, DPAkNN has the best performance on the benchmarking tasks with an average rank of 2.62, demonstrating strong predictive power with decent runtime and memory consumption. Our further assessment also shows that DPAkNN is robust to various forms of real concept drift and capable of class-incremental learning.