Online Passive-Aggressive Multilabel Classification Algorithms

Most existing multilabel classification methods are batch learning methods, which may suffer from expensive retraining costs when dealing with new incoming data. In order to overcome the drawbacks of batch learning, we develop a family of online multilabel classification algorithms, which can update the model instantly and efficiently, and make a timely online prediction when new data arrive. Our algorithms all take a closed-form update, which is obtained by solving a constrained optimization problem in each round of online learning. Label correlation is explicitly modeled in our optimization problem. The label thresholding function, an important component of our online classifier, can also be learned online. Our algorithms can be easily generalized to the nonlinear prediction cases using Mercer kernels. The worst case loss bounds for our algorithms are provided. The bounds are relative to the cumulative loss suffered by the best fixed predictive model that can be attained in hindsight. Finally, we corroborate the merits of our algorithms in both linear and nonlinear predictions on nine open multilabel benchmark datasets.

I. Introduction

Multilabel classification has recently received widespread attentions due to its increasing applications, such as in text categorization [1], [2], image classification [3]–[5], bioinformatics [6], [7], and medical diagnosis [8], [9]. In traditional single-label classification tasks, each data object is associated with a single label. In contrast, in multilabel classification tasks, each data object is associated with a subset of labels, called “relevant labels,” while the remaining labels are called “irrelevant labels.” Often, there exists a correlation among labels. For example, an image with the labels “sun” and “beach” is more likely related to the label “ocean” but less likely related to the label “rain.” The objective of multilabel classification is to recognize relevant labels from a predefined label set for a given input.