SLMOML: Online Metric Learning With Global Convergence

Metric and similarity learning are important approaches to classification and retrieval. To efficiently learn a distance metric or a similarity function, online learning algorithms have been widely applied. In general, however, existing online metric and similarity learning algorithms have limited performance in real-world classification and retrieval applications. In this paper, we introduce a convergent online metric learning model named scalable large margin online metric learning (SLMOML). SLMOML belongs to the passive-aggressive learning family. At each step, it adopts the LogDet divergence to maintain the closeness between two successively learned Mahalanobis matrices, and utilizes the hinge loss to enforce a large margin between relatively dissimilar samples. More importantly, the Mahalanobis matrix can be updated by closed-form solution at each step. Furthermore, if the initial matrix is positive semi-definite (PSD), the learned matrices in the following steps are always PSD. Based on the Karush–Kuhn–Tucker conditions and the built equivalence between the passive-aggressive learning family and the Bregman projections, we have proved the global convergence of SLMOML. Extensive experiments on classification and retrieval tasks demonstrate the effectiveness and efficiency of SLMOML.

I. Introduction

Distance metric and similarity function play an important role in many classification and retrieval applications, such as that based on the computation of -nearest neighbors. Selecting an appropriate distance or similarity measurement is crucial to the success of addressing these problems. In the past two decades, many metric and similarity learning approaches have been proposed, and their superiority over the standard Euclidean distance has been demonstrated [1]–[3]. However, metric and similarity learning is highly problem-specific and challenging.