Multiclass Generalization Memorization Machine

Recently, generalization memorization machine (GMM) has been proposed with zero empirical risk and well generalization in small sample size binary classification problems. This paper extends the GMM into multiclass classification. Different from the one-vs-one (1v1) or one-vs-rest (1vR) strategy, we propose a Multiclass Generalization Memorization Machine (MGMM) by only solving a quadratic programming problem, Compared with the strategy of extending the classical support vector machine into multiclass classification proposed by Crammer et al. MGMM has stronger generalization capabilities. The Crammer’s strategy was proposed two decades ago. Since its generalization ability is limited by the potential higher empirical risk, the extend of useage is less than the 1v1 and 1vR strategies. Correspondingly, our MGMM always obtains zero empirical risk theoretically and preserves the well generalization from the GMM. Preliminary experimental results verify the advantages of our MGMM compared with several state-of-the-art multiclass classification methods.