By Topic

A Geometric Perspective of Large-Margin Training of Gaussian Models [Lecture Notes]

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Lin Xiao ; Machine Learning Group, Microsoft Res., Redmond, WA, USA ; Li Deng

Large-margin techniques have been studied intensively by the machine learning community to balance the empirical error rate on the training set and the generalization ability on the test set. However, they have been mostly developed together with generic discriminative models such as support vector machines (SVMs) and are often difficult to apply in parameter estimation problems for generative models such as Gaussians and hidden Markov models. The difficulties lie in both the formulation of the training criteria and the development of efficient optimization algorithms. In this article, we consider the basic problem of large margin training of Gaussian models. We take the geometric perspective of separating patterns using concentric ellipsoids, a concept that has not generally been familiar to signal processing researchers but which we will elaborate on here. We describe the approach of finding the maximum-ratio separating ellipsoids (MRSEs) and derive an extension with soft margins. We show how to formulate the soft-margin MRSE problem as a convex optimization problem, more specifically a semidefinite program (SDP). In addition, we derive its duality theory and optimality conditions and apply this method to a vowel recognition example, which is a classical problem in signal processing.

Published in:

Signal Processing Magazine, IEEE  (Volume:27 ,  Issue: 6 )