Advances in Large-Margin Classifiers

This chapter contains sections titled: Half Title, Published by Morgan-Kaufmann, Title, Copyright, Contents, Series Forword, Preface View full abstract»

This chapter contains sections titled: A Simple Classification Problem, Theory, Support Vector Machines, Boosting, Empirical Results, Implementations, and Further Developments, Notations View full abstract»

This chapter contains sections titled: Support Vector Machines, Kernel Machines, Boosting, Leave-One-Out Methods, Beyond the Margin View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

This chapter contains sections titled: Introduction: Linear Methods using Kernel function, Applying Linear Methods to Structured Objects, Conditional Symmetric Independence Kernels, Pair Hidden Markov Models, Conditionally Symmetrically Independent PHMMs, Conclusion View full abstract»

This chapter contains sections titled: Introduction, Natural Kernels, The Natural Regularization Operator, The Feature Map of Natural Kernel, Experiments, Discussion View full abstract»

This chapter contains sections titled: Introduction, Fitting a Sigmoid After the SVM, Empirical Tests, Conclusions, Appendix: Pseudo-code for the Sigmoid Training View full abstract»

This chapter contains sections titled: Introduction, Basic Approximation Steps, Basic Algorithms, Kernel Machine Extension, Soft Margin Extension, Experimental Results, Discussion, Conclusions, Appendix: Details of comparison against six other methods for iterative generation of support vector machines View full abstract»

This chapter contains sections titled: Introduction, Classical Models for Ordinal Regression, A Risk Formulation for Ordinal Regression, An Algorithm for Ordinal Regression, Experimental Results, Discussion and Conclusion, Acknowledgments View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

This chapter contains sections titled: Introduction, GSVM: The General Support Vector Machine, Quadratic Programming Support Vector Machines, Linear Programming Support Vector Machines, A Simple Illustrative Example, Conclusion, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, What is a Linear Discriminant?, Formulation of the Linear Discriminant Training Problem, Training Algorithms, Which Linear Discriminant?, Conclusion, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, Overview of Statistical Learning Theory, Regularization Networks, Support Vector Machines, SRM for RNs and SVMs, A Bayesian Interpretation of Regularization and SRM?, Connections Between SVMs and Sparse Approximation Techniques, Remarks, Acknowledgments View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

This chapter contains sections titled: Introduction, Boosting and the Linear Programming Solution, υ-Algorithms, Experiments, Conclusion, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, Optimizing Cost Functions of the Margin, A Gradient Descent View of Voting Methods, Theoretically Motivated Cost Functions, Convergence Results, Experiments, Conclusions, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, Background and Results, Top Level Description of the Boosting Strategy, Generation of Weak Learners, Overall Algorithm, Experiments, Conclusions View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

This chapter contains sections titled: Introduction, SVM for Pattern Recognition, The Leave-one-out Procedure, Span of the Set of Support Vectors, The Bounds, Experiments, Conclusion, Appendix: Proofs View full abstract»

This chapter contains sections titled: Introduction, Leave-One-Out Support Vector Machines, Adaptive Margin SVMs, Relationship of AM-SVMs to Other SVMs, Theoretical Analysis, Experiments, Discussion View full abstract»

This chapter contains sections titled: Introduction, The SVM Variational Problem, The Dual Problem, The Generalized Comparative Kullback-Leibler Distance, Leaving-out-one and the GACV, Numerical Results, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, Gaussian Process Classification, Modeling the Noise, From Gaussian Processes to SVM, Leave-One-Out Estimator, Naive Mean Field Algorithm, Simulation Results, Conclusion View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

This chapter contains sections titled: Introduction, A Simple Geometric Problem, The Maximal Margin Perceptron, The Bayes Perceptron, The Kernel-Billiard, Numerical Tests, Conclusions, Appendix View full abstract»

This chapter contains sections titled: Introduction, Margin Distribution Bound on Generalization, An Explanation for the Soft Margin Algorithm, Related Techniques, Conclusion, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, The Basic SVM Setting, The Learning Problem, The Approach of Statistical Mechanics, Results I: General, Results II: Overfitting, Results III: Dependence on the Input Density, Discussion and Outlook, Acknowledgments View full abstract»

This chapter contains sections titled: Introduction, Tools from Functional Analysis, Convex Combinations of Parametric Families, Convex Combinations of Kernels, Multilayer Networks, Discussion, Appendix: A Remark on Traditional Weight Decay, Appendix: Proofs View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»