By Topic

Advances in Large-Margin Classifiers

Cover Image Copyright Year: 2000
Author(s): Smola, A.; Bartlett, P.; Schölkopf, B.; Schuurmans, D.
Publisher: MIT Press
Content Type : Books & eBooks
Topics: Computing & Processing
  • Print

Abstract

The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba.

  •   Click to expandTable of Contents

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Front Matter

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): i - ix
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Half Title, Published by Morgan-Kaufmann, Title, Copyright, Contents, Series Forword, Preface View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Introduction to Large Margin Classifiers

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 1 - 29
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: A Simple Classification Problem, Theory, Support Vector Machines, Boosting, Empirical Results, Implementations, and Further Developments, Notations View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Roadmap

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 31 - 35
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Support Vector Machines, Kernel Machines, Boosting, Leave-One-Out Methods, Beyond the Margin View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Introduction to Large Margin Classifiers

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 37
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Dynamic Alignment Kernels

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 39 - 50
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction: Linear Methods using Kernel function, Applying Linear Methods to Structured Objects, Conditional Symmetric Independence Kernels, Pair Hidden Markov Models, Conditionally Symmetrically Independent PHMMs, Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Natural Regularization from Generative Models

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 51 - 60
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Natural Kernels, The Natural Regularization Operator, The Feature Map of Natural Kernel, Experiments, Discussion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Probabilities for SV Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 61 - 73
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Fitting a Sigmoid After the SVM, Empirical Tests, Conclusions, Appendix: Pseudo-code for the Sigmoid Training View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Maximal Margin Perception

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 75 - 113
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Basic Approximation Steps, Basic Algorithms, Kernel Machine Extension, Soft Margin Extension, Experimental Results, Discussion, Conclusions, Appendix: Details of comparison against six other methods for iterative generation of support vector machines View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Large Margin Bank Boundaries for Ordinal Regression

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 115 - 132
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Classical Models for Ordinal Regression, A Risk Formulation for Ordinal Regression, An Algorithm for Ordinal Regression, Experimental Results, Discussion and Conclusion, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Kernel Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 133
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Generalized Support Vector Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 135 - 146
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, GSVM: The General Support Vector Machine, Quadratic Programming Support Vector Machines, Linear Programming Support Vector Machines, A Simple Illustrative Example, Conclusion, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Linear Discriminant and Support Vector Classifiers

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 147 - 169
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, What is a Linear Discriminant?, Formulation of the Linear Discriminant Training Problem, Training Algorithms, Which Linear Discriminant?, Conclusion, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Regularization Networks and Support Vector Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 171 - 203
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Overview of Statistical Learning Theory, Regularization Networks, Support Vector Machines, SRM for RNs and SVMs, A Bayesian Interpretation of Regularization and SRM?, Connections Between SVMs and Sparse Approximation Techniques, Remarks, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Boosting

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 205
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Robust Ensemble Learning

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 207 - 220
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Boosting and the Linear Programming Solution, υ-Algorithms, Experiments, Conclusion, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Functional Gradient Techniques for Combining Hypotheses

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 221 - 246
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Optimizing Cost Functions of the Margin, A Gradient Descent View of Voting Methods, Theoretically Motivated Cost Functions, Convergence Results, Experiments, Conclusions, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Towards a Strategy for Boosting Regressors

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 247 - 258
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Background and Results, Top Level Description of the Boosting Strategy, Generation of Weak Learners, Overall Algorithm, Experiments, Conclusions View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Leave-One-Out Methods

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 259
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Bounds on Error Expectation for SVM

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 261 - 280
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, SVM for Pattern Recognition, The Leave-one-out Procedure, Span of the Set of Support Vectors, The Bounds, Experiments, Conclusion, Appendix: Proofs View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Adaptive Margin Support Vector Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 281 - 295
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Leave-One-Out Support Vector Machines, Adaptive Margin SVMs, Relationship of AM-SVMs to Other SVMs, Theoretical Analysis, Experiments, Discussion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      GACV for Support Vector Machines

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 297 - 309
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, The SVM Variational Problem, The Dual Problem, The Generalized Comparative Kullback-Leibler Distance, Leaving-out-one and the GACV, Numerical Results, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Gaussian Processes and SVM: Mean Field and Leave-One-Out

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 311 - 326
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Gaussian Process Classification, Modeling the Noise, From Gaussian Processes to SVM, Leave-One-Out Estimator, Naive Mean Field Algorithm, Simulation Results, Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Beyond the Margin

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 327
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Computing the Bayes Kernel Classifier

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 329 - 347
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, A Simple Geometric Problem, The Maximal Margin Perceptron, The Bayes Perceptron, The Kernel-Billiard, Numerical Tests, Conclusions, Appendix View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Margin Distribution and Soft Margin

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 349 - 358
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Margin Distribution Bound on Generalization, An Explanation for the Soft Margin Algorithm, Related Techniques, Conclusion, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Support Vectors and Statistical Mechanics

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 359 - 367
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, The Basic SVM Setting, The Learning Problem, The Approach of Statistical Mechanics, Results I: General, Results II: Overfitting, Results III: Dependence on the Input Density, Discussion and Outlook, Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Entropy Numbers for Convex Combinations and MLPs

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 369 - 387
      Copyright Year: 2000

      MIT Press eBook Chapters

      This chapter contains sections titled: Introduction, Tools from Functional Analysis, Convex Combinations of Parametric Families, Convex Combinations of Kernels, Multilayer Networks, Discussion, Appendix: A Remark on Traditional Weight Decay, Appendix: Proofs View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      References

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 389 - 407
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Index

      Smola, A. ; Bartlett, P. ; Schölkopf, B. ; Schuurmans, D.
      Advances in Large-Margin Classifiers

      Page(s): 409 - 412
      Copyright Year: 2000

      MIT Press eBook Chapters

      The concept of large margins is a unifying principle for the analysis of many different approaches to the classification of data from examples, including boosting, mathematical programming, neural networks, and support vector machines. The fact that it is the margin, or confidence level, of a classification--that is, a scale parameter--rather than a raw training error that matters has become a key tool for dealing with classifiers. This book shows how this idea applies to both the theoretical analysis and the design of algorithms.The book provides an overview of recent developments in large margin classifiers, examines connections with other methods (e.g., Bayesian inference), and identifies strengths and weaknesses of the method, as well as directions for future research. Among the contributors are Manfred Opper, Vladimir Vapnik, and Grace Wahba. View full abstract»