Probabilistic Classification Vector Machines
Huanhuan Chen; Tino, P.; Xin Yao
Neural Networks, IEEE Transactions on
Volume 20, Issue 6, June 2009 Page(s):901 - 914
Digital Object Identifier   10.1109/TNN.2009.2014161
Summary:In this paper, a sparse learning algorithm, probabilistic classification vector machines (PCVMs), is proposed. We analyze relevance vector machines (RVMs) for classification problems and observe that adopting the same prior for different classes may lead to unstable solutions. In order to tackle this problem, a signed and truncated Gaussian prior is adopted over every weight in PCVMs, where the sign of prior is determined by the class label, i.e., +1 or -1. The truncated Gaussian prior not only restricts the sign of weights but also leads to a sparse estimation of weight vectors, and thus controls the complexity of the model. In PCVMs, the kernel parameters can be optimized simultaneously within the training algorithm. The performance of PCVMs is extensively evaluated on four synthetic data sets and 13 benchmark data sets using three performance metrics, error rate (ERR), area under the curve of receiver operating characteristic (AUC), and root mean squared error (RMSE). We compare PCVMs with soft-margin support vector machines (SVM_Soft), hard-margin support vector machines (SVM_Hard), SVM with the kernel parameters optimized by PCVMs (SVM_PCVM), relevance vector machines (RVMs), and some other baseline classifiers. Through five replications of twofold cross-validation F test, i.e., 5 times 2 cross-validation F test, over single data sets and Friedman test with the corresponding post-hoc test to compare these algorithms over multiple data sets, we notice that PCVMs outperform other algorithms, including SVM_Soft, SVM_Hard, RVM, and SVM_PCVM, on most of the data sets under the three metrics, especially under AUC. Our results also reveal that the performance of SVM_PCVM is slightly better than SVM_Soft, implying that the parameter optimization algorithm in PCVMs is better than cross validation in terms of performance and computational complexity. In this paper, we also discuss the superio- rity of PCVMs' formulation using maximum a posteriori (MAP) analysis and margin analysis, which explain the empirical success of PCVMs.

» View citation and abstract

Log in by entering your IEEE Web Account Username and Password.

IEEE Communications Society members: If you subscribe to the IEEE Electronic Periodicals Package or IEEE Electronic Periodicals Package Plus, you must access your subscription at www.comsoc.org.

Check with your librarian, information professional, or system manager to determine if you need to log in. Please complete the online Technical Support Form if you need assistance.

Select the Purchase History link to access the document. You will have 5 Days after purchase to access the Full Text PDF. Please complete the online Technical Support Form if you need assistance.

• Search and access Abstract records free of charge
• Register for table of contents alerts
• Purchase Full Text PDF documents

» Learn more about subscription options or how to become an IEEE Member.