By Topic

Active Learning and Basis Selection for Kernel-Based Linear Models: A Bayesian Perspective

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Paisley, J. ; Dept. of Electr. & Comput. Eng., Duke Univ., Durham, NC, USA ; Xuejun Liao ; Carin, L.

We develop an active learning algorithm for kernel-based linear regression and classification. The proposed greedy algorithm employs a minimum-entropy criterion derived using a Bayesian interpretation of ridge regression. We assume access to a matrix, ? ? BBRN?N, for which the (i,j)th element is defined by the kernel function K(?i,?j) ? BBR, with the observed data ?i ? BBRd. We seek a model, M:?i? yi, where yi is a real-valued response or integer-valued label, which we do not have access to a priori. To achieve this goal, a submatrix, ?Il,Ib ? BBRn?m, is sought that corresponds to the intersection of n rows and m columns of ?, indexed by the sets Il and Ib, respectively. Typically m ? N and n ? N. We have two objectives: (i) Determine the m columns of ?, indexed by the set Ib, that are the most informative for building a linear model, M: [1 ?i,Ib]T ? yi , without any knowledge of {yi}i=1N and (ii) using active learning, sequentially determine which subset of n elements of {yi}i=1N should be acquired; both stopping values, |Ib| = m and |Il| = n, are also to be inferred from the data. These steps are taken with the goal of minimizing the uncertainty of the model parameters, x, as measured by the differential entropy of its posterior distribution. The parameter vector x ? BBRm, as well as the model bias ? ? BBR, is then learned from the resulting problem, yIl = ?Il,Ibx + ?1+?. The remaining N-n responses/labels not included in yIl can be inferred by applying x to the remaining N-n rows of ? :, Ib. We show experim- - ental results for several regression and classification problems, and compare to other active learning methods.

Published in:

Signal Processing, IEEE Transactions on  (Volume:58 ,  Issue: 5 )