Regularizing Pattern Recognition with Conditional Probability Estimates

Recent contributions in non-parametric statistical pattern recognition have investigated augmenting the task with information about the conditional probability distribution P(Y|X) away from the 0.5 level set, i.e. the decision boundary. Many hypothesis spaces satisfy generous smoothness criteria, so the behavior of a function away from the decision boundary can serve as a regularizer for its behavior at the decision boundary. This paper proposes a paradigm to capture observable information about the conditional distribution and describe a learning formulation that can take advantage of it. Finally, it investigates why conditional probability can be an effective regularizer for inseparable pattern recognition problems.