Mixtures of Conditional Random Fields for Improved Structured Output Prediction

The conditional random field (CRF) is a successful probabilistic model for structured output prediction problems. In this brief, we consider to enlarge the representational capacity of CRF via mixture modeling. The motivation is that a single CRF can perform well if the data conform to the statistical dependence assumption imposed by the CRF model structure, whereas it may potentially fail to model the data that come from multiple different sources or domains. For the conventional conditional likelihood objective, we derive the expectation-maximization algorithm in conjunction with the direct gradient ascent method for learning a CRF mixture with sequence or image-structured data. In addition, we provide alternative mixture learning algorithms that aim to maximize either the classification margin or the sitewise conditional likelihood, which were previously shown to outperform the conventional estimator for single CRF models in a variety of situations. We demonstrate the improved prediction accuracy of the proposed mixture learning algorithms on several important sequence labeling problems.