Semi-supervised detection of collective anomalies with an application in high energy particle physics

We study a novel type of a semi-supervised anomaly detection problem where the anomalies occur collectively among a background of normal data. Such problem arises in experimental high energy physics when one is trying to discover deviations from known Standard Model physics. We solve the problem by first fitting a mixture of Gaussians to a labeled background sample. We then fit a mixture of this background model and a number of additional Gaussians to an unlabeled sample containing both background and anomalies. This way we not only detect but also perform pattern recognition of anomalies. Such mixture model allows us to perform classification of anomalies vs. background, estimate the proportion of anomalies in the sample and study the statistical significance of the anomalous contribution. We first verify the performance of the method using artificial data and then demonstrate its real-life applicability using a data set related to the search of the Higgs boson at the Tevatron collider.