Local Models for Expectation-Driven Subgroup Discovery

Subgroup discovery (also known as Pattern Mining or Supervised Descriptive Rule Discovery) searches for descriptions of subsets in a dataset that differ from the total population with respect to a given target concept. In this paper we argue that in the traditional approach potentially interesting complex patterns with an unexpected relative increase of the target share remain undiscovered while on the other hand less surprising patterns are returned. Therefore, we present a generalized approach on subgroup discovery, in which the target share in the subgroup is not compared to the target share in the total population, but to the expectations a user has given the knowledge of more general (simpler) patterns. We claim that the resulting complex patterns are more interesting for the user and are less biased towards simpler patterns with a positive influence on the target concept. In order to estimate these expectations we utilize local models, i.e., fragments of Bayesian Networks. The proposed approach is evaluated using data from the UCI repository as well as on two totally different real world applications that investigate university student drop-out rates and identify spammers in a social book marking system.