Explaining Classification by Finding Response-Related Subgroups in Data

A method for explaining results of a regression based classifier is proposed. The data is clustered using a metric extracted from the classifier. This way, clusters found are related to classifier predictions, and each cluster can be considered a possible explanation for classification result. The clusters are described by simple rules, meant to be easy for a human to understand. The key points of the work are presenting a modular framework for explaining the classification, and studying and comparing two different approaches for extracting a metric from a classifier model.