Class Specific Interpretability in CNN Using Causal Analysis

A singular problem that mars the wide applicability of machine learning (ML) models is the lack of generalizability and interpretability. The ML community is increasingly working on bridging this gap. Prominent among them are methods that study causal significance of features, with techniques such as Average Causal Effect (ACE). In this paper, our objective is to utilize the causal analysis framework to measure the significance level of the features in binary classification task. Towards this, we propose a novel ACE-based metric called “Absolute area under ACE (A-ACE)” which computes the area of the absolute value of the ACE across different permissible levels of intervention. The performance of the proposed metric is illustrated on (i) ILSVRC (Imagenet) dataset and (ii) MNIST data set $(\sim 42000$ images) by considering pair-wise binary classification problem. Encouraging results have been observed on these two datasets. The computed metric values are found to be higher - peak performance of 10x higher than other for ILSVRC dataset and 50% higher than others for MNIST dataset - at precisely those locations that human intuition would mark as distinguishing regions. The method helps to capture the quantifiable metric which represents the distinction between the classes learnt by the model. This metric aids in visual explanation of the model’s prediction and thus, makes the model more trustworthy.

1. Introduction

Machine learning models have enormous potential in solving sophisticated problems in variety of domains be it object detection [9], autonomous driving [2], DNA sequence generation [17], speech recognition [24] and language processing [6]. These models are capable of learning complex representations. However, human interpretability of these models has been very challenging due to the Black Box nature of it, thus making these models untrustworthy to be used in critical scenarios like healthcare applications as in disease discovery and diagnosis [11], drug discovery [10], autonomous driving [2] etc. Trusting these models in critical application requires us to be cognizant about the pertinent features and their effectiveness, that the model has learnt. These models would have been validated only on the perceived scenario and lacks to accommodate the unseen scenarios - drifting distribution. Hence there is a high need to strengthen the modelling methods for meaningful predictions from such models. The emerging field in machine learning, Explainable AI [7], aims to address this problem of discovering the Black Box decisions in deep neural networks.