Emergent Symbolic Language Based Deep Medical Image Classification

Modern deep learning systems for medical image classification have demonstrated exceptional capabilities for distinguishing between image based medical categories. However, they are severely hindered by their inability to explain the reasoning behind their decision making. This is partly due to the uninterpretable continuous latent representations of neural networks. Emergent languages (EL) have recently been shown to enhance the capabilities of neural networks by equipping them with symbolic representations in the framework of referential games. Symbolic representations are one of the cornerstones of highly explainable good old fashioned AI (GOFAI) systems. In this work, we demonstrate for the first time, the emergence of deep symbolic representations of emergent language in the framework of image classification. We show that EL based classification models can perform as well as, if not better than state of the art deep learning models. We demonstrate the EL classification framework on immune cell marker based cell classification and chest X-ray classification using the CheXpert dataset. Code is available at https://github.com/AriChow/EL.