RevelioNN: Retrospective Extraction of Visual and Logical Insights for Ontology-Based Interpretation of Neural Networks

The need for AI explainability, which involves helping humans understand why an AI algorithm arrived at a particular decision, is crucial in numerous critical applications. Although deep neural networks play a significant role in modern AI, they inherently lack transparency. Consequently, various approaches have been suggested to clarify their decision-making processes to human users. One promising category of such approaches involves ontology-based methods. These methods have the potential to generate explanations using concepts from an ontology that are familiar to domain experts and the logical connections between these concepts. Specifically, post-hoc ontology-based explanations typically rely on concept extraction, which establishes a link between the internal representations formed by the neural network's inner layers and the domain concepts outlined in the ontology. This paper introduces the RevelioNN library, which comprises post-hoc algorithms designed to explain predictions made by deep convolutional neural networks in binary classification tasks, with a focus on leveraging ontologies. The library incorporates cutting-edge concept extraction techniques centered around constructing mapping networks. Furthermore, it provides the capability to form both logical and visual explanations for the predictions of convolutional neural networks by utilizing ontology concepts derived from their internal representations. An essential benefit of this library is its adaptability to interpret predictions from any pre-trained convolutional network implemented using the PyTorch framework. The code is licensed under the BSD-3-Clause license and is freely available at github.com/cais-lab/revelionn.