Provenance-based Explanations for Machine Learning (ML) Models

Understanding machine learning (ML) models is extremely important since it improves trust and transparency and debugging the misbehavior of the models. This paper proposes a technique for computing well-known ML explanations from the literature using provenance. Observing that widely used ML models can be expressed into a set of Datalog queries, we develop a technique that rewrites the query to capture provenance and computes explanations based on that while training the model. The preliminary evaluation shows the reasonable computational cost of our algorithm.