Integrating ChatGPT-4: A Novel XAI Interface for Enhanced Clinician Understanding of MRI Image Segmentation Results

The poor explainability or interpretability of deep learning makes it difficult to introduce these technologies into the hospital workflow. Explainable Artificial Intelligence (XAI) addresses this problem by providing interpretable explanations of AI decisions. We present a system that employs large language models (LLM) with an image-to-text approach, converting AI results of medical segmentation into understandable natural language descriptions. This study first explores how ChatGPT-4 can interpret and describe Magnetic Resonance Image (MRI) brain scans and their segmentations both without previous input and with specific details. In particular, the effectiveness of 'prompt engineering' in improving the accuracy and relevance of language-based artificial intelligence model responses was investigated. Through a series of controlled experiments, we aim to demonstrate how the accurate and strategic formulation of prompts can significantly influence the behaviour of even ChatGPT-4, highlighting its potential but above all the care required when using it. Simultaneously, a user-friendly interface was developed to facilitate interactions between radiologists and the AI segmentation model via ChatGPT-4 generated explanations. This integrated approach aims to improve the reliability, transparency, and usability of AI-supported medical image analysis, potentially leading to more informed clinical decisions and better patient outcomes.