XAG: Enhancing Transparency by Integrating Explainable Artificial Intelligence with Large Language Models

Artificial intelligence (AI) and machine learning have recently become central to decision-making. However, decisions made by AI are often complex and not easily understood. Inferences derived from these black-box models require an explanation, highlighting the importance of explainable AI (XAI). Approximate inverse model explanations (AIME), as a notable method in this domain, derives the approximate inverse operators of black-box models and has been explored extensively for XAI. This method facilitates easily interpretable explanations of model behaviors and estimations while maintaining low computational complexity. However, interpreting these results can be challenging for end users, particularly with complex features. This study introduces explainable-augmented generation (XAG), which combines the strengths of XAI with advanced language models to make AI decisions clearer. XAG uses AIME to explain the decision-making of AI systems and utilizes large language models (LLMs) to translate technical details into easy-to-understand explanations. Well-crafted prompts allow integrating the results of XAI methodologies such as AIME with LLMs, thereby enabling XAG to identify the features that contribute the most to decision-making and interpret the significance of feature importance. This approach enables a nuanced understanding of the high or low importance assigned to certain features within the decision-making process of the model. By merging the robust capabilities of AIME to generate interpretable model insights with the narrative power of LLMs, XAG offers a pioneering approach to demystifying AI operations for non-expert users.