Ex(o)plain: Subgroup-Level Analysis of Exoplanet Atmospheric Parameters

Deep learning has been shown to be a valuable tool in astrophysics. In the field of exoplanetary science, deep learning-based approaches are being used extensively to automate the characterization of exoplanet atmospheres, reducing computational costs when compared to conventional methods. However, many atmospheric reconstruction models lack interpretability. We introduce Ex(o)plain, a model-agnostic framework to identify and describe the most meaningful traits that characterize exoplanet atmospheres. Our approach categorizes exoplanets into subgroups based on combinations of various metadata, such as surface gravity, planet radius, and star temperature. We analyze these subgroups to identify those for which the deep learning model performs better or worse than average. This provides useful insights into what is being effectively learned by these black box models and where they still struggle. We explore a practical case based on the synthetic observations generated for the upcoming Ariel mission. Experimental results demonstrate the effectiveness of adopting explanation techniques in revealing meaningful variations in reconstruction quality between individual models and their aggregated ensemble. We additionally show that ensemble approaches significantly outperform single learners. We leverage the same subgroup-based exploration techniques to assess the situations that are most beneficial for the ensemble. Our work provides a more nuanced understanding of deep learning results for exoplanet characterization, aiming to delineate feasible accuracy limits and enable more informed evaluations of these techniques’ atmospheric reconstruction capabilities.