An Extended Active Learning Approach to Multiverse Analysis: Predictions of Latent Variables from Graph Theory Measures of the Human Connectome and Their Direct Replication*

Multiverse analysis has been proposed as a powerful technique to disclose the large number of degrees of freedom in data preprocessing and analysis that strongly contribute to the current replication crisis in science. However, in the field of imaging neuroscience, where multidimensional, complex and noisy data are measured, multiverse analysis may be computationally infeasible. The number of possible forking paths given by different methodological decisions and analytical choices is immense. Recently, Dafflon et al. (2022) proposed an active learning approach as an alternative to exhaustively exploring all forking paths. Here, we aimed to extend their active learning pipeline by integrating latent underlying variables which are not directly observable. The extension to latent outcomes is particularly valuable for computational psychiatry and neurocognitive psychology, where latent traits are conceptualized as common cause of a variety of observable neural and behavioral symptoms. To illustrate our approach and to test its direct replicability, we analyzed the individual organization and topology of functional brain networks of two relatively large samples from the ABCD study dataset (N = 1491) and HCP dataset (N = 833). Graph-theoretical parameters that take into account both brain-wide and region-specific network properties were used as predictors of a latent variable reflecting general cognition. Our results demonstrate the ability of the extended method to selectively explore the multiverse when predicting a latent variable. First, the low-dimensional space created with the proposed approach was able to cluster the forking paths according to their similarity. Second, the active learning approach successfully estimated the prediction performance of all pipelines in both datasets. To interactively explore the multiverse of results, we developed a Shiny app to visualize the predictive accuracy resulting from each forking path and to illustrate the similarity between pipelines created by different combinations of data processing choice. The code for active learning and the app are available at the Github repository ExtendedAL.