Sparse PLS hyper-parameters optimisation for investigating brain-behaviour relationships

Unsupervised learning approaches, such as Partial Least Squares, can be used to investigate relationships between multiple sources of data, such as neuroimaging and behavioural data. In cases of high-dimensional datasets with limited number of examples (e.g. neuroimaging data) there is a need for regularisation to enable the solution of the ill-posed problem and prevent overfitting. Different approaches have been proposed to optimise the regularisation parameters in unsupervised models, however, so far, there has been no comparison between the different approaches using the same data. In this work, two optimisation frameworks (i.e. a permutation and a train/test framework) were compared using sparse PLS to investigate associations between brain connectivity and behaviour data. Both frameworks were able to identify at least one brain-behaviour associative effect. A second brain-behaviour effect was only found using the train/test framework. More importantly, the results show that the multivariate associative effects found with the train/test framework generalise better to new data, suggesting that results based on the permutation framework should be carefully interpreted.