SEQ-ALIGN: Self-Supervised Image Representation Learning from Multi-Sequence MRI

Learning a good visual representation in medical imaging is crucial for the design of decision support tools and aid in the interpretation of image content. In radiology, state-of-the-art methods based on the training of neural networks can be laborious to implement due to the difficulty of collecting and annotating training data and the complex multimodal nature of the images, leading to unresolved fusion issues. In this article, we propose to consider a self-supervised learning strategy to learn discriminative image representations that can be exploited in different downstream tasks such as image similarity assessment or classification. Our contribution is based on a strategy for aligning the different available modalities (from the same examination), benefiting from their complementarity, to supervise the optimization of the neural network. The interest of this approach is evaluated in the context of multi-sequence MR imaging for the analysis of ovarian lesions through three downstream tasks. Our experiments highlight that a self-supervised training may be advantageous not only in terms of requiring fewer annotations, but also because it may yield increased performance compared to its supervised counterparts.