Deep Learning for Volumetric Segmentation in Spatio-Temporal Data: Application to Segmentation of Prostate in DCE-MRI

Segmentation of the prostate in MR images is an essential step that underpins the success of subsequent analysis methods, such as cancer lesion detection inside the tumour and registration between different modalities. This work focuses on leveraging deep learning for analysis of longitudinal volumetric datasets, particularly for the task of segmentation, and presents proof-of-concept for segmentation of the prostate in 3D+T DCE-MRI sequences. A two-stream processing pipeline is proposed for this task, comprising a spatial stream modelled using a volumetric fully convolutional network and a temporal stream modeled using recurrent neural networks with Long-Short-term Memory (LSTM) units. The predictions of the two streams are fused using deep neural networks. The proposed method has been validated on a public benchmark dataset of 17 patients, each with 40 temporal volumes. When averaged over three experiments, a highly competitive Dice overlap score of 0.8688 and sensitivity of 0.8694 were achieved. As a spatiotemporal segmentation method, it can easily migrate to other datasets.