A deep learning multi-capture segmentation modality for retinal OCT imaging

Advances in image processing and automatic deep learning methods have enhanced the analysis of optical coherence tomography (OCT) scans, which provide high-quality cross-sectional images of the posterior part of the eye (retina and choroid). These automatic methods support diagnosis and monitoring of ocular conditions by automatically segmenting the required tissues and quantifying the thickness of tissue layers. The performance of these automatic methods is often affected by the image quality, such as the presence of speckle noise or the variations of focus used to capture the OCT images. These changes in image quality can negatively impact the segmentation performance of the methods. In this study, OCT images of different capture modalities (i.e. various focus and denoise settings) are used to analyze how image quality factors can affect the segmentation performance of a baseline U-Net method. To deal with the various OCT image capture modalities (i.e. different image quality and focus aspects), an image-to-image translation process with CycleGAN is proposed to standardize them and to facilitate the segmentation process of the baseline method. Additionally, the use of newly proposed vision transformers (TransU-Net segmentation method) is analyzed and compared against the baseline U-Net. Results demonstrate that using this image-to-image translation process as a denoising technique for OCT images captured with an enhanced depth imaging focus modality gave the best performance when using the TransU-Net method, with a Dice coefficient of 0.99, improving the segmentation performance of the baseline U-Net method. The proposed technique provides a viable alternative to OCT instrument-agnostic segmentation.