Retinal Vascular System Segmentation based on Non-Linear Map-Based Estimation of Joint MGRF Model

We introduce a novel, fully automated, unsupervised segmentation technique for extracting the vascular system from retinal Optical Coherence Tomography Angiography (OCTA) images, a crucial component in computer-aided diagnostic systems for retinal disease detection. Our approach integrates a two-level joint Markov-Gibbs Random Field (MGRF) framework, featuring a new probabilistic model—the Linear Combination of Discrete Gaussian (LCDG). The first level addresses OCTA image appearance and their spatially smoothed counterparts through LCDG, with parameters refined by an enhanced Expectation Maximization (EM) algorithm. The second level captures vascular and retinal tissue maps via MGRF, utilizing parameters analytically derived from images. A novel application of modified self-organizing maps, acting as a MAP-based optimizer, enables unsupervised, nonlinear optimization-driven segmentation, marking a departure from traditional methods. Evaluated on a dataset of 204 subjects, our framework achieves significant metrics: a Dice similarity coefficient of 0.90±0.04, a 95-percentile bidirectional Hausdorff distance of 0.79 ± 0.28, and an accuracy of 0.86 ± 0.06, underscoring its effectiveness.