An algorithm for evaluating inter- and intra-observer variability of boundary delineation providing local information about deviation for anisotropic image data

Quality assessment of segmentation algorithms is not a simple task. A comparison between a manual outline of an expert and segmentation results, widely used in most cases, appears to be insufficient. Taking into account the lack of reproducibility of the outlines made by experts, it is necessary to thoroughly analyze both inter- and intra-observer variability. To meet these requirements, the article proposes a new evaluation method for inter- and intra-observer variability of boundary delineation. The algorithm is based on three-dimensional signed Euclidean Distance Transformation. It allows to provide local information (for each point of the outline) about manual outlines deviation. The algorithm manages anisotropic image data such as magnetic resonance (MR) or computed tomography (CT) images. It can also be used for complex, non-convex structures. In addition, a method of selection the most reliable outline is proposed. The effectiveness of the proposed approach is assessed by examining the boundary deviation of anatomical structures in CT data. Finally, a quantitative analysis of the manual outlines in various schemes for a kidney and a femur are presented.