Analysis of Discrepancy Metrics Used in Medical Image Segmentation

Evaluation of medical image segmentation methods is an important task, frequently ignored in the medical image and computer vision community. Several scalar evaluation metrics have been proposed in the literature. Nevertheless, few efforts have been made to characterize the evaluation metrics. It is well-known that metrics measure different characteristics, in such way they might vary greatly among problem domains. Therefore, some of them will be more suitable in particular situations. In this paper, we analyze the behavior and ability of 17 discrepancy metrics to retain its value under a set of changes in a confusion matrix. We also perform an analysis of the consistency among peer metrics by using Pearson's correlation. Our aim is to provide a valuable insight to select the most suitable .discrepancy metric and show their advantages and weakness