BMPRE: An Error measure for evaluating disparity maps

The percentage of Bad Matched Pixels (BMP) is widely used for evaluating disparity maps. It counts the number of differences between estimated disparity values and ground-truth values that exceed a threshold. A small error is counted in the same way that a large one by computing this measure. Moreover, the BMP ignores the inverse relation between depth and disparity. Although, the percentages of BMP are equal, those disparity maps may produce different 3D reconstructions. The Mean Relative Error (MRE) is calculated as an average of ratios of error magnitudes against true disparity values. It considers the inverse relation between depth and disparity. However, using the MRE, every deviation from ground-truth is considered as an error, regardless the application domain for which the disparity map was estimated. In this paper, an error measure devised for evaluating disparity maps is introduced. The introduced measure combines the advantages of the BMP and the MRE. It allows a user to declare what an estimation error is - using a threshold - and considers both, the error magnitude and the inverse relation between depth and disparity. The proposed measures is termed BMPRE and it determines estimation errors in a widely adopted way by the community and, at the same time, offers not only information, but flexibility, about the impact of those errors in the context of a 3D recovery process. Comparative analysis and experimental evaluation show that the BMPRE allows a fair evaluation of disparity maps, which impacts on a quantitative comparison of stereo correspondence algorithms.