Object Remover Performance Evaluation Methods Using Classwise Object Removal Images

Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance. The performance of an object remover is quantitatively evaluated by measuring the quality of object removal results. Current works reporting quantitative performance evaluations utilize original images as references. In this letter, we show that the evaluation methods used in the previous works are not suitable for evaluating object removal results. In addition, we propose new evaluation methods tailored to gauge the performance of an object remover. The proposed methods evaluate the performance through classwise object removal results and utilize images without the target class objects as a comparison set. We confirm that the proposed methods can make judgments consistent with human evaluators and those obtained using object removal ground truth on the COCO and self-acquired datasets, respectively.