Global measures of coherence for edge detector evaluation

We propose a class of benchmarks for edge detector evaluation that require no ground truth. Each benchmark consists of a large number of images of a carefully designed scene for which we enforce a constraint on the edges, for example, that they are co-linear. We sample the space of edge appearances as densely as possible by capturing the images under widely varying imaging conditions. Not only do we change the viewing geometry and the illumination direction, but we also vary the camera parameters and the physical properties of the objects in the scene. We show that the degrees to which the constraints hold in the output edge-maps can be used as highly discriminating measures of edge detector performance. The code, images, and results which form our benchmarks are all available from the website http://www.cs.columbia.edu/CAVE/. The code and images enable a user to compare any new detector against several previous ones with minimal effort.