Metrics for Evaluating the Continuity Capabilities of Object Detection Systems

This paper proposes two novel metrics for evaluating the consistency of object detection systems on continuous video sequences. Its aim is to help differentiate between two such systems that have similar results if evaluated with traditional metrics, so two recomputed values for precision and recall are described. They are based on a continuity score, which received its name based on the fact that it favors detections that cover more grouped frames, rather than sparse sets of detections, for true positive detections. The opposite can be said about false positives, where fragmented ones are preferred. It relies on giving bonuses to detections that appear in consecutive frames, it allows a number of interruptions in such a series, before penalties are applied, and it is designed to be highly configurable. A procedure for verifying whether two different detections represent the same object is also described. Having this correlation method, the metrics can be used for simple object detection systems, but it can also be modified such that they are used with systems that assign ids to detected objects.