Single Shot Multibox Detector With Kalman Filter for Online Pedestrian Detection in Video

Pedestrian detection is a valuable and challenging problem in computer vision. To fully exploit the interframe information to improve the detector’s performance, many frameworks with high complexity for offline detection have been proposed. These methods cannot provide spontaneous responses or alerts. In this paper, we present a Kalman filter-based convolutional neural network (CNN) for online pedestrian detection in videos. First, the single shot multibox detector is implemented as the CNN detector, which incorporates the pedestrian’s aspect ratios. Fusion modules are implemented to improve the detector’s robustness for medium and far scale pedestrians. Then, bounding boxes are propagated according to the prediction from the Kalman filter. Finally, the location and confidence of the bounding boxes are refined by the Kalman filter. Our method is evaluated on two datasets with respect to both the miss rate and speed, and the results show that our method has a lower miss rate, more stable confidence, and a much higher speed.