Fusing Faster R-CNN and Background Subtraction Based on the Mixture of Gaussians Model

Convolutional Neural Networks (CNNs) have made important progress in object detection. Using the internal feature extractor, CNNs perform a refined spatial analysis of the images for acquiring context-dependent information. To improve the object detection performed for streaming data, this paper proposes a fusion between Faster R-CNN and Background Subtraction based on the Mixture of Gaussians model (BSMOG). Faster R-CNN is a widely recommended object image detector using CNNs both for region proposal and classification. The suggested method expands the spatial analysis with the temporal stochastic information acquired by BSMOG. The interaction between Faster R-CNN and BSMOG is cooperative. During background updating, the reliable CNN-based detections define areas where matched Gaussians will be disfavored and replacing Gaussians will be unadvised. Also, BSMOG refines the confidence scores assigned to CNN-based detections. The method is exemplified for a video surveillance application. The experimental results show improved pixel-wise segmentation of the dynamic objects and a reduced number of false-positive detections for the category of objects recognized by Faster RCNN.