Real-Time Single-Stage Vehicle Detector Optimized by Multi-Stage Image-Based Online Hard Example Mining

Vehicle detection is a fundamental function required for advanced driver assistance systems. Extensive research has shown that good performance can be obtained on public datasets by various state-of-the-art approaches, especially the deep learning methods. However, those methods are mostly two-stage approaches which inevitably require extensive computing resources and are hard to be deployed on an embedded computing platform with real-time computing performance. We introduce a single-stage vehicle detector which can work in real-time on NVIDIA DrivePX2 platform. The main contributions of this paper are threefold. We propose a detection scheme which includes multi-scale features and multi-anchor boxes to improve the accuracy of a single-stage detector. Secondly, a new data augmentation strategy is proposed to systematically generate a lot of vehicle training images whose appearances are randomly truncated, so our detector is trained to detect partially-seen vehicles better. Thirdly, we present a multi-stage image-based online hard example mining (MSI-OHEM) framework specifically designed for single-stage detectors. MSI-OHEM performs fine-tuning on hard examples and the ones with slightly-insufficient IOU that are considered true positives. Compared to other classical object detectors, the proposed detector achieves very competitive result in terms of average precision (AP) and computational speed. For the newly-defined vehicle class (car+bus) on VOC2007 test, our detector, using MobileNetV2, GoogLeNet, Inception-v2 and ResNet-50 as basenets, achieves 85.35%/85.62%/86.49%/87.81% AP and runs at 64/58/48/28 FPS on NVIDIA DrivePX2, respectively.