Accelerating Object Detection Models Inference within Deep Learning Workbench

Recent breakthroughs in the Deep Learning field have resulted in neural models surpassing human intelligence in a variety of specialized tasks. Modern neural networks have proven their effectiveness in object detection and segmentation applications. However, real-time performance is becoming more challenging for them, especially in cases when a vehicle's autonomy depends on its ability to perceive the surrounding environment within moments. As a result, model acceleration has gained extreme significance in the Deep Learning community. In this paper, we propose a simple yet effective lossy optimization that constitutes a part of the model optimization pipeline. This optimization allows to increase the model throughput by reducing the model input at the expense of an acceptable accuracy drop. For example, we demonstrate that by applying optimization pipeline including input reduction by 40%, 8-bit integer quantization and optimal inference configuration, RetinaNet throughput rocketing in 45 times with an accuracy drop of only 0.01%. We evaluated our acceleration technique on a range of object detection models and used the Intel® Distribution of OpenVINO™ toolkit Deep Learning Workbench as the optimization platform since it provides application engineers with all the required functionality to optimize and deploy the models.