Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision

Across most pedestrian detection datasets, it is typically assumed that pedestrians will be standing upright with respect to the image coordinate system. This assumption is not always valid for many vision-equipped mobile platforms, such as mobile phones, UAVs, or construction vehicles on rugged terrain. In these situations, the motion of the camera can cause images of pedestrians to be captured at extreme angles. This can lead to inferior pedestrian detection performance when using standard pedestrian detectors. To address this issue, we propose a Rotational Rectification Network (R2N) that can be inserted into any CNNbased pedestrian (or object) detector to adapt it to significant changes in camera rotation. The rotational rectification network uses a 2D rotation estimation module that passes rotational information to a spatial transformer network [12] to undistort image features. To enable robust rotation estimation, we propose a Global Polar Pooling (GPPooling) operator to capture rotational shifts in convolutional features. Through our experiments, we show how our rotational rectification network can be used to improve the performance of state-of-the-art pedestrian detectors under heavy image rotation by up to 45%.