Motion estimation and compensation for fisheye warped video

Video captured through fisheye lens is becoming very prevalent due to applications such as surveillance, automotive driver assistance systems, and recreational sports. Virtual reality applications such as 360 degrees video also use fish-eye lens video cameras to capture the 360 degree view. The main advantage of fisheye lenses is that they increases the field of view, however they also introduce warping distortion in the captured video. Due to the warping, the motion in the video is typically non-translational and traditional block motion compensation techniques are not fully effective for such video. This paper presents a warping motion compensation technique that models the fish-eye lens distortion to efficiently code warped video. Given the fisheye lens parameters (e.g., transmitted at a video sequence level), the warping model is implicitly derived in both the encoder and decoder and no block-level warping parameters are transmitted. The proposed approach was integrated into HEVC HM-14.0 and initial results on simulated fish-eye lens distorted video show promise especially when the global motion in the video is fast.