Electronic image stabilization algorithm based on smoothing 3D rotation matrix

In the camera shooting process, video jitter caused by camera motion results to instability of the image information and the fuzzy which is not conducive for people to observe, and reduces the compression ratio. To solve this problem, we propose an electronic image stabilization algorithm using 3D rotational motion model. 3D camera motion is calculated from image sequence. The image can be mapped on the virtual sphere where the optical center of the camera is in the sphere center. We match the feature points using optical flow method, and then calculate rotation matrix using Spherical SFM algorithm between previous and current frame image. The rotation matrix contains the rotation and translation of the camera. The motion trajectory of the camera can be expressed by the 3D rotation matrix sequence. We adopt the manifold structure of the rotation matrices sequence, and we smooth the 3D rotation matrix sequences based on geodesic distance on a non-linear manifold. We formulate the video stabilization problem as objective function of smoothed rotation, and use steepest gradient descent and Armijo rules to estimate the smoothed rotation. Finally, the current frame is compensated by calculating the difference between the smoothed path and the original path to suppress the influence of the camera translation on the obtained rotation matrix. Experimental results demonstrated that the proposed algorithm does can effectively reduce random jitter without special hardware, which achieved the effect of image stabilization. Compared to the image stabilization algorithm based on 2D model, the proposed image stabilization algorithm greatly reduced the jitter and image distortion.