Foreground rejection for parallax removal in video sequence stitching

In image stitching, parallax is key challenge that leads to inaccurate registration and ghosting effect of objects in the result. A novel foreground rejection method is proposed in this paper to remove parallax in video sequence. Firstly, the global motion is estimated between two frames using SIFT feature matching. The logical OR operators are then applied to the pixels that have high displaced frame difference for foreground extraction. There are two sets of foreground calculated from the near frames and far frames. Voting scheme is applied in the way that only the near-frame foreground appears inside the area of far-frame foreground that is considered as the actual foreground. The extracted foreground at this stage is mostly the edges of objects. Therefore, the foreground rejection method proposed in this paper aims to provide better motion vector estimation and ghosting effect reduction. The removal of the major parts of the high difference foreground is sufficient; hence the simple two-stage rendering is applied. The implemented results indicate that more accurate registration is achieved by including our proposed foreground rejection method. Moreover, the background of the stitched video sequence does not have the visible ghosting effect.