When Green Screen Meets Panoramic Videos: An Interesting Video Combination Framework for Virtual Studio and Cellphone Applications

In this paper, an interesting video combination framework was established based on Unreal Engine 4. By substituting the green screen image with panoramic videos, our framework could produce lifelike synthetic videos in which the foreground is the portrait captured by real-time TV cameras whereas the background is a pseudo scenario generated by previously captured panoramic videos. The pseudo background updates itself from the panoramic video according to the pose information of the TV camera so that the portrait looks as if immersed into the pseudo scenario rather than the green screen. Our proposed framework could be applied not only in virtual studios but also in cellphone applications, e.g., given a previously captured panoramic video of a schoolyard, users could shoot a video by their front cellphone camera before a green screen. The output is a synthetic video which seems as if the users are in the schoolyard, as the users rotate their camera, the schoolyard background also moves.